Home | LLMSC

The 3^rd International Workshop on Large Language Model Supply Chain Analysis
(LLMSC 2027)

Co-located with ICSE'27
📅 Tuesday, 27 April 2027 · 📍 Dublin, Ireland

🌟 About

Welcome to the 3rd International Workshop on Large Language Model Supply Chain Analysis (LLMSC)!

Large Language Models (LLMs) are no longer delivered as standalone models. They are increasingly embedded into agentic software systems that combine foundation models, datasets, prompts, tools, plugins, MCP servers, skills, harnesses, and multi-agent workflows. These components are developed, shared, installed, composed, updated, and governed by different stakeholders across open-source, industrial, and platform ecosystems. Together, they form an emerging LLM and agentic supply chain.

Workshop Structure

This shift enables increasingly autonomous agentic systems, but also introduces risks that traditional software supply chain analysis cannot fully capture. The emerging risks in this ecosystem may span heterogeneous artifacts, platforms, and lifecycle stages, making the LLM and agentic supply chain highly complex, dynamic, and difficult to analyze. Addressing these challenges requires new methods, tools, benchmarks, and governance frameworks for understanding and securing agentic software systems.

LLMSC aims to bring together researchers, practitioners, and policymakers from software engineering, security, AI, systems, and policy to discuss the analysis, security, reliability, and governance of the LLM and agentic supply chain. We welcome work on ecosystem measurement, dependency and provenance analysis, model and data governance, agent harness engineering, MCP and skill security, vulnerability detection, malicious component analysis, compliance, risk assessment, and responsible deployment practices, with the broader goal of building a more transparent, trustworthy, resilient, and sustainable agentic supply chain ecosystem.

🎉 NEW: Call for Skills!

🤖 LLMSC 2027 now welcomes Skill Papers! Share your reusable Agent Skill through a short paper of up to 2 pages + a verifiable Skill Artifact. 🔬 Present your research idea, demonstrate preliminary results, and let others run your contribution. 🚀 Explore the Skill Paper requirements ↓

🏆 Journal Invitation

Selected outstanding papers from LLMSC 2027 will be invited to submit revised and substantially extended versions for consideration by ASE Journal, subject to the journal's standard peer-review process.

📢 Call for Papers

🧭 Topics of Interest

Topics of interest include, but are not limited to:

🌐 Ecosystem Modeling and Empirical Analysis
- Modeling, measurement, and evolution analysis of LLM and agentic software ecosystems.
- Dependency, provenance, and reuse analysis across models, datasets, prompts, tools, agents, and applications.
- Empirical studies of model hubs, app stores, registries, repositories, and open-source LLM ecosystems.
🤖 Toolchains and Agentic Systems
- Engineering practices for LLM applications, RAG systems, AI coding tools, and agentic workflows.
- Design, testing, and evaluation of web agents, coding agents, workflow agents, and multi-agent systems.
- Integration of LLMs with APIs, plugins, vector databases, memory stores, enterprise systems, and external tools.
⚙️ Toolchains, Infrastructure, and Operations
- Analysis and benchmarking of LLM frameworks, SDKs, orchestration platforms, and deployment pipelines.
- Testing, monitoring, observability, and reliability assessment of LLM-based and agentic software systems.
- Runtime behavior analysis, action trace auditing, failure diagnosis, scalability, and production readiness.
🛡️ Security, Privacy, and Governance
- Security and privacy analysis of LLM supply chains, agent frameworks, tools, plugins, protocols, and registries.
- Threat modeling and vulnerability detection for prompt injection, tool poisoning, memory poisoning, unsafe tool use, and data leakage.
- Risk assessment, vulnerability propagation, license compliance, responsible disclosure, standardization, and governance.

📝 Submission Guidelines

We welcome the following three types of submissions:

🛠️ Skill Papers (up to 2 pages including references) — NEW: Reusable Agent Skills addressing an LLMSC-related research problem, accompanied by preliminary results and a verifiable Skill Artifact. See additional requirements for Skill Papers.
💡 Position Papers (1–4 pages including references): Well-argued positions or work in progress.
🔬 Research Papers (4–8 pages including references): Technical research, experience reports, empirical studies, etc.

📋 General Requirements

✨ Originality: All submissions must be original and not under review elsewhere. Workshop organizers and their current directly supervised students may not submit refereed papers to the workshop.
📄 Submission Format: Submissions must be written in English and submitted in PDF format. Papers must not exceed the page limits listed above and must follow the official IEEE Conference Proceedings Formatting Guidelines. LaTeX users should use \documentclass[10pt,conference]{IEEEtran}.
📤 Submission Site: Papers must be submitted via the submission site.
📚 Publication Date: The official publication date of the workshop proceedings is the date the proceedings are made available by IEEE. This date may be up to two weeks prior to the first day of ICSE 2027. The official publication date affects the deadline for any patent filings related to published work.
🎤 Registration and Presentation: To be included in the Workshop Proceedings, each accepted paper must be presented at the workshop, and at least one of its authors must register for the workshop. If no author has registered for an accepted paper within two weeks after the Early Registration Deadline, ICSE 2027 reserves the right to remove the paper from the Workshop Proceedings. If no author or designated presenter attends the workshop to present the paper, the paper will be considered a no-show.
⚖️ Submission Policies: By submitting to LLMSC 2027, authors acknowledge that they conform to the authorship policy of IEEE, the submission policy of IEEE, the authorship policy of ACM, and the ACM Publications Policy on Research Involving Human Participants and Subjects. Other detailed submission policies and formatting guidelines are aligned with the ICSE 2027 Research Track submission process.

🛠️ Additional Requirements for Skill Papers

Skill Papers explore a lightweight, executable, and reproducible form of research contribution. A Skill may consist of structured instructions, scripts, configurations, reference materials, or tool interfaces that an agent can load and use to perform a specific task. We welcome Skills for existing agent platforms as well as platform-independent implementations.

Each submission must include:

📄 Short Paper: Describe the research problem, skill design, preliminary results, and artifact access and verification instructions. The paper must not exceed 2 pages, including references.
📦 Skill Artifact: Provide the instructions, scripts, configurations, or other resources needed to use the Skill, together with environment requirements, installation and invocation instructions, and tasks, data, or scripts for reproducing the preliminary results. Clearly specify any model, framework, or environment dependencies.
🔬 Preliminary Results: Provide credible evidence of the Skill’s value, such as improved task success, new capabilities, reduced cost or manual effort, or safer and more reliable behavior.
🔒 Anonymization: Both the paper and artifact must be anonymized, including repositories, documentation, file paths, commit history, and metadata. Artifacts may be provided through anonymous repositories, anonymous archives, or other anonymous access mechanisms. For more details, please refer to this blog.

📌 Important!!

Skill Papers use the same submission system, deadlines, formatting requirements, and registration and presentation policies as the other paper types. Accepted Skill Papers will be presented and demonstrated at the workshop, with a public or persistently accessible artifact and reproduction materials.

🔎 Review Process

👥 Double-blind Review: All submissions will receive at least three reviews under a double-blind process.
🔒 Anonymous Submissions: Authors must anonymize submissions, supplementary materials, and metadata.
📊 Evaluation Criteria: All the submissions will be evaluated based on originality, relevance, technical soundness, and clarity of presentation.
🛠️ Skill Artifact Verification: Organizers will pre-execute submitted skills. Reviewers will assess the paper, artifact, and execution records, and may rerun key tasks as needed.

🗓️ Important Dates

All submission deadlines are Anywhere on Earth (AoE).