Building Mixture-of-Models: The Next-Generation Model Architecture for Heterogeneous LLM Inference

Documentation | Playground | Blog | Publications | Hugging Face

We believe Mixture-of-Models is the next-generation model architecture for heterogeneous LLM inference.

That is why we built vLLM Semantic Router—to turn signals and preferences into executable model paths for every user, product, and workload.

Explore how it works →

curl -fsSL https://vllm-semantic-router.com/install.sh | bash

For platform notes, detailed setup options, and troubleshooting, see the Installation Guide.

• [2026/06/29] New Blog: Micro-Agent: Beat Frontier Models with Collaboration inside Model API
• [2026/06/16] New Blog: Beyond One Model: Fusion in vLLM Semantic Router
• [2026/06/05] v0.3 Released: vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing

More announcements are available on the Blog and Publications pages.

For questions, feedback, or to contribute, please join the #semantic-router channel in vLLM Slack.

We host community meetings on the first and third Tuesday of each month to sync with contributors across different time zones:

• First Tuesday of the month: 9:00-10:00 AM EST (accommodates US EST, EU, and Asia Pacific contributors)
  • Zoom Link
  • Google Calendar Invite
  • ics file
• Third Tuesday of the month: 1:00-2:00 PM EST (accommodates US EST and California contributors)
  • Zoom Link
  • Google Calendar Invite
  • ics file
• Meeting recordings: YouTube

If you want to contribute, start with CONTRIBUTING.md.

For repository-native development workflow and validation commands, use AGENTS.md as the entrypoint and docs/agent/README.md as the canonical index.

If you find Semantic Router helpful in your research or projects, please consider citing it:

@misc{semanticrouter2025,
  title={vLLM Semantic Router},
  author={vLLM Semantic Router Team},
  year={2025},
  howpublished={\url{https://github.com/vllm-project/semantic-router}},
}

We are grateful to our sponsors who support us:

AMD provides us with GPU resources and ROCm™ software for training and researching frontier router models, enhancing E2E testing, and building the online models playground.