Skip to content

Latest commit

 

History

979 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Group 1(1)

One endpoint. Every model. Always the right one.

A drop-in proxy for Anthropic, OpenAI, and Gemini that picks the best model for every request: using a tiny on-box embedder, not a vibes-based prompt.

Weave Badge Go Tests License: ELv2 Managed deployment

Built by Weave: The #1 engineering intelligence platform, loved by Robinhood, PostHog, Reducto, and hundreds of others.


What it does

Point Claude Code, Codex, Cursor, or your own app at localhost:8080. The router:

  • 🎯 Routes per action. A cluster scorer derived from Avengers-Pro 1 picks the right model from your enabled providers, for every upstream API request. (See docs/SEMANTICS.md for the canonical terminology: the router routes per action, not per turn.)
  • 🔌 Speaks everyone's API. Anthropic Messages, OpenAI Chat Completions, Gemini native. Streaming, tools, vision, the works.
  • 🧠 Knows OSS too. DeepSeek, Kimi, GLM, Qwen, Llama, Mistral via OpenRouter (or any OpenAI-compatible endpoint).
  • 🔒 BYOK by default. Provider keys stay on your box, encrypted at rest.
  • 📊 Observable. OTLP traces out of the box. See them in the Weave dashboard (http://localhost:8080/ui/dashboard) or drop in Honeycomb, Datadog, Grafana, whatever.

30-second quickstart

The fastest way: point Claude Code, Codex, opencode, or pi at the hosted Weave Router with one command. No clone, no Docker, no Postgres.

npx @weave-os/router

That's it. The installer asks which tool (Claude Code, Codex, opencode, or pi), walks you through scope (user vs. project), grabs a router key, and wires the right config file. Other flavors:

npx @weave-os/router --claude              # skip the picker, Claude Code
npx @weave-os/router --codex               # skip the picker, OpenAI Codex CLI
npx @weave-os/router --opencode            # skip the picker, opencode
npx @weave-os/router --pi                  # skip the picker, pi + Loom UI
npx @weave-os/router --scope project       # per-repo, commits settings.json (or .codex/ / opencode.json)
npx @weave-os/router --local               # self-hosted localhost:8080
npx @weave-os/router --base-url https://router.acme.internal
npx @weave-os/router@0.1.0                 # pin a version

Requires Node ≥ 18 (Claude Code, opencode, and pi paths also need jq). Full flag reference: install/npm/README.md.

The npm package is published as @weave-os/router. The former @workweave/router package remains available as a compatibility alias and continues to receive the same releases.

Or: self-host the whole stack

If you want the router (and dashboard) running on your own box:

# 1. Drop a provider key in. OpenRouter is the recommended baseline.
echo "OPENROUTER_API_KEY=sk-or-v1-..." >> .env.local

# 2. Boot Postgres + router on :8080 and seed an rk_ key.
make full-setup

The router is up at http://localhost:8080, the dashboard at http://localhost:8080/ui/ (password: admin), and your rk_... key prints in the logs.

# Call it like Anthropic
curl -sS http://localhost:8080/v1/messages \
  -H "Authorization: Bearer rk_..." \
  -d '{"model":"claude-sonnet-4-5","max_tokens":256,
       "messages":[{"role":"user","content":"hi"}]}'

# ...or like OpenAI
curl -sS http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer rk_..." \
  -d '{"model":"gpt-4o-mini",
       "messages":[{"role":"user","content":"hi"}]}'

# Peek at the routing decision without proxying
curl -sS http://localhost:8080/v1/route -H "Authorization: Bearer rk_..." -d '...'

What that stack looks like

Only the grey boxes are off your machine. The router, the scorer, Postgres, and your provider keys all stay local; prompts go from the router straight to the provider you configured, never to Weave.

flowchart LR
    client["Claude Code, Codex, opencode,<br/>pi, Cursor, your own app"]
    router["Router :8080<br/>/v1/messages · /v1/chat/completions<br/>/v1beta/models · /v1/route"]
    scorer["Cluster scorer<br/>in-process ONNX embedder"]
    hmm["HMM policy sidecar :8093<br/>optional, make up-hmm"]
    pg[("Postgres<br/>installations, rk_ keys,<br/>encrypted BYOK keys, usage")]
    ui["Dashboard /ui<br/>selfhosted mode only"]
    providers["Anthropic · OpenAI · Gemini<br/>OpenRouter and any<br/>OpenAI-compatible endpoint"]
    otel["Your OTLP collector<br/>Honeycomb, Datadog, Grafana"]

    client -->|"rk_… bearer token,<br/>streamed response back"| router
    router -->|"embed and score the action"| scorer
    router -.->|"ROUTER_DEFAULT_STRATEGY=hmm"| hmm
    router -->|"auth, config, usage"| pg
    pg --> ui
    router -->|"provider key from env or BYOK"| providers
    router -.->|"spans and usage logs"| otel

    classDef external fill:#f4f4f5,stroke:#a1a1aa,color:#3f3f46
    class providers,otel external
Loading

Multi-replica deployments also need Pub/Sub (PUBSUB_*) for cache invalidation; docker compose runs the emulator for you.

Optional: self-host the frozen HMM policy

The default stack uses the in-process cluster scorer. To run the frozen HMM policy as a companion container, add a Google API key and use the opt-in target:

echo 'GOOGLE_API_KEY=...' >> .env.local
make up-hmm

This does not change the default strategy. See sidecars/hmm/README.md for artifact verification, embedding compatibility, and explicit HMM selection.

Wire it into your tools

Claude Code. Run make install-cc to wire Claude Code at the local self-hosted router (it's also invoked automatically at the end of make full-setup). For the hosted router, use npx @weave-os/router above.

Codex (OpenAI CLI). npx @weave-os/router --codex patches ~/.codex/config.toml (or <repo>/.codex/config.toml with --scope project) with a managed [model_providers.weave] block and sets model_provider = "weave". The provider preserves Codex's existing ChatGPT OAuth login while the router key rides in an X-Weave-Router-Key HTTP header and the installer selects the HMM strategy for the public hosted endpoint. --codex --local and custom self-hosted URLs keep their router's configured default because the HMM sidecar is optional. HMM and forced selections in the native Codex family (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna) use that OAuth credential; every other selected model uses its WorkWeave deployment or BYOK credential, matching the Claude Code plugin's model-to-credential dispatch. Codex does not load third-party slash-command files, so the installer ships the router directives as native Codex skills: $force-model <model-id> (alias $fm <model-id>), $unforce-model (alias $ufm), and $router-feedback <text> (alias $rf <text>). Each skill runs a local scripts/emit.sh that prints the leading-space directive (for example, /force-model gpt-5.6-terra); the router intercepts that exec output. You can type that form directly instead. Re-install and --uninstall --codex rewrite/remove only the managed block, leaving the rest of your Codex config untouched. Codex also gets $router-status, $router-off, $router-on, and $router-models as skills that call this installer's own verbs. Invoke $disable-routing (or $router-off) to switch the next Codex session back to its normal provider, or run npx @weave-os/router disable-routing in a shell; a literal /disable-routing is not a third-party extension point in Codex.

opencode. npx @weave-os/router --opencode merges a provider.weave entry into ~/.config/opencode/opencode.json (or <repo>/opencode.json with --scope project). It uses opencode's bundled @ai-sdk/anthropic provider pointed at the router's /v1 endpoint — the router speaks the Anthropic Messages API natively, so opencode works unmodified. The router key and identity headers ride alongside the provider config; re-install rewrites only the managed block and --uninstall --opencode strips it.

pi. npx @weave-os/router --pi keeps stock pi as the runtime and installs the router's pi extension. It adds the Loom header, Wooly's animated terminal mascot, a persistent WEAVE ROUTER route/savings line, /fm + /ufm model-pin commands with a [forced] status, and context-isolated subagents without shipping or maintaining a forked pi binary.

Cursor (early beta, performance may not be the best). Settings → Models → Override OpenAI Base URLhttp://localhost:8080/v1, paste rk_... as the API key.

Switching on/off. After installing, npx @weave-os/router off --claude (or --codex / --opencode) routes that client straight to its provider again without discarding the router config; on flips it back, and status reports which way it's pointing. Claude Code also gets /router-off, /router-on, and /router-status slash commands. Cursor toggles via the same Settings → Models override above. See install/README.md.

Choosing which models the router may pick. npx @weave-os/router models --claude lists every deployed model with its on/off state, and models enable / models disable change it — the same setting as the dashboard's settings page, edited from the terminal. Claude Code gets this as /router-models (alias /models). Requires a router that serves the model-selection API; against the Weave-hosted router the list still prints and points you at the dashboard, where selection is an organization-wide setting. See install/README.md.

Two keys, don't mix them up:

  • sk-or-... / sk-ant-... / sk-... = your upstream provider key. Lives in .env.local.
  • rk_... = your router key. Clients send this as a Bearer token.

Endpoints

Endpoint Format
POST /v1/messages Anthropic Messages, routed
POST /v1/chat/completions OpenAI Chat Completions, routed
POST /v1beta/models/:action Gemini generateContent, routed
POST /v1/route Returns the decision, no upstream call
GET /v1/models  ·  POST /v1/messages/count_tokens Anthropic passthrough
GET /health  ·  GET /readyz  ·  GET /validate liveness + dependency readiness + key check
GET /v1/sessions/:session_id/cost One session's committed cost + savings, scoped to your key
GET /v1/analytics/routing-decisions Raw routing decisions as cursor-paginated NDJSON (docs)
GET /v1/analytics/schema  ·  GET /v1/analytics/models Export field dictionary + price book

Keep liveness probes on /health. Point startup or readiness probes at /readyz when configured policy sidecars must be ready before traffic arrives.

Routed non-stream responses include x-router-cost-usd, x-router-cost-input-usd, x-router-cost-output-usd, x-router-cache-read-tokens, and x-router-cache-creation-tokens. Streaming responses cannot carry the final cost in HTTP headers because headers flush before usage is known; stream clients should read the weave_cost object on the final Anthropic message_delta usage event.

Deeper docs

  • 📐 Configuration reference: every env var, BYOK encryption, OTel knobs, cluster routing.
  • 🧭 Semantics and terminology: canonical definitions for session, round, turn, action, and step.
  • 📊 Analytics export: pulling raw routing decisions into your own warehouse with a read-only key.
  • Policy router harness: contract and rollout checklist for adding an out-of-process policy model.
  • 🛠️ Contributing: layering rules, hot-reload dev, migrations, tests, the whole engineering loop.
  • 🏗️ Architecture: package layout, import contracts, recipes for adding endpoints / providers / strategies.

Footnotes

  1. Zhang, Y. et al. Beyond GPT-5: Making LLMs Cheaper and Better via Performance–Efficiency Optimized Routing (Avengers-Pro). arXiv:2508.12631, 2025. https://arxiv.org/abs/2508.12631

About

Model router for agentic systems. Routes every prompt to the right model in <50ms. Cut costs 40-70% with just an endpoint change.

Topics

Resources

Contributing

Stars

4.1k stars

Watchers

2 watching

Forks

Releases

Used by

Contributors

Languages