01Routing
Routing by rule, with failover
Rules send requests to different upstreams by model, tools, images, extended thinking and more, and can set the reasoning effort. When an upstream fails before the answer begins, the next one takes over, a brief failure is retried after a short wait, and each session stays on one upstream so its prompt cache keeps hitting. Auxiliary requests such as title generation and warm-ups can be answered locally without using any quota.
02Protection
Protection against relays: keys replaced, malicious tool calls cut off
API keys, private keys, connection-string passwords, ID numbers and bank card numbers can be replaced with placeholders before a request leaves, so a relay never holds the real values, and an answer whose tool call downloads and runs code or sends out credentials can be cut off before the client runs it. Hidden characters that smuggle in instructions can be deleted, prompt injection refused, and chosen words deleted or replaced in requests and answers. The three protections start in Monitor, which records and notifies without changing anything; each is switched on separately, and every rule has its own action.
03Tracing
Every request, traceable
A request shows the rule it matched, each upstream it tried, any conversion between API formats and how its cost was calculated, and its content can be read live while the answer streams. A finished request can be replayed against another upstream and the two answers compared side by side, and the whole history can be searched, including the text of requests and answers.
04Cost
Costs stated as they are
Tokens, cost, cache hits, requests, time to first token and traffic on one screen, each against the previous period and broken down by model, upstream or key. Estimated amounts are marked, and requests without a price are counted separately instead of as zero; prices follow LiteLLM's public list, refreshed daily, or a custom price sheet.