How to put token rate limits on LLM traffic through agentgateway on Kubernetes.
Cap spend per route or across the whole gateway. local counters per replica; global when you need a shared budget.
v1.5.0 on agentgateway lets you issue a virtual key per team, put a rolling token or dollar cap on it, and restrict which models that key can call.
Over budget is a 429, or an audit event if you just want the receipt.
Check out the setup here:
How to put rate limits on the standalone agentgateway binary.
Cap requests and LLM token spend per route. local for one process, remote when you need shared counters across replicas.
