Quick start
- Create an account.
- Add at least $5 to your wallet. On your first funding, we add $10 in bonus credit automatically.
- Create an API key from the dashboard and copy it once.
- Set the base URL for your SDK or app to the compatible endpoint below.
- Put a supported model name in the
modelfield. - Send your first request and review usage in the dashboard.
Using an AI app?
Use the step-by-step Codex guide for the Responses API, the Cursor guide for Ask and Agent, or the Claude Code guide for the Anthropic-compatible Messages API. The integration directory covers other supported apps.
Base URLs
OpenAI-compatible
https://api.cheaperinference.com/v1
Anthropic-compatible
https://api.cheaperinference.com
Anthropic SDKs and Claude Code append /v1/messages themselves.
Authentication
Send Authorization: Bearer ci_live_YOUR_API_KEY.
The Messages endpoints also accept X-Api-Key.
Chat completion
One language setting updates every example on this page.
curl https://api.cheaperinference.com/v1/chat/completions \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-opus-4.6", "messages": [{"role": "user", "content": "Hello!"}] }'from openai import OpenAIclient = OpenAI( api_key="ci_live_YOUR_API_KEY", base_url="https://api.cheaperinference.com/v1",)response = client.chat.completions.create( model="claude-opus-4.6", messages=[{"role": "user", "content": "Hello!"}],)import OpenAI from "openai";const client = new OpenAI({ apiKey: "ci_live_YOUR_API_KEY", baseURL: "https://api.cheaperinference.com/v1"});const response = await client.chat.completions.create({ model: "gpt-5.4", messages: [{ role: "user", content: "Hello!" }]});cURL Responses request
curl https://api.cheaperinference.com/v1/responses \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-5.4", "input": "Hello!", "store": false, "stream": true }'The stateless Responses compatibility layer supports Codex streaming, function calls, and custom local tools such as apply_patch. Requests must use store: false. Stored responses, previous_response_id, conversations, background mode, and provider-hosted web or file search are not supported.
API reference
Every endpoint, header, and error code, with runnable request builders. The generated code updates as you edit the fields, so it is useful to read and copy whether or not you are signed in. The sections after it cover topics this reference does not, and go deeper on a few it only summarises.
Running a request needs an account, because the call bills to a wallet. Sign in to enable the live request builders. Everything else below is readable and copyable now.
One language setting updates every example.
Make the first request
Keep your OpenAI SDK and change the base URL plus API key. Generated snippets show the saved key prefix only because the full key is available once, at creation.
Base URL
https://api.cheaperinference.com/v1API keys
Create separate credentials for applications, reporting jobs, and environments. The complete secret is available only in the one-time creation window.
Run inference
Allows model requests that can spend wallet balance. The API scope is inference.
Read usage and spend
Allows usage reports and pricing-change reads. The API scope is usage:read.
Limit access
Restrict a key by model, IP range, expiration, request rate, concurrency, or monthly spend.
Chat completions
Send messages in the OpenAI chat format. Turn streaming on in Request controls to inspect the server-sent event response.
Try chat completions
Edit the prompt and controls. The generated code updates with every field.
Loading available models…
No published list price for this model. Savings show optimization only.
Request controls
Return server-sent events as tokens arrive.
Send X-CI-Concise: 1 for a shorter response.
Send X-CI-Route: auto for eligible prompts.
Output monitor
Live response
Sign in to run this request
The request and its generated code are complete and copyable now. Running the prompt against gemini-3.7-flash needs an account, because the call bills to a wallet.
Sign in to try itEstimated cost · $0.000000
POST /chat/completions
Text completions
Use the legacy prompt field for clients that still call the completions endpoint. The same pricing and optimization pipeline applies.
Try text completions
The request uses a 128-token output budget by default to keep the test inexpensive.
Loading available models…
No published list price for this model. Savings show optimization only.
Request controls
Return server-sent events as tokens arrive.
Send X-CI-Concise: 1 for a shorter response.
Send X-CI-Route: auto for eligible prompts.
Output monitor
Live response
Sign in to run this request
The request and its generated code are complete and copyable now. Running the prompt against gemini-3.7-flash needs an account, because the call bills to a wallet.
Sign in to try itEstimated cost · $0.000000
POST /completions
Image generation
Choose an image model and describe the image you want. The response uses the OpenAI Images shape and the playground previews returned URLs or base64 images.
Generate an image
The generated cURL and SDK examples use the image endpoint and a media-compatible payload.
Loading available models…
No published list price for this model. Savings show optimization only.
Output monitor
Live response
Sign in to run this request
The request and its generated code are complete and copyable now. Running the prompt against gemini-3.7-flash needs an account, because the call bills to a wallet.
Sign in to try itEstimated cost · $0.000000
POST /images/generations
Video generation
Video generation is buffered while the provider completes the job. The example requests a five-second 720p clip and allows up to five minutes.
Generate a video
Returned video URLs and base64 MP4 data are playable in the output monitor.
Loading available models…
No published list price for this model. Savings show optimization only.
Output monitor
Live response
Sign in to run this request
The request and its generated code are complete and copyable now. Running the prompt against gemini-3.7-flash needs an account, because the call bills to a wallet.
Sign in to try itEstimated cost · $0.000000
POST /videos/generations
Anthropic Messages API
POST /messages. Use Anthropic-compatible messages, streaming, vision, and client-side tools with a Cheaper Inference key. Token counting is available at POST /messages/count_tokens.
X-Api-Key or a Bearer token. Claude Code executes tools locally and returns tool results to this endpoint.Open the Claude Code setup guideResponses API
POST /responses. The stateless Responses compatibility layer supports streaming, function calls, and local tools for Codex clients. Set store to false.
wire_api = "responses" and point its base URL at this API's /v1 path.Open the Codex setup guideModels
GET /models. Fetch every model currently available. Curated models carry verified list pricing. Marketplace models use the cheapest seller ask plus the configured routing fee and claim no list-price discount.
List available models
Curated catalog models plus the current marketplace feed.
Response feedback
POST /feedback. Attach a 1 to 5 score to a request on your account. Experiment results use this signal to compare variants.
Headers
Request headers choose optional experiment and optimization behavior. Response headers identify what ran and what it saved.
Request headers
| Header | Description |
|---|---|
| X-CI-Experiment | Opts this request into a running experiment. Set the value to the slug shown on the Experiments page. |
| X-CI-End-User | Your stable, non-sensitive user id. For sticky assignment, the same id stays on the same variant. |
| X-CI-Concise | Set to 1 to opt into concise output. |
| X-CI-Route | Set to auto to opt into model routing. |
Response headers
| Header | Description |
|---|---|
| x-ci-request-id | Request id for logs and feedback. |
| x-ci-tokens-saved | Input tokens removed or avoided. |
| x-ci-saved-usd | Savings in USD, reported to six decimal places. |
| x-ci-techniques | Comma-separated optimization technique ids. |
| x-ci-experiment-variant | The assigned variant name. Present only when X-CI-Experiment matched a running experiment. |
| x-ci-cache | hit when exact-match caching served the response. |
Usage reporting
GET /usage. Read request-level history or daily aggregates with a key that has the Read usage and spend permission. Reporting-only keys cannot run inference.
GET /usage/requests
Per-request token counts, billed cost, status, model, latency, and cursor pagination.
GET /usage/daily
Daily request and spend totals across an optional date range, including zero-usage dates.
Rate limits and retries
CheaperInference does not publish a fixed account-level request quota today. Marketplace sellers can return 429 responses when capacity changes.
Retry safely
Use exponential backoff with jitter. Respect Retry-After when the upstream response includes it.
Bound each call
Set max_tokens on every request. The dashboard examples default to 128.
No failed-request charge
Provider errors are recorded for diagnosis, but wallet settlement only runs after a successful response.
Machine-readable spec and integration guides
The OpenAPI document is generated from the service and checked in CI, so it cannot drift from what these endpoints actually do.
OpenAPI 3.1 document
Every /v1 endpoint, its parameters and every error status. Point a client generator at it. No API key required.
Claude Code
Anthropic Messages. Set ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY.
Codex
OpenAI Responses. Add a model provider with wire_api = "responses".
Cursor
OpenAI Chat Completions. Override the base URL in Settings → Models.
Guides and detail
Topics the reference above does not cover, and deeper notes on the ones it summarises.
Available models and pricing
Copy the exact model ID into the model field. Every model we serve, with its current price and capabilities, is on the live model catalog. Each GET /v1/models item also includes an aliases array with the vendor-qualified id, including vendor-specific version punctuation, so migrations do not need a hand-maintained map. IDs absent from this catalog return HTTP 404.
How pricing and routing work
For every request, Cheaper Inference ranks the eligible provider routes by estimated cost and starts with the lowest-cost route available for that model and request. If that provider is unavailable, has insufficient capacity, or fails before output begins, the gateway automatically tries the next-lowest-cost eligible route. Your public model ID and API response format remain unchanged.
A fallback route can have a different rate from the first route. The final customer charge follows the route that successfully serves the request, but it will never exceed the model maker's applicable direct API list price. Cheaper Inference adds no separate routing surcharge; the settled API rate is all-in.
Provider prices can change over time. The exact amount charged for each completed request is available in your dashboard.
Model catalog API
Use your API key to retrieve currently available models and their current catalog rates. A fallback route can change the final rate for a request, subject to the direct-list-price ceiling above. Filter by type, vision, video, reasoning, streaming, or provider to find models that meet a workload's requirements.
curl 'https://api.cheaperinference.com/v1/models?type=text&vision=true&streaming=true' \ -H 'Authorization: Bearer ci_live_YOUR_API_KEY'
Token rates are USD per 1 million tokens. Fixed media rates use media_input_unit_price and media_unit_price per unit. For tiered models, use pricing.above_threshold when the request exceeds input_token_price_threshold.
Model prices do not change on a fixed customer-facing schedule: they can change when an upstream rate or an eligible route changes. Do not treat a locally cached rate as the final charge for a request. Use these response fields to validate a cache:
pricing_versionis a SHA-256 version of the returned, API-key-filtered catalog. An unchanged value means that catalog is unchanged.pricing_checked_atis the UTC time of the latest successful provider-price check.pricing_updated_atis the latest UTC catalog change among the returned models.
Pricing change feed
curl 'https://api.cheaperinference.com/v1/pricing/changes?since=2026-07-29T00:00:00Z' \ -H 'Authorization: Bearer ci_live_YOUR_API_KEY'
since is a required RFC 3339 timestamp. The endpoint returns the latest current customer-facing model object for each catalog row changed at or after that time. A removal has change_type: "removed" and model: null. Follow next_cursor while has_more is true; keep the original since value on every page.
{ "object": "pricing.changes", "pricing_version": "sha256:8f4c...", "pricing_checked_at": "2026-07-29T14:32:10Z", "pricing_updated_at": "2026-07-29T14:31:52Z", "data": [{ "model_id": "gpt-5.4", "change_type": "updated", "changed_at": "2026-07-29T14:31:52Z", "model": { "id": "gpt-5.4", "object": "model", "type": "text", "endpoint": "/v1/chat/completions", "pricing": { "currency": "USD", "input_per_million": "1.250000", "cache_read_input_per_million": "0.125000", "cache_write_input_per_million": "1.250000", "output_per_million": "10.000000" } } }], "has_more": false, "next_cursor": null}There is no pricing webhook at this time. Poll this feed, process changes idempotently by model_id and changed_at, then use the newest changed_at as the next since value. Because since is inclusive, the boundary change may be returned again. Periodically compare pricing_version with GET /v1/models to reconcile the full cache.
The catalog identifies modality, endpoint, vision, video, reasoning and streaming support. Tool calling, structured output, context limits, and accepted optional parameters can still vary by model and serving provider; verify the exact request shape before production use.
Parameters
OpenAI-compatible parameters
Chat requests use the OpenAI Chat Completions shape. Common fields such as temperature, top_p, max_tokens, max_completion_tokens, stop, tools, tool_choice, response_format, and reasoning are forwarded to the selected model.
Support and interpretation are model-specific. An unsupported parameter may be ignored or rejected by the serving provider, so test the exact model and request shape before moving production traffic.
Anthropic-compatible parameters
Messages requests accept model, messages, max_tokens, system, stream, temperature, top_p, stop_sequences, tools, tool_choice, and thinking. Use exact model IDs from GET /v1/models rather than Anthropic aliases.
Tools and structured output
Models that support tool calling accept the standard OpenAI tools and tool_choice fields. Models that support JSON output accept response_format, including json_object or a JSON schema where the model supports it.
"response_format": {
"type": "json_object"
}Price ceiling
Add min_discount_percent to a chat completions, completions, messages, or responses request. The value sets the minimum discount off list price that the request will accept.
The field takes a number from 0 to 99.99. The gateway checks the value against our published list price for the model. It never checks it against a price claimed by a supply source. The gateway removes the field from the request body before it forwards the request, so no upstream provider sees it.
curl https://api.cheaperinference.com/v1/chat/completions \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-5.4", "messages": [{"role": "user", "content": "Hello!"}], "min_discount_percent": 40 }'Only supply priced at or under the ceiling can serve the request. The gateway checks the input rate and the output rate separately. A source must clear the ceiling on both rates to qualify. The gateway also caps the bill for the request at list price times (1 − min_discount_percent / 100) for each rate, even when the serving source would allow a higher charge.
When no supply currently clears the ceiling, the request fails. The gateway does not fall back to a source priced above the ceiling.
Errors
400/invalid_min_discount:min_discount_percentis not a number, or it is outside 0–99.99.400/list_price_unknown: the gateway has no list price for this model, so it cannot check a ceiling.503/min_discount_unavailable: no supply is currently priced at or under the ceiling. Lowermin_discount_percentor retry later.
{ "error": { "message": "No supply for gpt-5.4 currently meets min_discount_percent=40 (a required discount of at least 40%; customer ceiling $1.80 in / $9.00 out per 1M tokens; upstream list $3.00 / $15.00). Lower min_discount_percent or retry later.", "type": "api_error", "param": null, "code": "min_discount_unavailable" }}Check available supply first
Use GET /v1/models/supply to count sources that currently clear a ceiling for a model, before you send a real request.
curl 'https://api.cheaperinference.com/v1/models/supply?model=gpt-5.4&min_discount_percent=40' \ -H "Authorization: Bearer ci_live_YOUR_API_KEY"
{ "object": "model.supply", "model": "gpt-5.4", "min_discount_percent": 40, "list_input_per_million": "3.00", "list_output_per_million": "15.00", "max_input_per_million": "1.80", "max_output_per_million": "9.00", "supply_buffer_percent": 4, "candidate_count": 3, "candidates": ["primary", "network_a", "network_b"], "primary_market_offers": 2, "primary_market_min_discount_percent": 12, "note": "Candidates are priced under the ceiling now. Health and fill are not guaranteed."}candidate_count is the number of sources priced under the ceiling right now. max_input_per_million and max_output_per_million are the capped billing rates for this ceiling. primary_market_offers is null when the primary marketplace could not be checked for this request. A candidate count above zero reports current pricing only. It does not guarantee provider health or a successful fill.
For the full parameter and schema reference, see the API reference.
Zero data retention
Add zdr: true to a chat completions, completions, messages, or responses request. The gateway then routes the request only through providers that support a zero-data-retention policy. A zero-data-retention provider does not keep your prompts or responses and does not train on them.
curl https://api.cheaperinference.com/v1/chat/completions \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-sonnet-5", "messages": [{"role": "user", "content": "Hello!"}], "zdr": true }'The gateway removes the field from the request body before it forwards the request. The flag adds to the Data Protection setting on your account. A request cannot switch that setting off with zdr: false.
A ZDR request excludes every provider that does not support a zero-data-retention policy. The pool is smaller, so the price is usually higher than the standard price for the same model. The catalog shows the ZDR price when you switch on the ZDR only toggle, and GET /v1/models?zdr=true returns the same view.
curl 'https://api.cheaperinference.com/v1/models?zdr=true' \ -H "Authorization: Bearer ci_live_YOUR_API_KEY"
Errors
400/invalid_zdr:zdris not a boolean.503/zdr_capacity_unavailable: no provider with zero-data-retention support currently serves the model. Choose another model. Ifzdrcame from the request body, you can also remove it; if it came from the account's Data Protection setting, change that setting instead.
Use GET /v1/models/supply?model=…&zdr=true to count the zero-data-retention sources for a model before you send a real request.
Ranking: speed, discount, balance
Several discounted sources can usually serve one request: the marketplace, seller networks, and direct providers. The gateway tries them in order until one succeeds, within a time limit. Only when none of them can serve the request in time does it fall back to a reference rail (OpenRouter or the Vercel AI Gateway). Add ranking to a chat completions, completions, messages, or responses request to choose the order of the discounted sources. The rails are never part of that ranking.
curl https://api.cheaperinference.com/v1/chat/completions \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-5.4", "messages": [{"role": "user", "content": "Hello!"}], "stream": true, "ranking": "speed" }'"discount": the cheapest source first. This is the order every request used before this field existed."speed": the source with the highest recent speed score first. Cost breaks ties."balance"(default): speed score multiplied by discount, highest first.
The speed score comes from the gateway's own traffic on the same model, not from a fixed list of "fast" providers. For each discounted source it measures the time to the first visible output token and the output tokens per second over the last twelve hours, with the last hour weighted most. The score is relative to the other sources on that model. A source with fewer than five recent requests scores neutral, so a model with little traffic ranks by cost until the signal builds.
Speed and discount are not a trade-off. The fastest source for a model is often also one of the cheapest, so "speed" can pick the same route as "discount", and "discount" does not promise a slower answer. The field says how the gateway ranks the candidates given the recent telemetry. It does not add or remove candidates, and it does not change the price rule: the source that serves the request sets the price you pay.
The gateway removes the field from the request body before it forwards the request. Combine it freely with min_discount_percent and zdr: those two decide which sources are eligible, ranking decides the order among them.
Errors
400/invalid_ranking:rankingis not one ofspeed,discount,balance.
Images and vision
Vision input
Vision-capable chat models accept OpenAI image_url parts and Anthropic-style base64 image blocks. Check capabilities.vision in GET /v1/models before sending one.
Streaming
Models marked Streaming accept "stream": true on Messages, chat, text completion, and Responses endpoints. Streams use server-sent events, and wallet usage is settled when the stream completes. The gateway can retry and change routes before output begins. After the first event is delivered, a failed partial stream cannot be replaced transparently; retry the complete request.
Runnable vision request
Use an HTTPS image URL or a base64 data URL in an OpenAI image_url content part:
curl https://api.cheaperinference.com/v1/chat/completions \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-5.4", "messages": [{ "role": "user", "content": [ {"type": "text", "text": "Describe this image."}, { "type": "image_url", "image_url": {"url": "https://example.com/image.png"} } ] }] }'Generate an image
Image-generation models use POST /v1/images/generations, not Chat Completions. The response contains data entries with either a URL or base64 image data.
curl https://api.cheaperinference.com/v1/images/generations \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "nano-banana", "prompt": "A tiny observatory on a mossy hill at blue hour", "n": 1 }'Edit an image
Use POST /v1/images/edits only with a model whose capabilities.image_edit value is true in GET /v1/models. Multipart requests accept one to ten PNG, JPEG, or WebP source images and an optional PNG mask; decoded sources plus the mask may total at most 20 MiB. An unsupported edit is refused and is never converted into a text-to-image generation.
curl https://api.cheaperinference.com/v1/images/edits \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -F "model=gpt-image-2" \ -F "prompt=Replace the background with a quiet library" \ -F "image=@source.png;type=image/png" \ -F "mask=@mask.png;type=image/png"The JSON extension accepts source HTTP(S) URLs in input_references. Cheaper Inference passes those URLs only to compatible providers and never downloads them itself. Masks must remain PNG uploads or PNG data URLs so their semantics cannot be silently dropped by a provider route.
Generate a video
Video models use POST /v1/videos/generations. Generation is buffered while the provider completes the job, so use a client timeout of at least five minutes. The response contains avideos array with URLs or base64 MP4 data.
curl https://api.cheaperinference.com/v1/videos/generations \ -H "Authorization: Bearer ci_live_YOUR_API_KEY" \ -H "Content-Type: application/json" \ --max-time 300 \ -d '{ "model": "seedance-2.0", "prompt": "A paper boat drifting through a neon-lit rain puddle", "duration": 5, "resolution": "720p", "n": 1 }'Audio and voice: This API does not currently expose speech generation, transcription, or voice endpoints, and the model catalog contains no audio-generation rows. Do not send audio models to Chat Completions; audio examples will be added when a billable audio route is available.
Prompt caching
cache_control is passed through when supported by the selected model. Cache reads and writes use imported model-specific rates when available. If no cache-read rate is available, cached reads use 10% of the normal input rate; cache writes without a separate rate use the normal input rate.
cURL Anthropic Messages request
Available to all users: every Cheaper Inference account can use these endpoints with an active API key that has Inference scope. No Anthropic account or separate enablement is required. Normal wallet balance, model access, and API-key limits still apply.
Compatibility scope: this integration implements POST /v1/messages and POST /v1/messages/count_tokens. It is not the complete Anthropic platform API; unsupported features are listed below.
One language setting updates every example on this page.
curl https://api.cheaperinference.com/v1/messages \ -H "X-Api-Key: ci_live_YOUR_API_KEY" \ -H "Anthropic-Version: 2023-06-01" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-opus-4.8", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello!"}] }'from anthropic import Anthropicclient = Anthropic( api_key="ci_live_YOUR_API_KEY", base_url="https://api.cheaperinference.com",)message = client.messages.create( model="claude-opus-4.8", max_tokens=1024, messages=[{"role": "user", "content": "Hello!"}],)POST /v1/messages accepts the Anthropic Messages shape and returns an Anthropic-style message. It supports buffered and streamed text, system content, vision blocks, client-side tools and tool results, multi-turn replay, stop sequences, tool choice, usage accounting, and extended-thinking request hints. Streams use Anthropic event names such as message_start, content_block_delta, message_delta, and message_stop.
Use POST /v1/messages/count_tokens with the same model, system, messages, and tools fields for an estimated input_tokens count. Counting does not submit an inference request or charge the wallet.
Client-side tools work because Claude Code executes them locally and returns tool_result blocks. Anthropic-hosted server tools, Message Batches, Files, and signed thinking-block replay are not compatibility targets. Thinking settings can guide an eligible model, but responses do not expose Anthropic-signed thinking blocks.
For the complete environment setup and model mapping, use the Claude Code integration guide.
Errors and limits
Billing and wallet
- Wallet balance is checked before a request starts.
- Usage is deducted after a completed request.
- Add funds manually or enable auto-recharge.
Security and keys
- Store API keys in environment variables, not frontend code.
- Create separate keys for production and testing.
- Configure model, IP, expiration, rate, concurrency, daily quota, and monthly budget controls from API Keys.
- Revoke exposed keys immediately.
Common errors
OpenAI-compatible endpoints use the envelope below. The Messages endpoints return Anthropic's {"type":"error","error":{...}} shape instead. Use the error type for program logic and keep the message for logs and operator context.
{ "error": { "message": "Invalid API key.", "type": "authentication_error", "param": null, "code": "invalid_api_key" }}400/invalid_request: unsupported model, endpoint, or invalid request body. Check the JSON body, model id, and required fields.401/invalid_api_key: invalid or missing API key. Create or rotate a key, then update the Bearer token.402/insufficient_balance: add funds in Billing, then retry the request.403The API key does not allow the model, client IP, or request.404/request_not_found: feedback can only target a request on your own account.413Request body or vision payload is too large.422A query or typed request parameter failed validation.429/rate_limit_error: a rate, concurrency, or quota limit was reached. Retry with exponential backoff and jitter, and respect theRetry-Afterresponse header before retrying.502/upstream_error: provider or transport failure after eligible routes were exhausted. The request is recorded but not charged.503No eligible provider route is currently available.504The serving provider timed out.
Reliability
Automatic retries and fallback
Network failures and HTTP 404, 408, 409, 425, 429, and 5xx responses are retried once. Other 4xx responses are not retried; when another route is eligible, the gateway immediately tries it in price order. Your request keeps the same public model ID and response format.
This failover happens automatically for provider capacity, availability, and transport failures. Validation, authentication, insufficient-balance, and oversized-payload errors still require a change from the caller.
Usage and response privacy
Successful responses include the model's token usage and a namespaced cheaper_inference object with the request ledger ID and exact settled customer charge. Streaming responses include this object in the protocol-specific final event. The request ID is also returned in the X-Cheaper-Inference-Request-Id header, and in x-ci-request-id. Provider cost and private routing metadata are removed before the response is returned to you.
{ "usage": { "prompt_tokens": 1234, "completion_tokens": 321, "total_tokens": 1555, "cost": 0.012345, "cost_details": { "upstream_inference_cost": 0.012345 } }, "cheaper_inference": { "request_id": 4812, "billing": { "status": "settled", "billed_cost_usd": "0.012345", "currency": "USD" } }}usage.cost and usage.cost_details.upstream_inference_cost both report the exact all-in customer charge after settlement. From the caller's perspective Cheaper Inference is the upstream inference service; the private amount paid to the selected serving provider is not exposed. cheaper_inference.billing.billed_cost_usd is the persisted amount deducted for that request after routing and settlement. For Chat Completions and Completions, streaming sends this metadata in a final empty-choice event immediately before [DONE]. Responses API streams include it in the final response.completed object. Messages streams include it on the final message_delta event. A failed or unsettled request does not report a billed amount.
Cheaper Inference does not store prompt or response bodies in its application database. Prompts are forwarded to the provider that serves the request, and provider-side handling is subject to that provider's applicable terms. Prompt caching may also create provider-side cached state. Request metadata needed for billing and operations is retained, including model, endpoint, token counts, charged amount, status, and a sanitized failure reason. Review the privacy policy for the full statement.
Providers and request shapes
Every request is served by one of several supply sources: a marketplace, seller networks that bring their own provider credentials, direct providers on our credentials, and benchmark rails. Before ranking them by cost, the router removes any source that does not honour the shape of your request. A shape is the set of features a request uses: the endpoint, streaming, tools and a forced tool_choice, a reasoning effort, JSON mode, image input, logprobs.
The table lists every shape a source is known not to honour, from live probes we re-run when a provider changes. A source that is not listed for a shape honours it on every model we tested. When no source is left for a shape, the request is still served on the cheapest eligible source and the affected field may come back empty.
| Supply source | Shape not honoured | Models | What happens |
|---|---|---|---|
| Primary marketplace | Tools together with a reasoning effort | gpt-5.6-luna, gpt-5.6-terra, gpt-5.6-sol | The marketplace rewrites reasoning_effort to none on the default provider mix; a per-model pin picks a provider that keeps it. |
| Primary marketplace | Tools together with a reasoning effort on /v1/chat/completions | glm-5.3 | The marketplace returns reasoning_tokens = 0 for glm-5.3 when tools are present on the chat surface; the Responses surface keeps them. |
| Primary marketplace | response_format (json_object, json_schema) on Claude | claude-sonnet-5, claude-opus-5, claude-fable-5 | The marketplace rejects response_format on Claude models with HTTP 400 ("Extra inputs are not permitted"). |
| Primary marketplace | logprobs | all | The marketplace returns 200 without a logprobs object on every model. |
| DeepInfra | /v1/responses | all | DeepInfra has no /v1/responses endpoint (HTTP 404). |
| DeepInfra | Reasoning effort | glm-5.3, glm-5.3-flash, deepseek-v4-flash, kimi-k3, claude-opus-5, claude-sonnet-5 | DeepInfra returns 200 with reasoning_tokens = 0 for a reasoning effort on these models, with or without tools; only deepseek-v4-pro reasons. |
| DeepInfra | response_format json_schema | glm-5.3, claude-opus-5, claude-sonnet-5 | DeepInfra accepts a json_schema response_format and returns prose that is not JSON on these models. |
| DeepInfra | logprobs | deepseek-v4-pro, claude-opus-5, claude-sonnet-5 | DeepInfra returns 200 without a logprobs object on these models. |
| Vercel AI Gateway | Legacy /v1/completions | all | The Vercel AI Gateway has no /v1/completions endpoint (HTTP 404). |
| Vercel AI Gateway | logprobs | all | The Vercel AI Gateway returns 200 without a logprobs object, including on OpenAI models that support it natively. |
| Vercel AI Gateway | Tools together with a reasoning effort | claude-opus-5, deepseek-v4-flash | The Vercel AI Gateway returns reasoning_tokens = 0 when tools are present with a reasoning effort on these models, while the same request without tools reasons. |
| Vercel AI Gateway | Tools together with reasoning on /v1/responses | glm-5.3-flash | On the Responses surface the Vercel AI Gateway returns reasoning_tokens = 0 for glm-5.3-flash when tools are present. |
| OpenRouter | logprobs | gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, gpt-5.5 | OpenRouter rejects logprobs on these reasoning models with HTTP 400 (logprobs are not supported with reasoning models). |
| OpenRouter | logprobs | claude-sonnet-5, claude-opus-5, glm-5.3-flash | OpenRouter returns 200 without a logprobs object for these models. |
| OpenRouter | Tools together with a reasoning effort | claude-opus-5 | OpenRouter returns reasoning_tokens = 0 for claude-opus-5 when tools are present with a reasoning effort, on chat and Responses; the same request without tools reasons. |
| OpenRouter | Tools together with a reasoning effort on /v1/responses | glm-5.3-flash, glm-5.3 | On the Responses surface OpenRouter returns reasoning_tokens = 0 for these GLM models when tools are present; the chat surface keeps them. |
| OpenRouter | response_format json_schema | glm-5.3-flash | OpenRouter accepts a json_schema response_format for glm-5.3-flash and returns prose. |
| GMI Cloud | logprobs | claude-opus-5, claude-sonnet-5, gpt-5.6-luna, deepseek-v4-flash | GMI Cloud returns 200 without a logprobs object on these models, or HTTP 400 on deepseek-v4-flash. |
| GMI Cloud | Legacy /v1/completions | claude-opus-5, claude-sonnet-5, glm-5.3, glm-5.3-flash | GMI Cloud rejects the legacy Completions surface for these models (HTTP 422 on Claude, 400 on GLM). |
| GMI Cloud | response_format json_schema | deepseek-v4-pro | GMI Cloud rejects a json_schema response_format on deepseek-v4-pro with HTTP 400. |
| Novita AI | Reasoning effort | claude-opus-5, claude-sonnet-5 | Novita AI returns 200 with reasoning_tokens = 0 for a reasoning effort on these models, with and without tools. |
| Novita AI | Tools together with a reasoning effort on /v1/chat/completions | gpt-5.6-luna | Novita AI rejects tools together with a reasoning effort on the chat surface with HTTP 400. |
| Novita AI | response_format json_schema | glm-5.3, glm-5.3-flash, claude-opus-5, claude-sonnet-5, deepseek-v4-flash, deepseek-v4-pro | Novita AI returns prose for a json_schema response_format on four models and HTTP 400 on the two DeepSeek models. |
| Novita AI | logprobs | glm-5.3, glm-5.3-flash, deepseek-v4-flash, claude-opus-5, claude-sonnet-5, gpt-5.6-luna | Novita AI returns 200 without a logprobs object on five models and HTTP 400 on gpt-5.6-luna. |
| Novita AI | Legacy /v1/completions | all | Novita AI has no legacy Completions surface; it answers HTTP 400 on every model. |
| Novita AI | /v1/responses | deepseek-v4-flash, deepseek-v4-pro, claude-opus-5, claude-sonnet-5 | Novita AI answers HTTP 400 on the Responses surface for these models. |
| EZModel | Reasoning effort | deepseek-v4-flash | EZModel returns 200 with reasoning_tokens = 0 for a reasoning effort on deepseek-v4-flash, on chat and Responses. |
| EZModel | response_format json_schema | deepseek-v4-flash, kimi-k3 | EZModel returns prose for a json_schema response_format on deepseek-v4-flash and HTTP 400 on kimi-k3. |
| EZModel | logprobs | deepseek-v4-flash, deepseek-v4-pro | EZModel returns 200 without a logprobs object on deepseek-v4-flash and HTTP 400 on deepseek-v4-pro. |
| EZModel | Legacy /v1/completions | all | EZModel validates a messages field on the legacy Completions surface, so every legacy body gets HTTP 400. |
| W&B Inference | Reasoning effort on /v1/chat/completions | deepseek-v4-flash, deepseek-v4-pro | W&B Inference returns 200 with reasoning_tokens = 0 for a reasoning effort on the chat surface; deepseek-v4-pro reasons on the Responses surface. |
| Microsoft Foundry seller network | Tools together with a reasoning effort on /v1/chat/completions | all | Foundry's Chat Completions surface refuses tools together with a non-none reasoning effort, so the seller leg downgrades the effort; the native Responses surface keeps it. |
Two shapes are surface-bound rather than source-bound: Anthropic Messages requests that carry signed thinking blocks are served only by sources with a native Messages surface, and namespaced tool choices on /v1/responses only by sources with a native Responses surface. Model capabilities (vision, reasoning, video) come from the model catalog and are not repeated here.
Usage and billing API
Usage reporting is workspace-wide and accepts the same API-key authentication as inference. Create a key with Usage reporting only access in API Keys when the reporting system should not be able to submit inference requests or spend wallet funds. Add api_key_id to restrict either endpoint to one key.
Request history
curl 'https://api.cheaperinference.com/v1/usage/requests?start_at=2026-07-01T00:00:00Z&end_at=2026-08-01T00:00:00Z&limit=100' \ -H 'Authorization: Bearer ci_live_YOUR_API_KEY'
start_atandend_atare optional RFC 3339 timestamps. The start is inclusive and the end is exclusive.api_key_idoptionally filters to a key owned by the same workspace.limitaccepts 1–100 and defaults to 100.cursoraccepts the previous page'snext_cursor. Results are newest first.
{ "object": "list", "scope": "workspace", "data": [{ "request_id": 4812, "api_key_id": 19, "api_key_name": "production", "model": "gpt-5.4", "endpoint": "/v1/chat/completions", "status": "settled", "prompt_tokens": 1234, "completion_tokens": 321, "cache_read_input_tokens": 0, "cache_write_input_tokens": 0, "total_tokens": 1555, "billed_cost_usd": "0.012345", "currency": "USD", "total_latency_ms": 1890, "time_to_first_token_ms": 420, "created_at": "2026-07-29T14:30:00Z", "completed_at": "2026-07-29T14:30:02Z" }], "has_more": false, "next_cursor": null}billed_cost_usd is non-null only when status is settled. Request history intentionally excludes prompts, generated content, provider cost, private route names, and profit or margin data.
Daily spend
curl 'https://api.cheaperinference.com/v1/usage/daily?start_at=2026-07-01T00:00:00Z&end_at=2026-08-01T00:00:00Z' \ -H 'Authorization: Bearer ci_live_YOUR_API_KEY'
- When dates are omitted, the endpoint returns the preceding 30 days through the current time.
- Every calendar date in the range is returned, including dates with zero requests.
{ "object": "usage.daily", "scope": "workspace", "currency": "USD", "start_at": "2026-07-01T00:00:00Z", "end_at": "2026-08-01T00:00:00Z", "total_requests": 820, "settled_requests": 811, "spend_usd": "148.420015", "prompt_tokens": 1043200, "cached_tokens": 786400, "cache_reported_request_count": 811, "cache_hit_pct": 75.4, "daily_spend": [{ "date": "2026-06-30", "request_count": 24, "settled_request_count": 24, "spend_usd": "3.910042", "prompt_tokens": 30500, "cached_tokens": 22100, "cache_reported_request_count": 24, "cache_hit_pct": 72.5 }]}Daily spend sums persisted charges for settled requests only. Therefore, total_requests can be greater than settled_requests.
Balance
curl 'https://api.cheaperinference.com/v1/account/balance' \ -H 'Authorization: Bearer ci_live_YOUR_API_KEY'
{ "object": "account.balance", "currency": "USD", "balance_usd": 124.50, "available_usd": 118.02, "reserved_usd": 6.48, "auto_recharge_enabled": true, "threshold_usd": 25.00, "recharge_amount_usd": 100.00, "workspace_name": "Acme", "funded_by_owner": false}Spend against available_usd, not balance_usd. The difference is reserved_usd: money already held against requests that have not settled yet, so the balance on its own overstates what can still be spent. This endpoint needs the account:read scope — a usage-only key receives a 403.
API-key access
inferencepermits inference requests.usage:readpermits request history, daily spend, and pricing-change reads.account:readpermits balance and account reads.GET /v1/modelsaccepts any valid API key.- Use a reporting-only key for billing exports and monitoring systems.
Cursor compatibility
Cursor Ask and Agent can use OpenAI-family models through the OpenAI base-URL override. In Cursor Settings → Models, add a dedicated Cheaper Inference key, enable Override OpenAI Base URL, and enter https://api.cheaperinference.com/v1. Select an exact model ID from the live catalog, then verify both a short Ask prompt and one harmless Agent edit.
Cursor sends Agent tool traffic through Chat Completions. The gateway normalizes the current streamed ApplyPatch tool shape and the older Responses-shaped BYOK request format. Cursor's override is global for OpenAI-family models; tab completion, Claude- and Gemini-family models, and other Cursor-hosted features do not use this connection.
Use key scope, model, rate, spend, and expiration controls. Do not restrict the key to your workstation IP because Cursor sends these requests from its own service. See the complete Cursor setup and troubleshooting guide.
Production checklist
- Load your API key from a server-side environment variable.
- Use
GET /v1/modelsto confirm the model is available and supports the required modality. - Set an explicit client timeout appropriate for long model generations.
- For direct HTTP clients, retry transport errors,
429, and5xxresponses with exponential backoff. - Do not retry
400,401,402, or413without correcting the request. For402, add funds before retrying. - Treat an interrupted stream as incomplete and retry the entire request.
- Review token usage, spend, and failures in the dashboard before increasing traffic.
For the generated public request, response, authentication, and error schemas, open the API reference.