OpenAI-Compatible API

DocsGPT exposes /v1/chat/completions following the standard chat completions protocol. Point any compatible client — opencode, Aider, LibreChat or the OpenAI SDKs — at your DocsGPT Agent by changing only the base URL and API key.

Quick Start

from openai import OpenAI
 
client = OpenAI(
    base_url="http://localhost:7091/v1",   # or https://gptcloud.arc53.com/v1
    api_key="your_agent_api_key",
)
 
response = client.chat.completions.create(
    model="docsgpt-agent",
    messages=[{"role": "user", "content": "Summarize our refund policy"}],
)
print(response.choices[0].message.content)

curl -X POST http://localhost:7091/v1/chat/completions \
  -H "Authorization: Bearer your_agent_api_key" \
  -H "Content-Type: application/json" \
  -d '{"model":"docsgpt-agent","messages":[{"role":"user","content":"Summarize our refund policy"}]}'

The model field is accepted but ignored — the agent bound to your API key determines the model. The agent’s prompt, sources, tools, and default model are loaded automatically.

Base URL & Auth

Authenticate with Authorization: Bearer <agent_api_key>. Every /v1 request is an agent-key request, even from the agent’s owner.

Endpoints

Streaming

Set "stream": true. You’ll receive SSE chunks with choices[0].delta.content. DocsGPT-specific events (sources, tool calls) arrive as extra frames that carry a top-level docsgpt key on an otherwise-empty chunk — standard clients ignore them.

stream = client.chat.completions.create(
    model="docsgpt-agent",
    stream=True,
    messages=[{"role": "user", "content": "Explain vector search"}],
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Set "stream_options": {"include_usage": true} to get a chunk with usage (prompt, completion and total tokens, summed over every model call of the turn). It arrives just before the chunk that carries finish_reason, not after it as in OpenAI’s API, so read it wherever it appears rather than only from the last chunk. Non-streaming responses always include usage. When the model provider reports its prompt cache, usage also has prompt_tokens_details with cached_tokens (prompt tokens read from the cache) and, where the provider reports it, cache_write_tokens (prompt tokens written to it), summed the same way.

Sampling Parameters

Standard OpenAI sampling parameters are forwarded to the model. When omitted, the agent’s configured defaults apply. Supported: temperature, max_tokens (or max_completion_tokens), top_p, frequency_penalty, presence_penalty, stop, seed. When the request has tools, tool_choice and parallel_tool_calls are forwarded too.

Options DocsGPT can’t honor are rejected with HTTP 400 (invalid_request_error) rather than ignored: n other than 1, logprobs set to anything but false, and a stream_options that is not an object.

{
  "model": "docsgpt-agent",
  "messages": [{"role": "user", "content": "Write a haiku about search"}],
  "temperature": 0.2,
  "max_tokens": 256,
  "seed": 42
}

Structured Output

You can force the model to return JSON matching a schema, using either the OpenAI response_format field or the response_schema convenience field.

{
  "model": "docsgpt-agent",
  "messages": [{"role": "user", "content": "Extract the order id and total"}],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "order",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "order_id": {"type": "string"},
          "total": {"type": "number"}
        },
        "required": ["order_id", "total"]
      }
    }
  }
}

{
  "model": "docsgpt-agent",
  "messages": [{"role": "user", "content": "Extract the order id and total"}],
  "response_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "total": {"type": "number"}
    },
    "required": ["order_id", "total"]
  }
}

• response_format follows OpenAI Structured Outputs. strict defaults to true; set strict: false to relax enforcement.
• response_format: {"type": "json_object"} requests JSON without a fixed schema (the model is steered by the prompt).
• On Claude models the schema is always enforced, even with strict: false. Constraints Claude cannot enforce, such as minimum or maxLength, are passed to the model as hints in the field description. json_object has no Claude equivalent and is ignored.
• response_schema is a DocsGPT convenience: pass a raw JSON Schema object (or a {"schema": {...}} wrapper) directly.

Files and Images

Send files the way the OpenAI API takes them, in the content array of a user message:

{
  "model": "docsgpt-agent",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Compare the two contracts"},
        {"type": "file", "file": {"filename": "contract_a.pdf", "file_data": "data:application/pdf;base64,JVBERi0..."}},
        {"type": "file", "file": {"filename": "contract_b.docx", "file_data": "data:application/vnd.openxmlformats-officedocument.wordprocessingml.document;base64,UEsDB..."}}
      ]
    }
  ]
}

Each file becomes an attachment of the agent’s owner, parsed like a file uploaded in the chat, and the parts are taken out of the turn before the model sees it. Your messages are never rewritten or sent back to you.

• Re-sent files cost nothing extra. Files are matched by their bytes (SHA-256). A file the agent’s owner already has, from this conversation or any other, reuses its parsed text instead of being parsed again, so you can keep re-sending every file on every turn and tool round.
• Only what fits goes into the prompt. Up to half of the model’s context window (ATTACHMENT_BUDGET_SHARE) goes to the turn’s files, in the order you sent them. A file that does not fit whole is shown from the start with a note saying how much is included; the rest are left out of the prompt. Images and PDFs go to models that read them natively (at most ATTACHMENT_MAX_NATIVE_PARTS per turn); other models get the extracted text.
• The model gets a list of the files. A short manifest is added to the user message on the server: one line per file with a reference (F1, F2, …), its size and whether it is in the prompt. Files from earlier turns of a stored conversation stay listed there.
• The model can read the rest itself. When the model takes tools, the turn gets server-side tools to list, read (by token offset, by PDF page, or by the page markers of a text file such as --- page 12 --- lines or form feeds) and search the conversation’s files, and to look at images and scanned pages on vision models. A file’s text is stored for the prompt up to 100,000 tokens; beyond that, the whole extracted text (up to ATTACHMENT_FULL_TEXT_MAX_BYTES) is kept next to the file, so search and reads reach its last page too. They run on the server and never appear in tool_calls or pause your tool loop, even when you send your own tools with similar names. Their activity shows up only in docsgpt frames. A model without tool support is told which files were left out.
• Limits. A file can be up to UPLOAD_MAX_FILE_BYTES (100 MB by default). A file that is larger, is a damaged image, or is a type no parser reads is removed from the request, and the manifest tells the model why it could not be read. A file that could not be stored or parsed in time stays in the request as you sent it. Remote image URLs (https://...) and file_id parts are passed through unchanged.

Reference mode

Some files are better worked on than read whole: a 600-page bundle, a folder of contracts, a spreadsheet to compute over. Set docsgpt.attachments_mode to "reference" and none of the turn’s files go into the prompt, however small. The model gets only the manifest: each file’s ref, name, type, size in tokens and pages, with an instruction to search and read the parts it needs with the attachments tools, or to pass the ref to run_code in inputs when the Code Executor is enabled.

{
  "model": "docsgpt-agent",
  "messages": [
    {"role": "user", "content": [
      {"type": "text", "text": "Which of these leases allow subletting?"},
      {"type": "file", "file": {"filename": "leases.pdf", "file_data": "data:application/pdf;base64,JVBERi0..."}},
      {"type": "file", "file": {"filename": "rent_roll.xlsx", "file_data": "data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,UEsDB..."}}
    ]}
  ],
  "docsgpt": {"attachments_mode": "reference"}
}

• "auto", the default, plans what fits into the prompt as described above. Any other value is rejected with HTTP 400 and "code": "invalid_attachments_mode".
• The mode applies to the files of this turn, sent inline or by id in docsgpt.attachments. Files of earlier turns are never put into the prompt again in either mode. In a client tool loop, send the same docsgpt object on every round, since a round that re-sends the files plans them again.
• It needs a model that takes tools. With a model that doesn’t, the mode is ignored and the files are planned as usual.

Every response for a conversation with files lists them in docsgpt.attachments (when streaming, in a docsgpt frame of "type": "attachments" before the last frames):

[
  {"ref": "F1", "id": "1b1f0a2e-...", "filename": "leases.pdf", "mime_type": "application/pdf", "status": "reference", "size": 4718204, "pages": 612, "tokens": 301544},
  {"ref": "F2", "id": "77c0d9b4-...", "filename": "rent_roll.xlsx", "mime_type": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "status": "reference", "size": 48211, "tokens": 9120}
]

ref is the name the model knows the file by in this conversation; status says what this turn did with it (inline, partial, reference, tool, sandbox, earlier, …). To use the files again, continue the conversation (pass its conversation_id) and refer to them by ref in your message, such as “compare F1 with F2”: they are still listed for the model and its tools, with nothing re-sent. To bring a file into another conversation of the same agent, pass its id in docsgpt.attachments.

Tool Calling (client-side, stateless)

You can register your own tools and execute them on the client. The flow is stateless — OpenAI clients that don’t carry a conversation_id re-send the full message history each turn, and DocsGPT rebuilds the agent from it.

1. Send a request with a tools array.
2. If the agent decides to call a tool, the response comes back with finish_reason: "tool_calls" and a tool_calls array (and content: null).
3. Execute the tool(s) on your side, then re-POST the full message history with the assistant’s tool_calls message followed by role: "tool" result messages.
4. DocsGPT continues the run and returns the final answer.

If you send back the conversation_id of the response (top-level or in docsgpt), DocsGPT resumes the paused run from what it saved at the pause instead of from the messages you re-send. The run keeps its server-side tool calls and their results, its reasoning and its files, so the model doesn’t redo work, and the provider’s prompt cache still covers what the paused round sent.

{
  "model": "docsgpt-agent",
  "messages": [
    {"role": "user", "content": "What's the weather in Paris?"},
    {"role": "assistant", "tool_calls": [
      {"id": "call_1", "type": "function",
       "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"}}
    ]},
    {"role": "tool", "tool_call_id": "call_1", "content": "18°C, clear"}
  ],
  "tools": [ { "type": "function", "function": { "name": "get_weather", "...": "..." } } ]
}

Artifacts

When the agent has the Artifact or Code Executor tool, the files it makes are artifacts. Each completed server-side tool call in docsgpt.tool_calls (when streaming, each docsgpt frame of "type": "tool_call") lists the artifacts that call created or changed:

{
  "id": "0b5c7c1e-6f1a-4a43-9a5e-2c1d8f3b7e10",
  "ref": "A1",
  "filename": "Services Agreement.docx",
  "version": 2,
  "kind": "document",
  "title": "Services Agreement",
  "mime_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
  "size": 18234,
  "sha256": "9f2c...",
  "download_url": "/api/artifacts/0b5c7c1e-6f1a-4a43-9a5e-2c1d8f3b7e10/download?conversation_id=6a1e...&version=2"
}

docsgpt.artifacts lists every artifact the turn created or changed, once each, at its latest version, so you don’t have to collect them from the tool calls. When streaming, the same list arrives in a docsgpt frame of "type": "artifacts" just before the frame that carries finish_reason. A turn that made no files has neither. In a client tool loop, a response can list only what its own round made, so collect the lists across the rounds, keeping the highest version of each id.

Download a file by sending GET to its download_url with the same Authorization: Bearer <agent API key> header as your chat requests:

curl -L -H "Authorization: Bearer $DOCSGPT_AGENT_KEY" \
  "http://localhost:7091/api/artifacts/0b5c.../download?conversation_id=6a1e...&version=2" -o agreement.docx

• The URL never holds a credential. It is a path on the API (prefixed with PUBLIC_API_BASE_URL when that is set) pinned to the version the call produced, so sha256 and size describe exactly the bytes it returns.
• An agent key reads only its own conversations. The download, like GET /api/artifacts/<id>, needs the conversation_id in the URL, which must be a conversation of the key’s agent; anything else returns 403. The key can’t delete or restore artifacts. The older ?api_key= query parameter still works, but query strings end up in access logs, so prefer the header.
• On S3 storage (URL_STRATEGY=s3) the route answers 302 with a presigned bucket URL that is valid for 5 minutes. HTTP clients such as requests, httpx and curl drop the Authorization header when they follow a redirect to another host, which is what S3 expects. Add disposition=url to get {"url": "..."} as JSON instead. With the default URL_STRATEGY=backend the bytes are streamed directly.
• Downloads are served by the ASGI app, as docsgpt api and the Docker images run it.

Errors

A request that does not fit the model’s context window gets the error OpenAI returns for one, with HTTP 400:

{
  "error": {
    "message": "This message and its attached files need about 262,790 tokens, more than the model can take (200,000 tokens). Send fewer or smaller files, or add large documents to the agent's sources instead.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

Text pasted into the last user message is never cut to fit. If that message alone (with the system prompt) is longer than the model can take for one message, the request fails with this error, the same as an oversized file. Earlier messages do not count toward it: they are summarized or dropped as usual. The limit reported is what one message may take, which is less than the full window: part of the window is kept for the answer and the conversation.

When streaming, the size may only be known after the stream has started. The same error object then arrives as the last frame, followed by data: [DONE]. Retrying the same request fails the same way: send fewer or smaller files, shorten pasted text, or add large documents to the agent’s sources.

Reasoning

For models that emit reasoning (“thinking”) tokens, the response surfaces them in a non-standard reasoning_content field (a reasoning_content delta when streaming). Standard clients ignore it; clients that understand it can display the model’s thinking separately from the answer.

Idempotent Retries

Add an Idempotency-Key header so a retried request returns the stored first response instead of re-running the agent (which would duplicate the answer and double-bill tokens).

curl -X POST http://localhost:7091/v1/chat/completions \
  -H "Authorization: Bearer your_agent_api_key" \
  -H "Idempotency-Key: 8f1c...unique-per-request" \
  -H "Content-Type: application/json" \
  -d '{"model":"docsgpt-agent","messages":[{"role":"user","content":"hi"}]}'

• Opt-in — no header means today’s behavior (every request runs).
• Non-streaming only — streaming replay is not supported.
• A completed key replays the cached body (and status) for 24 hours.
• A request with a key whose first attempt is still in flight returns HTTP 409.
• Keys are scoped per agent and capped at 256 characters (oversized keys are rejected).

System Prompt Override

System messages are dropped by default — the agent’s configured prompt is used. To allow callers to override it, enable Allow prompt override in the agent’s Advanced settings.

Conversation Persistence

Conversations are always persisted server-side, and the response includes docsgpt.conversation_id. They never appear in the agent owner’s sidebar — /v1 traffic is stored hidden, so external clients can’t clutter the owner’s conversation list.

A tool-result round that carries no conversation (no conversation_id and no linked session, as with a client that resends the whole transcript) is not stored, so it doesn’t leave orphan rows. docsgpt.persist and the legacy docsgpt.save_conversation flag from older releases have no effect.

Continuing a conversation

By default each request is a new conversation built from the messages you send. To continue a stored one, pass its id in any of these, in order of precedence:

1. the X-DocsGPT-Conversation-ID header;
2. a top-level conversation_id field;
3. docsgpt.conversation_id.

The conversation must belong to the agent bound to your key; otherwise the request fails with HTTP 400 and "code": "conversation_not_found".

In a stored conversation, a new user message sent while a tool round still waits for your results ends that round. The stored turn keeps what ran and records the calls it waited on as never run. Results you post afterwards for those calls, while a later round of the same conversation is waiting, are refused with HTTP 409 and "code": "tool_call_not_pending".

Session headers

Chat-completions requests have no conversation field, so coding clients such as opencode send a stable session header instead. When a request carries no conversation id, DocsGPT reads the first of these it finds and links the session to a conversation:

• the X-DocsGPT-Session-ID header, or docsgpt.session_id in the body;
• the X-Session-ID or X-Session-Affinity header.

Requests with the same session value, the same agent and the same system and developer messages continue the same conversation. The link is kept in Redis, only as a hash of the value, for V1_SESSION_TTL_SECONDS (24 hours by default) after the last request that used it: each request restarts the time limit. If the linked conversation is gone, a new one starts. Without Redis, session headers have no effect.

DocsGPT Extension Fields

DocsGPT adds an optional docsgpt object to both requests and responses for features outside the OpenAI schema.

Request (docsgpt.*):

Response (docsgpt.*):

When streaming, these arrive on otherwise-empty chunks that carry a top-level docsgpt key, so strict OpenAI clients still validate each frame.

Which model answered

The standard model field always echoes the agent, so OpenAI clients see no change. When the agent’s model fails (an upstream 5xx, a rate limit, a rejected request) and a backup model takes over, the answer comes from that backup: docsgpt.model, docsgpt.fallback and docsgpt.reason say so. A tool loop can switch models from one round to the next, so docsgpt.models lists each switch.

When streaming, a frame names the answering model as soon as it starts producing output, and again each time a different model takes over, before the text that model wrote:

{"id": "chatcmpl-...", "object": "chat.completion.chunk", "model": "my-agent",
 "choices": [{"index": 0, "delta": {}, "finish_reason": null}],
 "docsgpt": {"type": "model", "model": "kimi-k3", "provider": "openai_compatible", "fallback": true, "reason": "InternalServerError/500"}}

A fallback that takes over in the middle of a stream starts its answer over, so text streamed before its model frame came from the model named earlier. When a fallback answered any part of a turn, the same list is also stored with the message, as metadata.answered_by.

When to Use Native Endpoints Instead

Use /api/answer or /stream if you need passthrough template variables or sidebar visibility control via visibility. To upload a file once and refer to it by id instead of sending its bytes, use the native /api/store_attachment; the resulting id then works in docsgpt.attachments.