RAG Tool

Give your agents access to document knowledge bases with background indexing, multiple retrieval strategies, and hybrid search.

Overview

The rag toolset lets agents search through your documents to find relevant information before responding. Knowledge bases are declared once at the top of the config under rag: and then referenced from any agent via type: rag, ref: <name>. Docker Agent supports:

• Background indexing — Files are indexed automatically and re-indexed on change
• Multiple strategies — Semantic embeddings, BM25 keyword search, and LLM-enhanced search
• Hybrid search — Combine strategies with result fusion for best results
• Reranking — Re-score results with specialized models for improved relevance

RAG is the strategy to reach for when a document collection is too large to inline directly, or gets queried repeatedly across turns/sessions — see Choosing a Large-Input Strategy for how it compares to @//attach attachments and prompt files.

Quick Start

rag:
  my_docs:
    tool:
      description: "Technical documentation"
    docs: [./documents, ./some-doc.md]
    strategies:
      - type: chunked-embeddings
        embedding_model: openai/text-embedding-3-small
        database: ./docs.db
        vector_dimensions: 1536

agents:
  root:
    model: openai/gpt-4o
    instruction: |
      You have access to a knowledge base. Use it to answer questions.
    toolsets:
      - type: rag
        ref: my_docs

Retrieval Strategies

Chunked Embeddings (Semantic Search)

Uses embedding models to find semantically similar content. Best for understanding intent, synonyms, and paraphrasing.

strategies:
  - type: chunked-embeddings
    embedding_model: openai/text-embedding-3-small
    database: ./vector.db
    vector_dimensions: 1536
    similarity_metric: cosine_similarity
    threshold: 0.5
    limit: 10
    embedding_batch_size: 50
    chunking:
      size: 1000
      overlap: 100

Semantic Embeddings (LLM-Enhanced)

Uses an LLM to generate semantic summaries of each chunk before embedding, capturing meaning and intent. Best for code search and understanding implementations.

strategies:
  - type: semantic-embeddings
    embedding_model: openai/text-embedding-3-small
    vector_dimensions: 1536
    chat_model: openai/gpt-4o-mini
    database: ./semantic.db
    ast_context: true # include AST metadata
    chunking:
      size: 1000
      code_aware: true # AST-aware chunking

BM25 (Keyword Search)

Traditional keyword matching using the BM25 algorithm. Best for exact terms, technical jargon, and code identifiers.

strategies:
  - type: bm25
    database: ./bm25.db
    k1: 1.5 # term frequency saturation
    b: 0.75 # length normalization
    threshold: 0.3
    limit: 10
    chunking:
      size: 1000
      overlap: 100

Hybrid Search

Combine multiple strategies for best results. Strategies run in parallel and results are fused together:

rag:
  hybrid:
    docs: [./docs]
    strategies:
      - type: chunked-embeddings
        embedding_model: openai/text-embedding-3-small
        database: ./vector.db
        vector_dimensions: 1536
        limit: 20
        chunking: { size: 1000, overlap: 100 }
      - type: bm25
        database: ./bm25.db
        limit: 15
        chunking: { size: 1000, overlap: 100 }
    results:
      fusion:
        strategy: rrf # Reciprocal Rank Fusion
        k: 60
      deduplicate: true
      limit: 5

Fusion Strategies

Reranking

Re-score retrieved documents with a specialized model to improve relevance:

results:
  reranking:
    model: openai/gpt-4o-mini
    top_k: 10 # only rerank top 10
    threshold: 0.3 # minimum score after reranking
    criteria: |
      Prioritize official documentation over blog posts.
      Prefer recent information and practical examples.
  limit: 5

Supported reranking providers: DMR (native /rerank endpoint), OpenAI, Anthropic, Gemini.

Code-Aware Chunking

For source code, enable AST-based chunking to keep functions and methods intact:

chunking:
  size: 2000
  code_aware: true # Uses tree-sitter for AST-based chunking

Indexing failures, retries and backoff

When a knowledge-base fails to start — because the embedding provider is rate-limiting your requests or returning a transient server error — Docker Agent spaces out retry attempts with bounded exponential backoff instead of hammering the provider on every agent turn.

What triggers backoff

For RAG indexing, the toolset gate arms on two different signals depending on the failure:

• HTTP 429 (rate limit) aborts the whole indexing run at the first failure — continuing would just keep hammering a provider that asked for backoff — and that abort error reaches the gate immediately.
• HTTP 408 (request timeout) and a fixed 5xx set (500, 502, 503, 504, 529) are otherwise handled per-file: a single file's transient failure is skipped so the run can keep indexing the rest. But if no file in the run is successfully indexed — every attempted file hit one of these retryable statuses — the run treats that as a sustained backend failure rather than a one-off hiccup, and surfaces the error so the gate arms on the next turn (fixed in #4097; previously only 429 reached the gate).

Retry policy and parameters

The backoff is bounded exponential with additive jitter:

• Base delay: 15 seconds
• Maximum delay: up to ~6 minutes (5-minute cap plus up to 20% additive jitter)
• Growth: doubles after each consecutive retryable failure (15s → 30s → 1m → 2m → 4m → 5m)
• Jitter: each wait is a random value in [nominal, 1.2×nominal] (additive 0–20%) so concurrent knowledge-base sources spread their retries and avoid hammering the provider together
• Retry-After override: if the embedding provider responds with a Retry-After header, that hint overrides the computed delay (capped at the 5-minute maximum, with the same additive jitter applied to spread concurrent retries)

The gate is a lightweight wall-clock check — it creates no background threads or timers. A Stop command or agent shutdown takes effect immediately regardless of how much of the backoff window remains.

Long indexing runs

Starting any toolset — RAG included — is bounded by a 30-second wait budget (tools.DefaultStartTimeout): if a toolset's Start has not returned within 30s, the caller stops waiting and the turn proceeds without that toolset's tools, exactly as it would for a wedged MCP server. This budget exists to detect toolsets that never come up; it is not a deadline on indexing itself.

A large knowledge base can legitimately take much longer than 30s to index. Rather than abort in-flight indexing at the 30s mark — which used to discard any embeddings not yet committed for the file being processed — the RAG toolset detaches indexing from the caller's wait budget the same way it already detaches its file watcher. When the 30s budget expires:

• The current turn proceeds without the RAG tool (the existing "taking too long to start" warning and TUI progress events still apply).
• Indexing keeps running in the background.
• Files that finish indexing are persisted atomically per file (a file is always either fully indexed or untouched — see #4073/#4158), so progress is never lost to this budget.
• A later turn's Start call picks the toolset up: if indexing has finished, the tool is available immediately; if it is still running, that turn also proceeds without the tool.

This is a visible behavior change: knowledge bases that used to finish indexing (and thus offer the tool) on the very first turn, within the old 30-second window, now do so on whichever turn happens to land after indexing completes. If indexing takes under 30s, nothing changes.

indexing_timeout bounds indexing itself, independent of the 30s wait budget — it exists only so a hung provider connection cannot pin a knowledge base's indexing lock forever:

rag:
  codebase:
    indexing_timeout: 2h # Go duration; "0s" = unbounded; default 30m
    docs: [./knowledge-base]
    strategies:
      - type: chunked-embeddings
        embedding_model: openai/text-embedding-3-small

• Default: 30m when omitted.
• 0s means no bound. A negative value is rejected at config validation time.
• Expiry is a plain timeout, not a provider error: it does not arm the backoff gate described above, and the next turn's Start resumes indexing immediately rather than waiting out a cooldown. Files already persisted are kept; unfinished files are re-indexed on resume.
• A 429/408/5xx from the provider during a detached indexing run is unaffected by any of this — it still aborts the run and arms the backoff gate exactly as described above.

Operational impact

Before: a rate-limited knowledge base was re-indexed on every agent turn — max_indexing_concurrency × max_embedding_concurrency concurrent provider calls could relaunch within milliseconds, easily tripping rate limits for both the knowledge base and the agent's own model calls.

After: retries are spaced out and jittered so the provider has room to recover before the next attempt. The agent continues working with any other toolsets that are not affected.

What you will see

• Docker Agent logs a single warning when a knowledge base first fails to start. Repeated failures in between are logged at debug level only, so you are not flooded with alerts on every turn. Recovery is intentionally silent — the tool appearing in the agent's tool list is the signal that indexing succeeded.
• The knowledge-base tool does not appear in the agent's tool list until indexing succeeds. A successful start is silent — the tool is listed and the agent uses it.

Troubleshooting repeated 429/5xx/408 errors

If you see persistent 429, 5xx, or 408 errors in the logs:

1. Check provider rate limits. Your embedding API key may have a low requests-per-minute quota. Upgrading the plan or using a different API key can help.

2. Reduce concurrency. The chunked-embeddings and semantic-embeddings strategies accept max_indexing_concurrency (default 3) and max_embedding_concurrency (default 3) parameters. Lowering these reduces simultaneous requests:

   rag:
     docs:
       docs: [./knowledge-base]
       strategies:
         - type: chunked-embeddings
           max_indexing_concurrency: 1
           max_embedding_concurrency: 1
   

3. Use a model with a higher quota. Some providers offer higher rate limits on specific embedding model tiers.

Debugging RAG

Enable debug logging to see retrieval details:

$ docker agent run config.yaml --debug --log-file debug.log

Look for log tags: [RAG Manager], [Chunked-Embeddings Strategy], [BM25 Strategy], [RRF Fusion], [Reranker].

Permanent model errors abort early. If the embedding model, semantic-LLM model, or reranking model returns a permanent error (HTTP 400, 401, 404, or 429 — invalid config, bad auth, unknown model, or rate limit), Docker Agent treats the model configuration as invalid and stops immediately rather than retrying doomed requests:

• Indexing — the entire indexing run is aborted after the first permanent failure (including 429). The error is surfaced in the logs so you know immediately if a model name or API key is wrong, rather than silently producing incomplete results.
• Reranking — a permanent error (including 429) permanently disables the reranker for the lifetime of the manager. Subsequent queries fall back to un-reranked results. Only transient errors (5xx, timeouts) fall back and retry on the next query.

Configuration Reference

Top-Level RAG Fields

Chunked-Embeddings Strategy

Semantic-Embeddings Strategy

BM25 Strategy

Results (Post-Processing)