Skip to content

feat: add X-Cagent-Compacting header for session-compaction LLM calls - #3777

Merged
dgageot merged 2 commits into
docker:mainfrom
dgageot:feat/compacting-header
Jul 22, 2026
Merged

feat: add X-Cagent-Compacting header for session-compaction LLM calls#3777
dgageot merged 2 commits into
docker:mainfrom
dgageot:feat/compacting-header

Conversation

@dgageot

@dgageot dgageot commented Jul 21, 2026

Copy link
Copy Markdown
Member

Session-compaction (summary) LLM calls are currently indistinguishable from ordinary chat completions at the AI gateway layer. This means gateway-side policies — rate limiting, routing, cost attribution — cannot treat them differently from user-facing conversation turns. Adding an explicit marker gives the gateway a reliable signal it can act on, mirroring the already-established X-Cagent-GeneratingTitle pattern.

The change introduces an X-Cagent-Compacting: 1 HTTP header on gateway-bound requests that originate from the compactor. A new WithCompacting() option and Compacting() getter are added to pkg/model/provider/options, and FromModelOptions ensures the flag survives provider cloning. GatewayHTTPOptions in the base gateway is refactored to accept *options.ModelOptions instead of a bare generatingTitle bool, which lets it emit both the title and compacting headers from a single options struct and treat a nil pointer as zero options. The anthropic, openai, and gemini gateway clients are updated accordingly, and the compactor clones its summary model with options.WithCompacting(). The DMR client's skip-reconfiguration guard is extended to cover compaction clones, which also override max_tokens.

No breaking changes. The new header is purely additive; gateways that do not inspect it are unaffected.

dgageot added 2 commits July 21, 2026 22:16
Mirrors the existing X-Cagent-GeneratingTitle marker. DMR also skips
model reconfiguration when compacting, matching title-generation behaviour.

Assisted-By: Claude
@dgageot
dgageot requested a review from a team as a code owner July 21, 2026 20:27

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The implementation is clean and correct. The X-Cagent-Compacting header follows the established X-Cagent-GeneratingTitle pattern faithfully across all five touch-points: the new WithCompacting()/Compacting() option pair, FromModelOptions propagation, the refactored GatewayHTTPOptions (nil-safe, emits both headers), the DMR skip-reconfiguration guard extension, and the compactor clone site. No bugs found in the changed code.

@aheritier aheritier added area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/gemini Google Gemini provider support area/providers/docker-model-runner Docker Model Runner (DMR) local inference area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 21, 2026
@dgageot
dgageot merged commit aaaa604 into docker:main Jul 22, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/docker-model-runner Docker Model Runner (DMR) local inference area/providers/gemini Google Gemini provider support area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants