feat: enforce lifecycle.call_timeout for MCP tool calls - #3814
Conversation
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
The timeout enforcement is well-structured: the budget correctly covers both the initial call and the reconnect-retry as a single budget, cleanly distinguishes caller-cancel / call-timeout / generic errors, and the prevents context leaks. The nil-receiver guards on make the pre-nil-check call in safe (matching the pre-existing pattern). Gateway policy threading and schema/doc updates look correct. Test coverage for timeout firing, no-op when unset, parent-cancel priority, retry budgeting, and transport-level abort is comprehensive.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
The implementation is clean and correct. Key design points reviewed:
- Timeout scoping:
callCtxis correctly a child ofctx(never replaces it), so the session/connection context is unaffected by a firedcall_timeout. - Error classification in
classifyCallToolError: properly distinguishes parent cancellation (propagates as-is), own timeout (ctx.Err()==nil && callCtx.Err()==DeadlineExceeded), and generic errors. Parent-deadline-exceeded correctly falls to the generic path. - Reconnect guard:
callCtx.Err() == nilprevents retrying when the timeout budget is already exhausted — correct. - Zero-value default:
if ts.callTimeout > 0guard means unset toolsets are byte-for-byte identical to the prior behavior. PolicyFromConfignil-ordering: callingcfg.EffectiveCallTimeout()before the nil guard matches the pre-existingEffectiveStartupTimeout()pattern and is safe (the method handles nil receivers).- Test coverage: timeout fires, no-op when unset, parent cancel wins, reconnect covered by single budget, transport-level abort verified — comprehensive.
startup_timeout has been enforced by the supervisor since v1.94.0 (pkg/tools/lifecycle/supervisor.go's connect() races Connect against a timer and returns ErrInitTimeout on expiry), but agent-schema.json and docs/configuration/tools/index.md still described it as "not yet enforced" / "the eager-startup phase that enforces it ships in a follow-up." Noticed in passing while working on call_timeout enforcement (AIDT-619); unrelated to that change, so splitting it out. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Jed Perkins <jed.perkins@docker.com>
A Harbor Watch investigation stalled 178s on a single hung MCP tool call because docker-agent had no per-call deadline; a stuck call was bounded only by the session context (~600s). This enforces the existing but previously informational call_timeout lifecycle field: callTool wraps the call plus its one reconnect-retry in a context.WithTimeout derived from the toolset's configured call_timeout, classifies a fired timeout distinctly from cancellation/generic errors (with a WARN log + span event, since the original hang logged nothing), and surfaces it to the model as a plain tool error so the session continues. Also wires lifecycle policy through NewGatewayToolset, which previously dropped it entirely for ref:-based toolsets. Default (unset/zero) preserves today's behavior exactly — this is a strictly opt-in, per-toolset change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Jed Perkins <jed.perkins@docker.com>
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
The timeout enforcement design is sound: context.WithTimeout is correctly scoped to a child of the caller's context (so callCtx never outlives ctx), defer cancel() is always called preventing timer leaks, the zero-value default is a genuine no-op, and error classification distinguishes cancellation from timeout correctly in the common case. Tests are thorough and the gateway policy wiring is properly threaded.
Lower-confidence findings (not posted inline)
- [low]
pkg/tools/mcp/mcp.go:803—classifyCallToolErrorcould misclassify a parent-context deadline as a call timeout (confidence: 🟠 weak 52/100). When the parentctxhas its own deadline that fires nearly simultaneously withcallCtx's timeout,ctx.Err() == nilmay not yet reflect the parent's cancellation at the moment the check runs. The misclassification would reportErrCallTimeoutinstead of the session/run deadline to the caller. The impact is limited — it's a label difference, not silent swallowing, and doesn't affect retry behaviour — and the testTestCallToolParentCancelWinsOverTimeoutcovers the cancellation case. Because the practical race window is extremely narrow and the consequence is a label swap rather than data loss or a hang, this sits below the inline threshold.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
This PR correctly enforces lifecycle.call_timeout as a single budget covering both the initial tool call and the reconnect-retry. The timeout guard (if ts.callTimeout > 0), context lifecycle (defer cancel), error sentinel (ErrCallTimeout), span events, and dispatcher translation are all well-implemented. One logic gap was found in classifyCallToolError.
| slog.DebugContext(ctx, "CallTool canceled by context", "tool", toolCall.Function.Name) | ||
| return err | ||
| } | ||
| if ts.callTimeout > 0 && ctx.Err() == nil && errors.Is(callCtx.Err(), context.DeadlineExceeded) { |
There was a problem hiding this comment.
[medium] Parent context DeadlineExceeded is misclassified as a generic tool error
classifyCallToolError uses ctx.Err() == nil to prove that a deadline on callCtx came from the call-timeout timer rather than from the parent context. That works when the parent is canceled (context.Canceled), but it breaks when the parent itself has a deadline that fires first: ctx.Err() then returns context.DeadlineExceeded, making ctx.Err() == nil false. Neither errors.Is(err, context.Canceled) nor ctx.Err() == nil is true, so the function falls through to the generic path and logs at slog.ErrorContext with "Failed to call MCP tool" — misleading for what is really a context-deadline expiry.
Concretely: if the session context (~600 s) has an explicit deadline and that deadline fires while a tool call is in progress, the log will say "Failed to call MCP tool" at ERROR level instead of the expected cancellation/deadline handling.
A minimal fix is to add an explicit branch for the parent-deadline case before the generic fallthrough:
if errors.Is(err, context.Canceled) || errors.Is(ctx.Err(), context.Canceled) {
slog.DebugContext(ctx, "CallTool canceled by context", "tool", toolCall.Function.Name)
return err
}
// Parent context expired with a deadline (not a cancellation): propagate cleanly.
if ctx.Err() != nil {
slog.DebugContext(ctx, "CallTool aborted by parent deadline", "tool", toolCall.Function.Name)
return ctx.Err()
}
if ts.callTimeout > 0 && errors.Is(callCtx.Err(), context.DeadlineExceeded) {
// ... timeout path unchanged ...
}| Confidence | Score |
|---|---|
| 🟢 strong | 100/100 |
Summary
lifecycle.call_timeouttoolset field, which was previously parsed but never acted on ("informational today" per its own doc comment). A Harbor Watch investigation stalled 178s on a single hung MCP tool call because docker-agent had no per-call deadline — a stuck call was bounded only by the session context (~600s).Toolset.callToolnow wraps the call and its one reconnect-retry in a singlecontext.WithTimeoutbudget derived from the toolset's configuredcall_timeout. A fired timeout is classified distinctly from cancellation/generic errors, logs a WARN + emits anmcp.tool_call.timeoutspan event (the original hang logged nothing), and surfaces a plaintool call timed out after Nserror to the model — the session continues normally (Role=tool, IsError=true, same mechanics as any other tool error).lifecycle.PolicythroughNewGatewayToolset, which previously dropped it entirely forref:-based (MCP-catalog) toolsets — a pre-existing gap this change also closes since gateway toolsets share the same constructor plumbing. Verified behaviorally identical to the historical zero-policy default when nolifecycle:block is configured (TestPolicyFromConfig_NilEquivalentToZeroPolicy). Let me know if this was intentional, please.if ts.callTimeout > 0skipscontext.WithTimeoutentirely, socallCtxis literally the caller'sctxand every code path (including the retry guard) behaves byte-for-byte as before. This is a strictly opt-in, per-toolset change, same precedent asstartup_timeoutenforcement in v1.94.0.agent-schema.json) and docs (docs/configuration/tools/index.md) updated to describe the enforcedcall_timeoutsemantics instead of "informational."CHANGELOG.mdin this PR — every existing entry there is versioned/dated and appears to be added in batch at release-cut time (docs: update CHANGELOG.md for vXPRs), not accumulated per-feature-PR, so I'm leaving that to whoever cuts the next release.Ref: AIDT-619
Test plan
go vet ./...,go build ./...,go mod tidy --diffall cleangolangci-lint run(full repo) — 0 issues;go run ./lint .(custom repo lint) — no offensesgo test ./...— all packages pass; two failing tests are pre-existing/environment-specific (SSRF-probe tests behave differently under sandbox network policy), unrelated to this changeTestCallToolTimeoutFires), no-op when unset (TestCallToolNoTimeoutWhenCallTimeoutUnset), parent cancel wins over timeout (TestCallToolParentCancelWinsOverTimeout), timeout covers the reconnect-retry as one budget rather than stacking on top of the 35s retry window (TestCallToolTimeoutCoversReconnectRetry), constructors (incl. gateway) wirecallTimeoutfrom policy,translateErrormodel-facing message, and a real streamable-transport test (TestRemoteClientCallToolAbortsOnContextDeadline) confirming a canceled client context actually aborts the POST and the server observes the cancellationdocker/internal-agentschange (harbor-watch.yamllifecycle.call_timeout: 60s+ config version bump) validated end-to-end against this branch's config parser🤖 Generated with Claude Code