Skip to content

feat: add ContextLimit helper to modelinfo#2982

Merged
dgageot merged 2 commits into
docker:mainfrom
dgageot:board/61e4651d475d6baf
Jun 3, 2026
Merged

feat: add ContextLimit helper to modelinfo#2982
dgageot merged 2 commits into
docker:mainfrom
dgageot:board/61e4651d475d6baf

Conversation

@dgageot
Copy link
Copy Markdown
Member

@dgageot dgageot commented Jun 3, 2026

Model context-window handling was scattered across providers. Each provider either hardcoded a limit (Anthropic had anthropicContextLimit embedded in its package) or lacked a fallback when the models.dev store was unavailable, risking silent failures or degraded behavior in production environments.

This change centralizes context-window logic into a new ContextLimit(ctx, store, id, fallback) helper in pkg/modelinfo. It first attempts to source the window from the models.dev store with a bounded timeout, then falls back to a provided default. A new DefaultAnthropicContextLimit constant (200000) replaces Anthropic's hardcoded function, and all five provider packages now use modelinfo.ContextLimit consistently.

Additionally, LoadCaps now accepts a context.Context as its first parameter so callers can implement proper timeout behavior. This is wrapped internally with loadCapsTimeout to bound the underlying models.dev load operation, though callers should understand that the timeout does not cover contention from the shared store lock.

dgageot added 2 commits June 3, 2026 10:23
Add ContextLimit to centralize model context-window logic, sourcing from
models.dev with a fallback. Also update LoadCaps to accept context.Context
for proper timeout handling.
@dgageot dgageot requested a review from a team as a code owner June 3, 2026 08:44
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The ContextLimit helper is implemented correctly. The nil-store early-return, bounded context.WithTimeout, and <= 0 fallback guard all work as intended. The DefaultAnthropicContextLimit constant migration is type-safe (untyped integer constant implicitly converts to int64). All five provider convertDocument call-sites are updated consistently. The LoadCaps context threading is correct. Tests cover the happy path, unknown model, and nil-store cases.

No high or medium severity bugs were found in the changed code.

@dgageot dgageot merged commit 8e9b755 into docker:main Jun 3, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants