🪙 feat: SDK-Aligned Context-Usage Projection (gauge for window-switch & snapshot-less branches)#13801
Conversation
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f0c8905d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (currentActive) { | ||
| effective = snapshot; |
There was a problem hiding this comment.
When the user changes model or max-context on the currently viewed branch, currentActive is still true because the old snapshot's anchor remains on that branch even though isSubmitting is false. This path therefore assigns effective = snapshot before checking branchSnapshotFresh or using the fetched projection, so the gauge continues to show the old window and prune boundary in the G1 model/window-switch case this change is meant to fix. Gate this branch to live streaming or require the active snapshot's maxContextTokens to match the resolved window before preferring it.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 3a963b1. currentActive now also requires the live snapshot's breakdown.maxContextTokens to match the resolved window (unless streaming) — so a model/window switch on the current branch no longer outranks the projection, and G1 fires as intended.
There was a problem hiding this comment.
Follow-up: I've reverted this window-match guard in 3125171. Verified it was unsafe in the common default case — when no explicit maxContextTokens is set, the SDK snapshot's window is reserve-derived (~0.9·(modelContext − maxOutputTokens), see initialize.ts:1240-1243) while useTokenLimits resolves the raw model context, so snapshot.maxContextTokens === resolvedMax is false for the SAME model and the guard would drop a valid current-branch snapshot post-stream (a regression). The projection is now strictly additive (snapshot-less branches only); reliable G1 detection needs the snapshot to carry its model/window, deferred to the fidelity follow-up.
| const stored = await deps.getMessages( | ||
| { conversationId: params.conversationId }, | ||
| 'messageId parentMessageId tokenCount isCreatedByUser text', |
There was a problem hiding this comment.
For an authenticated request that supplies another user's conversationId and messageId, this resolver loads messages with only { conversationId }, unlike the existing message routes that include user: req.user.id. The endpoint would authorize projecting conversations the caller cannot read and return their message count/token breakdown, so pass the request user through the dependency/filter or verify conversation ownership before resolving the branch.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 3a963b1. The resolver now scopes the lookup to the authenticated user: the controller injects userId: req.user.id and getMessages filters { conversationId, user }, matching the existing message routes — so a caller can't project another user's conversation.
| @@ -0,0 +1,139 @@ | |||
| import { HumanMessage, AIMessage } from '@langchain/core/messages'; | |||
| import { Providers, createTokenCounter, projectAgentContextUsage } from '@librechat/agents'; | |||
There was a problem hiding this comment.
This import depends on projectAgentContextUsage, but this commit does not update api/package.json, packages/api/package.json, or package-lock.json from the currently pinned @librechat/agents 3.2.35; the commit text also says the helper is only in an unpublished agents PR. A normal npm ci/production build will therefore install a package that lacks this export and @librechat/api cannot build/load unless a local symlink is present, so land the SDK release and lockfile bump before wiring the endpoint.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Acknowledged — this is the intended state for now. The PR is a draft precisely because projectAgentContextUsage is only in the unpublished agents PR; testing uses a local symlink. The package.json + lockfile bump lands once that agents release is published (the plan is: confirm the LibreChat fix → publish agents → bump here → real build). Not merging before then.
| endpoint: conversation.endpoint, | ||
| model: conversation.model ?? undefined, | ||
| agentId: conversation.agent_id ?? undefined, | ||
| spec: conversation.spec ?? undefined, | ||
| maxContextTokens: resolvedMax, |
There was a problem hiding this comment.
When a snapshot-less branch uses conversation-level custom instructions (promptPrefix) or a model-spec preset that sets them, the live agent path folds that prompt into the agent instructions, but this projection payload only sends endpoint/model/agent/spec/window and never sends the actual prompt text or keys the query on it. The server then projects with instructions unset unless a persistent agent is loaded, so the gauge undercounts instruction tokens and can keep a stale projection after custom instructions change.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Deferred — same bucket as the documented instruction/tool-fidelity limitation. This first cut targets message-windowing accuracy and only loads instructions for a persistent agent; conversation-level promptPrefix and model-spec preset instructions (and tool schemas) come with the follow-up that reuses the full initializeAgent path, which will also key the query on them. Until then the gauge under-counts instruction overhead for those branches — called out in the resolver JSDoc + PR description.
| const message = branch[i]; | ||
| const text = message.text ?? ''; | ||
| messages.push(message.isCreatedByUser === true ? new HumanMessage(text) : new AIMessage(text)); | ||
| indexTokenCountMap[String(i)] = message.tokenCount ?? 0; |
There was a problem hiding this comment.
For imported or pre-feature messages where tokenCount is absent, this writes an explicit 0 into indexTokenCountMap. Because the SDK receives a count for that message index, it has no way to distinguish “unknown” from a real zero-token message, so projections for the snapshot-less histories this endpoint is meant to fix can report almost no message tokens and render as authoritative instead of recounting or falling back.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 3a963b1. Messages with no stored tokenCount (imported / pre-feature) are now recounted via the tokenizer instead of charging 0, so the snapshot-less histories this endpoint targets don't under-report. Only a present, positive stored count is reused as-is.
| const stored = await deps.getMessages( | ||
| { conversationId: params.conversationId }, | ||
| 'messageId parentMessageId tokenCount isCreatedByUser text', |
There was a problem hiding this comment.
For a branch that has already been compacted/summarized and then needs projection, such as after a window switch invalidates the persisted snapshot, this query only loads raw message fields and gives the SDK the entire parent chain. The next live call would use the saved summary plus the post-summary tail instead of the discarded pre-summary messages, so the projection can prune/count the wrong context and omit the summary tokens entirely; include the summary marker/content or fall back rather than projecting from unsummarized history.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 3a963b1. The resolver now detects a summarized branch (any message with summaryTokenCount > 0) and returns null, so the client falls back to its summary-baseline-aware estimate rather than projecting from the full raw parent chain. Replaying the actual summary boundary (summary + post-summary tail) is the follow-up; this avoids a wrong projection in the meantime.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a963b19e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| * history would prune/count the wrong context and omit the summary. Until the | ||
| * follow-up replays the summary boundary, fall back (null) so the client's | ||
| * summary-baseline-aware estimate handles these branches. */ | ||
| if (branch.some((message) => (message.summaryTokenCount ?? 0) > 0)) { |
There was a problem hiding this comment.
For summarized agent turns the persisted compaction marker used by the client is metadata.summaryUsedTokens (written in packages/api/src/agents/usage.ts and read in client/src/utils/tokens.ts), not summaryTokenCount; this query also does not select metadata. On a summarized branch without a fresh snapshot—such as an aborted turn or after a model/window switch forces projection—this condition stays false, so the projection walks the full raw parent chain and omits the saved summary instead of returning null for the summary-baseline fallback, making the gauge/prune boundary wrong.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c5f2542. Now detects the live path's metadata.summaryUsedTokens marker (and selects metadata) instead of summaryTokenCount, returning null so the summary-baseline-aware estimate handles summarized branches. Smoke-tested.
| let model = params.model; | ||
| let maxContextTokens = params.maxContextTokens; | ||
| if (params.agentId != null && params.agentId !== '') { | ||
| const agent = await deps.getAgent({ id: params.agentId }); |
There was a problem hiding this comment.
When agentId is supplied, this loads the agent solely by public id and uses its private instructions/provider/model for the projection, while the new route is only guarded by JWT/config middleware rather than the canAccessAgentResource checks used by /api/agents/:id. Any authenticated caller who knows an agent id can POST it with their own conversation branch and get projection fields derived from that agent's private configuration; scope this lookup through the same VIEW/USE resource permission path before using the agent.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c5f2542 by removing the server-side agent load entirely. Provider/model/window now come from the client-resolved request, so no agent private config is read here — no access-control surface. Proper agent-scoped resolution (with the access checks) comes with the full-fidelity follow-up that reuses initializeAgent.
| const branchSnapshotFresh = | ||
| branchSnapshot != null && | ||
| (resolvedMax == null || branchSnapshot.breakdown.maxContextTokens === resolvedMax); |
There was a problem hiding this comment.
This freshness check only compares the stored snapshot's context window to the resolved window. If the user switches to another endpoint/model with the same maxContextTokens, the old snapshot is treated as fresh, projectionParams stays null, and the gauge keeps showing the previous model/provider's tokenization and instruction/tool breakdown even though the new query key would otherwise refetch for endpoint/model changes.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid — deferring (not a regression). Detecting a same-window endpoint/model switch needs the snapshot to carry its endpoint/model so freshness can compare more than the window; the persisted blob doesn't yet. A model switch was equally unhandled before this PR (the gauge showed the stale snapshot regardless), so this isn't a regression. It lands with the fidelity follow-up that persists endpoint/model on snapshots. The different-window case is handled now.
There was a problem hiding this comment.
Resolved by reverting the window guard entirely (3125171) rather than extending it — the guard couldn't reliably compare windows (SDK reserve-derived vs client raw) without mis-flagging valid snapshots. The projection now only covers snapshot-less branches; all window/model-switch cases (this one included) defer to the follow-up that persists model/window on the snapshot. Net: no behavior change vs pre-PR for any branch that already has a snapshot.
| return null; | ||
| } | ||
|
|
||
| let instructions: string | undefined; |
There was a problem hiding this comment.
For conversations using a model spec that provides private preset fields such as promptPrefix or instructions, the normal send path applies that spec before initializing the agent, but this resolver never reads params.spec or req.config.modelSpecs; instructions is only populated from an explicit agent. Snapshot-less spec conversations therefore project with missing system/instruction tokens and can prune against the wrong budget until a live snapshot exists.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed by the same change as the agent-access finding: the resolver no longer loads agent OR model-spec config, so there's no partial/special-cased instruction handling. Instructions (agent, promptPrefix, model-spec presets) and tool schemas are uniformly deferred to the full-fidelity follow-up. Not a regression — the estimate fallback doesn't count instructions either.
| [ | ||
| QueryKeys.contextProjection, | ||
| params?.conversationId, | ||
| params?.messageId, | ||
| params?.endpoint, | ||
| params?.model, | ||
| params?.agentId, | ||
| params?.maxContextTokens, | ||
| ], |
There was a problem hiding this comment.
The projection query is cached forever under only the branch tail/config fields. Editing a message invalidates QueryKeys.messages while preserving the same messageId, so snapshot-less or stale-snapshot branches keep rendering the old projected token count/prune boundary until the tail, endpoint, model, agent, or window changes; either include a message revision/cache dependency here or invalidate QueryKeys.contextProjection with message edits.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in c5f2542. The projection query key now includes a branch content revision (sum of the branch's per-message token counts), so an in-place edit that keeps the same tail id still refetches when a message's tokenCount changes.
| @@ -0,0 +1,159 @@ | |||
| import { HumanMessage, AIMessage } from '@langchain/core/messages'; | |||
| import { Providers, createTokenCounter, projectAgentContextUsage } from '@librechat/agents'; | |||
There was a problem hiding this comment.
This new resolver imports projectAgentContextUsage, but the committed dependency metadata still installs @librechat/agents 3.2.35 in api/package.json, packages/api/package.json, and package-lock.json; the commit message says this helper only exists in the unpublished SDK PR. A clean install therefore cannot provide the helper, so the backend projection path will fail to build or call an undefined export and every projection request will fall back/500 until the SDK version is committed with this change.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Acknowledged (same as the prior round) — intended draft state. The package.json + lockfile bump to the published @librechat/agents lands once that SDK release ships; until then testing uses the local symlink and the PR stays draft. Not merging before the bump.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5f2542cdc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,145 @@ | |||
| import { HumanMessage, AIMessage } from '@langchain/core/messages'; | |||
| import { Providers, createTokenCounter, projectAgentContextUsage } from '@librechat/agents'; | |||
There was a problem hiding this comment.
In a clean install from this commit, package-lock.json still resolves @librechat/agents to 3.2.35, whose published exports include createTokenCounter but not projectAgentContextUsage; importing the new helper here therefore breaks the packages/api build or leaves this endpoint returning 500 when the controller is loaded. Please update the SDK dependency/lockfile to the version that actually exports this projection API before wiring the route.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Acknowledged — pending the @librechat/agents publish (the PR is draft for exactly this reason). The package.json + lockfile bump lands when that SDK release ships; until then testing uses the local symlink.
| const branchSnapshotFresh = | ||
| branchSnapshot != null && | ||
| (resolvedMax == null || branchSnapshot.breakdown.maxContextTokens === resolvedMax); |
There was a problem hiding this comment.
When a user switches the current branch to a different model/provider/agent that resolves to the same maxContextTokens as the persisted snapshot, this check still marks the old snapshot fresh and projectionParams stays null, so the query key’s endpoint/model fields never get a chance to refetch. In that scenario the gauge keeps showing the previous model’s tokenizer/prune/instruction breakdown until the next generation; freshness needs to account for the resolved endpoint/model/agent (or force projection on model changes), not just the window size.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Moot as of 3125171 — I reverted the window-match guard entirely. The SDK snapshot window (reserve-derived) doesn't equal the client-resolved raw window in the default no-explicit-maxContextTokens case (initialize.ts:1240-1243), so comparing them would mis-flag valid snapshots. The projection is now strictly additive (snapshot-less branches only); any branch with a snapshot keeps pre-PR behavior, and model/window-switch detection defers to the follow-up that persists model/window on the snapshot.
| return projectAgentContextUsage({ | ||
| agent: { | ||
| agentId: params.agentId ?? 'projection', | ||
| provider: resolveProvider(params.endpoint), | ||
| maxContextTokens, | ||
| }, |
There was a problem hiding this comment.
For agent chats that have instructions, prompt prefixes, skills/MCP/tools, or deferred tool schemas, this synthetic agent only carries provider and window size, so the projected breakdown omits the same instruction/tool overhead that the live path includes. Because the client renders any projection as isEstimate: false, snapshot-less branches and window-switched branches will under-report used context until the next real generation; resolve the full run config here or keep these projections in the estimate/fallback path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 022384a. A first-cut projection carries the SDK's windowing but omits instruction/tool overhead, so it's now surfaced as an ESTIMATE (isEstimate: true, snapshotActive: false, no snapshot) — a better-windowed number than sumBranch, but with the estimate framing (no false-authoritative breakdown, no misleading ~0-tool granular rows). Full overhead resolution comes with the initializeAgent follow-up; real snapshots stay authoritative.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 022384a525
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,145 @@ | |||
| import { HumanMessage, AIMessage } from '@langchain/core/messages'; | |||
| import { Providers, createTokenCounter, projectAgentContextUsage } from '@librechat/agents'; | |||
There was a problem hiding this comment.
In a clean install this resolves @librechat/agents from the unchanged lockfile/package manifests at 3.2.35, but this line imports the new projectAgentContextUsage SDK helper. Unless the agents dependency and lockfile are updated with the SDK release that exports it, @librechat/api cannot load this module (or the route returns 500) outside of a local symlink, so the new projection endpoint is unusable in normal deployments.
Useful? React with 👍 / 👎.
| * endpoint/model/window/revision. */ | ||
| const projectionParams: TContextProjectionRequest | null = | ||
| !isSubmitting && | ||
| branchSnapshot == null && |
There was a problem hiding this comment.
When a user switches model or maxContextTokens on a branch that already has a persisted snapshot, branchSnapshot is non-null, so this guard prevents the projection query from firing; the render path then prefers that old snapshot and takes maxTokens from its baked contextBudget/breakdown.maxContextTokens. The query key includes the new model/window, but it never refetches in this scenario, so the window-switch gauge remains stale until another generation writes a fresh snapshot.
Useful? React with 👍 / 👎.
| const text = message.text ?? ''; | ||
| const lcMessage = | ||
| message.isCreatedByUser === true ? new HumanMessage(text) : new AIMessage(text); |
There was a problem hiding this comment.
For snapshot-less branches that contain prior agent tool calls, the live agent path formats the persisted payload with formatAgentMessages before pruning, which expands assistant content into the AI/tool-message sequence the SDK actually sends. This endpoint selects only text and builds one Human/AI message per DB row, so those tool-result messages are absent from the projected pruning input and the gauge can under-report the next-call context after tool use.
Useful? React with 👍 / 👎.
| snapshot: activeSnapshot, | ||
| snapshotActive: true, | ||
| isEstimate: projected, | ||
| snapshot: projected ? null : (effective as ContextSnapshot), |
There was a problem hiding this comment.
When a projection is used, the gauge computes usedTokens from the projected breakdown, but returning snapshot: null makes Breakdown fall back to branchTotals.input/output. On long snapshot-less branches where projection prunes the context, the popover still shows the unpruned sumBranch rows while the gauge uses the pruned projected total, so the detailed display contradicts the gauge.
Useful? React with 👍 / 👎.
GitNexus: ❌ deploy failedThe deploy failed — the previous index (if any) continues to be served. |
Consumer side of the SDK-aligned context projection (agents `projectAgentContextUsage`). Adds the `/api/endpoints/context-projection` data-provider plumbing (endpoint, service, query key, `TContextProjectionRequest`) and a `useContextProjectionQuery` gated to fire only when no fresh snapshot covers the viewed branch. Wires `useTokenUsage` precedence to: live snapshot → fresh persisted snapshot (window matches the resolved one) → server projection → per-message estimate. A model/window switch marks the baked snapshot stale (its `maxContextTokens` no longer matches) and falls to the projection — closing the gauge's window-switch (G1) and snapshot-less-branch (G2) gaps. Snapshot and projection share the render-relevant fields, so they render uniformly. Backend endpoint + agents version bump land in follow-up commits. Includes the design spec (CONTEXT_PROJECTION_SPEC.md).
POST /api/endpoints/context-projection → resolveContextProjection (packages/api): reconstructs the viewed branch (parent-chain walk from messageId), resolves the agent config (instructions/provider/model/maxContextTokens), reuses LibreChat's stored per-message tokenCounts as the index map (no re-tokenizing), and calls the agents SDK projectAgentContextUsage — no model call. Thin controller injects db.getMessages/db.getAgent; route mirrors /token-config. First cut targets message-windowing accuracy; tool-schema tokens are deferred to a follow-up that reuses the full initializeAgent path.
…ummary) - Guard `currentActive` against a stale window: a model/window switch on the current branch left the live snapshot outranking the projection (G1 didn't fire). Now defers to the projection unless streaming or the window matches. - Scope branch lookups to the authenticated user (`getMessages` filter + injected `userId`) — was loading any conversation by id (IDOR). - Recount messages with no stored `tokenCount` via the tokenizer instead of charging 0, so snapshot-less/imported histories don't under-report. - Fall back (null) for already-summarized branches rather than projecting from the full raw parent chain (the next call would send summary + tail); the client's summary-baseline-aware estimate handles them until a follow-up replays the summary boundary.
…tion - Stop loading agent/model-spec config server-side (closes the agent-access IDOR and the spec-prompt special-casing). Provider/model/window now come from the client-resolved request (`limits.endpoint`/model — the agent's real provider, not the `agents` endpoint, so the tokenizer is right). Agent/spec/ promptPrefix instructions are uniformly deferred to the full-fidelity follow-up. - Detect summarized branches via the live path's `metadata.summaryUsedTokens` marker (was the wrong `summaryTokenCount` field) and fall back to the summary-aware estimate. - Invalidate the projection query on in-place message edits via a branch content `revision` in the cache key (the tail id is unchanged on edit). Deferred (valid, not a regression): same-window endpoint/model switch keeps a window-matched snapshot — needs endpoint/model persisted on the snapshot, which lands with the fidelity follow-up. Smoke-tested: fits / prunes / summarized→null / no-window→null.
Revert the G1 window-match guard on the live/branch snapshot. When no explicit maxContextTokens is set (the common default), the SDK's snapshot window is reserve-derived (~0.9·(modelContext − maxOutputTokens)) while useTokenLimits resolves the raw model context — so `snapshot.maxContextTokens === resolvedMax` is false for the SAME model, and the guard would wrongly drop a valid current-branch snapshot to projection/estimate post-stream (a regression in the default case, per initialize.ts:1240-1243). The projection now activates ONLY for snapshot-less branches (G2): the precedence is live snapshot → persisted branch snapshot → projection → estimate, where the first two are byte-for-byte the prior behavior and the projection just slots ahead of the estimate. Window/model-switch (G1) detection needs the snapshot to carry its model/window and defers to the fidelity follow-up.
A first-cut projection carries the SDK's windowing but omits instruction/tool overhead, so rendering it as `isEstimate: false` showed a confident under-count for snapshot-less branches. Mark projection-sourced views `isEstimate: true` + `snapshotActive: false` (and drop the snapshot field) so they present as a better estimate than sumBranch — improved used/window number, estimate framing, no misleading granular breakdown with ~0 tools. Real snapshots stay authoritative. (Codex round 3, projection.ts:139.)
2a936d2 to
2f3281e
Compare
…kage-lock.json and related package.json files
Inside the ternary, branchSnapshot is narrowed to null (the gate is ), so accessed a property on (frontend typecheck failure). It was also dead — there is never a snapshot to seed from in this branch — so just remove it.
This reverts commit 4cdb862.
… & snapshot-less branches) (danny-avila#13801) * 🪙 feat: Context-usage projection — data-provider + client wiring Consumer side of the SDK-aligned context projection (agents `projectAgentContextUsage`). Adds the `/api/endpoints/context-projection` data-provider plumbing (endpoint, service, query key, `TContextProjectionRequest`) and a `useContextProjectionQuery` gated to fire only when no fresh snapshot covers the viewed branch. Wires `useTokenUsage` precedence to: live snapshot → fresh persisted snapshot (window matches the resolved one) → server projection → per-message estimate. A model/window switch marks the baked snapshot stale (its `maxContextTokens` no longer matches) and falls to the projection — closing the gauge's window-switch (G1) and snapshot-less-branch (G2) gaps. Snapshot and projection share the render-relevant fields, so they render uniformly. Backend endpoint + agents version bump land in follow-up commits. Includes the design spec (CONTEXT_PROJECTION_SPEC.md). * 🪙 feat: Context-projection backend endpoint POST /api/endpoints/context-projection → resolveContextProjection (packages/api): reconstructs the viewed branch (parent-chain walk from messageId), resolves the agent config (instructions/provider/model/maxContextTokens), reuses LibreChat's stored per-message tokenCounts as the index map (no re-tokenizing), and calls the agents SDK projectAgentContextUsage — no model call. Thin controller injects db.getMessages/db.getAgent; route mirrors /token-config. First cut targets message-windowing accuracy; tool-schema tokens are deferred to a follow-up that reuses the full initializeAgent path. * 🩹 fix: Codex review on context projection (G1 guard, IDOR, recount, summary) - Guard `currentActive` against a stale window: a model/window switch on the current branch left the live snapshot outranking the projection (G1 didn't fire). Now defers to the projection unless streaming or the window matches. - Scope branch lookups to the authenticated user (`getMessages` filter + injected `userId`) — was loading any conversation by id (IDOR). - Recount messages with no stored `tokenCount` via the tokenizer instead of charging 0, so snapshot-less/imported histories don't under-report. - Fall back (null) for already-summarized branches rather than projecting from the full raw parent chain (the next call would send summary + tail); the client's summary-baseline-aware estimate handles them until a follow-up replays the summary boundary. * 🩹 fix: Codex round 2 — drop agent load, summary marker, edit-invalidation - Stop loading agent/model-spec config server-side (closes the agent-access IDOR and the spec-prompt special-casing). Provider/model/window now come from the client-resolved request (`limits.endpoint`/model — the agent's real provider, not the `agents` endpoint, so the tokenizer is right). Agent/spec/ promptPrefix instructions are uniformly deferred to the full-fidelity follow-up. - Detect summarized branches via the live path's `metadata.summaryUsedTokens` marker (was the wrong `summaryTokenCount` field) and fall back to the summary-aware estimate. - Invalidate the projection query on in-place message edits via a branch content `revision` in the cache key (the tail id is unchanged on edit). Deferred (valid, not a regression): same-window endpoint/model switch keeps a window-matched snapshot — needs endpoint/model persisted on the snapshot, which lands with the fidelity follow-up. Smoke-tested: fits / prunes / summarized→null / no-window→null. * 🛡️ fix: make context projection strictly additive (no-regression) Revert the G1 window-match guard on the live/branch snapshot. When no explicit maxContextTokens is set (the common default), the SDK's snapshot window is reserve-derived (~0.9·(modelContext − maxOutputTokens)) while useTokenLimits resolves the raw model context — so `snapshot.maxContextTokens === resolvedMax` is false for the SAME model, and the guard would wrongly drop a valid current-branch snapshot to projection/estimate post-stream (a regression in the default case, per initialize.ts:1240-1243). The projection now activates ONLY for snapshot-less branches (G2): the precedence is live snapshot → persisted branch snapshot → projection → estimate, where the first two are byte-for-byte the prior behavior and the projection just slots ahead of the estimate. Window/model-switch (G1) detection needs the snapshot to carry its model/window and defers to the fidelity follow-up. * 🩹 fix: surface projections as estimates, not authoritative snapshots A first-cut projection carries the SDK's windowing but omits instruction/tool overhead, so rendering it as `isEstimate: false` showed a confident under-count for snapshot-less branches. Mark projection-sourced views `isEstimate: true` + `snapshotActive: false` (and drop the snapshot field) so they present as a better estimate than sumBranch — improved used/window number, estimate framing, no misleading granular breakdown with ~0 tools. Real snapshots stay authoritative. (Codex round 3, projection.ts:139.) * 🧹 chore: drop CONTEXT_PROJECTION_SPEC.md from the PR * 🎨 style: fix import-sort order in projection.ts (CI sort-imports check) * 🔧 chore: update @librechat/agents dependency to version 3.2.36 in package-lock.json and related package.json files * chore: npm audit fix * 🎨 style: fix import-sort order in data-service.ts (CI sort-imports check) * 🩹 fix: drop dead calibrationRatio in projectionParams (tsc never error) Inside the ternary, branchSnapshot is narrowed to null (the gate is ), so accessed a property on (frontend typecheck failure). It was also dead — there is never a snapshot to seed from in this branch — so just remove it. * Revert "chore: npm audit fix" This reverts commit 4cdb862.
… & snapshot-less branches) (danny-avila#13801) * 🪙 feat: Context-usage projection — data-provider + client wiring Consumer side of the SDK-aligned context projection (agents `projectAgentContextUsage`). Adds the `/api/endpoints/context-projection` data-provider plumbing (endpoint, service, query key, `TContextProjectionRequest`) and a `useContextProjectionQuery` gated to fire only when no fresh snapshot covers the viewed branch. Wires `useTokenUsage` precedence to: live snapshot → fresh persisted snapshot (window matches the resolved one) → server projection → per-message estimate. A model/window switch marks the baked snapshot stale (its `maxContextTokens` no longer matches) and falls to the projection — closing the gauge's window-switch (G1) and snapshot-less-branch (G2) gaps. Snapshot and projection share the render-relevant fields, so they render uniformly. Backend endpoint + agents version bump land in follow-up commits. Includes the design spec (CONTEXT_PROJECTION_SPEC.md). * 🪙 feat: Context-projection backend endpoint POST /api/endpoints/context-projection → resolveContextProjection (packages/api): reconstructs the viewed branch (parent-chain walk from messageId), resolves the agent config (instructions/provider/model/maxContextTokens), reuses LibreChat's stored per-message tokenCounts as the index map (no re-tokenizing), and calls the agents SDK projectAgentContextUsage — no model call. Thin controller injects db.getMessages/db.getAgent; route mirrors /token-config. First cut targets message-windowing accuracy; tool-schema tokens are deferred to a follow-up that reuses the full initializeAgent path. * 🩹 fix: Codex review on context projection (G1 guard, IDOR, recount, summary) - Guard `currentActive` against a stale window: a model/window switch on the current branch left the live snapshot outranking the projection (G1 didn't fire). Now defers to the projection unless streaming or the window matches. - Scope branch lookups to the authenticated user (`getMessages` filter + injected `userId`) — was loading any conversation by id (IDOR). - Recount messages with no stored `tokenCount` via the tokenizer instead of charging 0, so snapshot-less/imported histories don't under-report. - Fall back (null) for already-summarized branches rather than projecting from the full raw parent chain (the next call would send summary + tail); the client's summary-baseline-aware estimate handles them until a follow-up replays the summary boundary. * 🩹 fix: Codex round 2 — drop agent load, summary marker, edit-invalidation - Stop loading agent/model-spec config server-side (closes the agent-access IDOR and the spec-prompt special-casing). Provider/model/window now come from the client-resolved request (`limits.endpoint`/model — the agent's real provider, not the `agents` endpoint, so the tokenizer is right). Agent/spec/ promptPrefix instructions are uniformly deferred to the full-fidelity follow-up. - Detect summarized branches via the live path's `metadata.summaryUsedTokens` marker (was the wrong `summaryTokenCount` field) and fall back to the summary-aware estimate. - Invalidate the projection query on in-place message edits via a branch content `revision` in the cache key (the tail id is unchanged on edit). Deferred (valid, not a regression): same-window endpoint/model switch keeps a window-matched snapshot — needs endpoint/model persisted on the snapshot, which lands with the fidelity follow-up. Smoke-tested: fits / prunes / summarized→null / no-window→null. * 🛡️ fix: make context projection strictly additive (no-regression) Revert the G1 window-match guard on the live/branch snapshot. When no explicit maxContextTokens is set (the common default), the SDK's snapshot window is reserve-derived (~0.9·(modelContext − maxOutputTokens)) while useTokenLimits resolves the raw model context — so `snapshot.maxContextTokens === resolvedMax` is false for the SAME model, and the guard would wrongly drop a valid current-branch snapshot to projection/estimate post-stream (a regression in the default case, per initialize.ts:1240-1243). The projection now activates ONLY for snapshot-less branches (G2): the precedence is live snapshot → persisted branch snapshot → projection → estimate, where the first two are byte-for-byte the prior behavior and the projection just slots ahead of the estimate. Window/model-switch (G1) detection needs the snapshot to carry its model/window and defers to the fidelity follow-up. * 🩹 fix: surface projections as estimates, not authoritative snapshots A first-cut projection carries the SDK's windowing but omits instruction/tool overhead, so rendering it as `isEstimate: false` showed a confident under-count for snapshot-less branches. Mark projection-sourced views `isEstimate: true` + `snapshotActive: false` (and drop the snapshot field) so they present as a better estimate than sumBranch — improved used/window number, estimate framing, no misleading granular breakdown with ~0 tools. Real snapshots stay authoritative. (Codex round 3, projection.ts:139.) * 🧹 chore: drop CONTEXT_PROJECTION_SPEC.md from the PR * 🎨 style: fix import-sort order in projection.ts (CI sort-imports check) * 🔧 chore: update @librechat/agents dependency to version 3.2.36 in package-lock.json and related package.json files * chore: npm audit fix * 🎨 style: fix import-sort order in data-service.ts (CI sort-imports check) * 🩹 fix: drop dead calibrationRatio in projectionParams (tsc never error) Inside the ternary, branchSnapshot is narrowed to null (the gate is ), so accessed a property on (frontend typecheck failure). It was also dead — there is never a snapshot to seed from in this branch — so just remove it. * Revert "chore: npm audit fix" This reverts commit 4cdb862.
1 participant
Problem
The context gauge is accurate while streaming and on reload of a turn that emitted a live snapshot. Two states fall back to a client-side
sumBranchestimate that isn't pruning/window-aware:Approach: the SDK is the source of truth in every state
Rather than re-derive pruning client-side (the calibration-bug trap), ask the agents SDK to project "what context would the next call send for this branch under this config" — without invoking the model.
AgentContext.projectContextUsage()+ top-levelprojectAgentContextUsage()helper. Runs the same pruner + budget math the live path uses (createPruneMessages → getTokenBudgetBreakdown → syncBudgetDerivedFields), never mutates the supplied messages, recounts/accepts a token map, applies acalibrationRatioseed (no re-derivation).POST /api/endpoints/context-projection+getContextProjection+TContextProjectionRequest+QueryKeys.contextProjection.useContextProjectionQuery(gated to fire only when no fresh snapshot covers the branch);useTokenUsageprecedence → live snapshot → fresh persisted snapshot (window matches) → projection → estimate, with the G1 window-match guard./api/endpoints/context-projectioncontroller + route +resolveContextProjection(loads the branch, reuses stored per-messagetokenCounts, calls the SDK helper).Status / how to test
npm run build:data-provider).node_modules/@librechat/agents→ the local agentsclaude/context-projectionworktree (built), sinceprojectAgentContextUsageisn't in the published version yet.Known limitation (first cut): the backend projection targets message-windowing accuracy and reuses stored per-message token counts; tool-schema tokens are deferred to a follow-up that reuses full
initializeAgent.Design details:
CONTEXT_PROJECTION_SPEC.md.