Skip to content

feat(metering): route compaction model calls through the canonical seam - #1877

Merged
Astro-Han merged 7 commits into
apache:mainfrom
ARE404:are404/feat-compact-call-routing
Aug 3, 2026
Merged

feat(metering): route compaction model calls through the canonical seam#1877
Astro-Han merged 7 commits into
apache:mainfrom
ARE404:are404/feat-compact-call-routing

Conversation

@ARE404

@ARE404 ARE404 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Routes the last two model-call kinds that #1679 left outside the canonical seam. After #1755 every main send settles into a ModelCallAttempt, but history_compact and semantic_compact were still metered by hand into the frozen LlmCallRecord table — so compaction was the one place where usage was recorded twice as two different shapes, and where a cost that could not say whether it was real landed as a number.

Both now settle the way a main send does: one canonical record per physical provider request, carrying usageBasis and costBasis.

Accounting is supplied per call, by the backend. The Host wires the summarizers once at composition time and cannot know runId — that is per-turn state only AiSdkBackend holds. Rather than plumbing a turn-to-run resolver down through the kernel and BackendFactoryContext so the Host could look up something the backend already has, AiSdkBackend.modelCallAccounting(callKind) builds the identity and the caller passes it in at the moment the call is made. Adding kinds this way removed code: the main send path now uses the same factory it does.

  • history_compact (6235a9a9) — the summarizer already built a ProviderRequestTracker for capture and attempt diagnostics; HistoryCompactSummaryInput gained an optional accounting that AiSdkCompaction supplies at both call sites.
  • semantic_compact (f33a4e4a) — this one had no tracker at all. ModelAdapter.generateCompactSummary now takes an optional tracker and wraps the model with a wrapGenerate middleware calling trackGenerate, the exact mirror of what startStream already does with wrapStream — in the one place that already owns "attach a tracker to a model". The backend hands compaction a built tracker rather than the capture, attempt, and id sinks it is made of: a half-wired tracker is what produces records nothing can attribute. Success, failure, and abort all settle inside the tracker, so the hand-rolled try/catch that recorded three ways is gone.
  • Legacy writers go out in the same commits as the kinds they served — keeping both would make compaction the double-metered path this work removes everywhere else.

Refs #1679.

Correction to #1679: recordLlmCall cannot be deleted yet

The RFC says that once both compaction kinds are routed, the send-level recordLlmCall has no writers left and provenance.legacyRecords falls to zero as the old table ages out of the queried range. Both statements are wrong, and I would rather say so here than have the issue keep asserting it.

goal_evaluation is the fourth ModelCallKind, and createHostGoalEvaluator still writes it to the frozen table. It cannot follow the compaction kinds through this seam as-is: a ModelCallAttempt is identified by (sessionId, runId, turnId), and the Host evaluates a goal against a sessionId alone — there is no run or turn at that layer to attribute the call to. Giving it one is a design question for the RFC, not something to smuggle into a routing change.

So this PR deletes only what genuinely has no writer left: AiSdkCompactionCapabilities.recordLlmCall and its LlmTelemetryRecorder type, the Host and Desktop wiring into that backend field (both were feeding a field nothing read any more), and five backend tests that collected records nothing asserted on. The recordLlmCall helper, its exports, and TelemetryIndexWriter.recordLlmCall stay for the goal evaluator.

Practical consequence for reviewers of the read path: the merged read path stays load-bearing. legacyRecords will not reach zero while goal evaluations keep landing in usage_llm_calls. Happy to open a follow-up issue for routing goal_evaluation if you agree that's the right next slice.

Known gaps, stated rather than hidden

Two things this PR deliberately does not do, both surfaced in review:

  • errorClass is not carried on canonical records. The old semantic_compact row had one. No canonical record has one — for any kind, including main, whose errorClass goes to the send_diagnostics_recorded run-trace event rather than to ModelCallAttempt. The field is declared in the contract and populated by nobody. Implementing it means classifying at the tracker for all four kinds, which is a [RFC] Canonical model-call accounting: one ModelCallAttempt record per real provider request #1679 change, not a routing one.
  • Manual compactHistory settles nothing. Fixed in b5280caa — and the claim that nothing reaches that path was wrong. Desktop sessions:compact and CLI /compact both reach RuntimeKernel.compactSession(). BackendCompactHistoryInput.runId is now required (silence is the failure mode, so an optional field would be one a caller can forget), the kernel passes run.runId, and a SessionManager.compactSession() test pins exactly one history_compact record carrying it.

Behaviour change worth flagging

The old semantic_compact row copied the AI SDK's normalized cacheRead through as a cache hit. The canonical record attributes cache tokens only when the provider's own payload claims them (strictProviderRequestUsage), so a provider that reports no native cache fields now yields an absent field rather than a number no provider ever said. This is the same rule every other kind has followed since #1687; compaction was the outlier.

Verification

  • @maka/runtime 2678/2691 — the 4 failures are this machine's rg-is-a-shell-function sandbox noise, identical on the parent commit.
  • @maka/runtime-host 524/524.
  • @maka/desktop 1090/1128 — the 38 failures are this worktree's missing Astryx peer dep; verified identical on the parent commit by stashing the change and re-running, so they are not from this PR.
  • npm run format:check and node scripts/check-console.mjs clean.
  • Typecheck clean for @maka/runtime and @maka/runtime-host after rebuilding @maka/core / @maka/storage / @maka/runtime dists onto the rebased base.
  • Rebased onto c61550d7; the one overlapping file with upstream (apps/desktop/src/main/session-stream.ts) merged without conflict.

Test evidence for the routing itself is behavioural, not incidental: the history_compact summarizer test and the backend's semantic-compact test both now decode the canonical ModelCallAttempt and assert run attribution, usageBasis, and costBasis — the old assertions on the frozen table's row shape are gone rather than kept alongside.

🤖 Generated with Claude Code

@Astro-Han

Copy link
Copy Markdown
Contributor

Verified the routing is correct: no double metering, each summarizer call settles exactly once through the tracker, the main-send refactor onto createProviderRequestTracker is field-for-field equivalent to the old inline construction, and the deletions are complete. goal_evaluation is genuinely the last writer to the frozen table, so keeping recordLlmCall for it is right. Runtime and runtime-host suites are green on this branch.

P0: none.

P1

  • Test gap: the deleted "does not record semantic compact usage when provider usage is unavailable" pinned a private method that no longer exists, so deleting it was right. But the replacement behavior, usage missing now settles a canonical record with usageBasis: 'missing', has no test anywhere; the missing branch of resolveUsageBasis is never driven. Please add one.

P2

  • Metering is now gated on the optional capture seam. createProviderRequestTracker returns undefined when recordProviderRequestCapture is absent, while on main the two compaction paths were metered independently of capture (desktop and host wired recordLlmCall unconditionally). In a "capture off, compaction on" config, semantic_compact now records nothing. The canonical record treats captureArtifactId as optional; metering shouldn't depend on a diagnostic sink. Suggest building the tracker when persistCapture || accounting and making capture optional inside the tracker.
  • The "one place that owns attach a tracker to a model" claim doesn't hold. There are now three wrapLanguageModel sites: startStream (wrapStream), buildLlmHistorySummarizer (wrapGenerate), and the new one in generateCompactSummary. The last two are near-identical, and ProviderMiddlewareGenerateInput is declared twice. Extracting a shared helper (e.g. withGenerateTracking(model, tracker, abortSignal)) would make that claim true in fact.
  • Manual compactHistory loses metering. BackendCompactHistoryInput has no runId, compactHistory runs outside send(), and emitModelCallAttempt returns when runId is undefined, so a manual compact settles nothing while the mid-turn path records. Nothing reaches it today, but the two history_compact paths now behave differently. Worth a runId in the input or an explicit statement.
  • Error/abort paths: no compaction test exercises a failed or aborted summarizer, and the old semantic record's errorClass is gone. Nothing sets it on the canonical record for compaction (the main path still does). Either implement it or state it's deliberately dropped.
  • Test gap: the cache-attribution test pins a mock shape (no raw) that no real adapter produces; real adapters always carry raw, so the "provider claims cache tokens" branch is untested end-to-end. Adding raw to the mock usage would cover it.
  • Test gap: dry-run modes (validate_only, prepare_step_dry_run) really call the summarizer and settle billing records; worth pinning that behavior explicitly.
  • Test gap: the history_compact backend glue (modelCallAccounting('history_compact') in compactHistory and midTurnCompactHistorySummaries) has no end-to-end test; runId resolution is stubbed in the unit test.

P3

  • Dead import normalizeAiSdkUsage in ai-sdk-compaction.ts, introduced by the final cleanup commit, plus a pre-existing one, llmCallUsageFields in ai-sdk-backend.ts.
  • The comment in apps/desktop/src/main/usage-ipc-main.ts still says the frozen table "still receives the compaction calls"; only goal evaluations remain.
  • history_compact canonical records lack contextWindow and always carry step: 0, while semantic_compact records carry the real send step and window. The two compaction kinds now record at different quality inside the same seam.
  • On desktop, the summarizer never gets providerRequestTracking, so the accounting computed for history_compact is computed and discarded.
  • When policy.summarizerModel differs from the main model, cost is resolved against the main model's rates while modelId on the record is the summarizer model. Pre-existing, but the record now carries pricingRates making it auditable.

Happy to file a follow-up for routing goal_evaluation if you agree that's the right next slice.

ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 2, 2026
… it settles

Review round 1 on apache#1877. The P1 and the P2s that were about this PR's own
behaviour, plus every P3.

**Metering no longer depends on a diagnostic.** `createProviderRequestTracker`
returned undefined without `recordProviderRequestCapture`, so a deployment with
capture off — a reachable config; both hosts wire it conditionally — silently
stopped metering compaction, which the deleted `recordLlmCall` had metered
unconditionally. Capture is now optional inside the tracker: `preparedCapture`
is pure, so `requestHash`, `requestBytes`, and `segments` survive without a
sink, and only the artifact join keys (`captureId`, `captureArtifactId`) go
absent. The tracker is built when there is either sink to feed. The Host now
wires the summarizer's tracking unconditionally too, and the Desktop wires it at
all — it never had, so the accounting the backend computed for a Desktop history
compaction was handed to a summarizer that had nowhere to settle it.

**The "one place owns attaching a tracker" claim is now true rather than
asserted.** `withProviderGenerateTracking` is shared by `generateCompactSummary`
and `buildLlmHistorySummarizer`; `ProviderMiddlewareGenerateInput` is declared
once. Also fixed: history_compact records carried no `contextWindow` while
semantic_compact did.

**Tests for what was only claimed:**

- `usageBasis: 'missing'` — the branch had no test anywhere. A call the provider
  reported no usage for records `missing`, not zero tokens, and stays unpriced
  whatever the resolver would have said.
- Metering with capture switched off, asserting the attempt still carries the
  locally-computed request shape and no artifact ids.
- Cache attribution end-to-end: the semantic mock now ships a `raw` provider
  payload, so the assertion moved from "absent because this mock claims nothing"
  to `cacheReadInputTokens: 2` through the provider branch. The no-`raw` rule
  stays pinned at the unit level.
- A dry-run (`validate_only`) semantic compaction really is a billed call: the
  summarizer runs to completion and only then is its block refused. Confirmed by
  the test, which is why it is worth pinning — a mode named "dry run" that bills
  is what a later reader would assume otherwise.
- Mid-turn history compaction settles a canonical record end-to-end through the
  backend glue, with the real summarizer against a mock provider, asserting the
  live `runId` resolves — a stubbed resolver cannot show that.

P3 cleanups: two dead imports, and the Usage IPC comment that still claimed the
frozen table receives compaction calls.

`@maka/runtime` 2684/2696 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep, unchanged from the parent commit).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404

ARE404 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — fixed in c96498f4. Everything except three items, and one of those is a correction back at you.

P1 — usageBasis: 'missing' untested. Correct, and the gap was wider than the deleted test: nothing anywhere drove that branch. Added to the tracker suite — a call the provider reported no usage for records missing (not zero tokens) and stays unpriced whatever the cost resolver would have said.

P2 — metering gated on the capture seam. Correct, and it was a regression this PR introduced: both hosts wire capture conditionally, so "capture off, compaction on" is reachable, and recordLlmCall had metered it unconditionally. Took your suggestion. persistCapture is now optional inside the tracker, and it costs less than it looks: preparedCapture is pure, so requestHash / requestBytes / segments survive without a sink and only captureId / captureArtifactId go absent. The tracker is built when either sink is present. Two consequences I went ahead and fixed while there: the Host now wires the summarizer's providerRequestTracking unconditionally, and the Desktop wires it at all — which is your P3 about the Desktop, and it means Desktop history compaction was never metered rather than newly unmetered.

P2 — the "one place" claim. Correct, and it was my commit message asserting a property the code did not have. withProviderGenerateTracking is now shared by both generate sites and ProviderMiddlewareGenerateInput is declared once.

P2 — cache attribution / dry-run / mid-turn glue test gaps. All three added. The dry-run one confirmed your reading: validate_only runs the summarizer to completion and only then refuses the block, so it really does bill. Pinned explicitly, because a mode called "dry run" that charges money is what a later reader would assume otherwise. The mid-turn test goes through the real summarizer against a mock provider so the live runId actually resolves — the point you made about the stub.

P2 — errorClass: I think this one is wrong in its premise. The main path does not set errorClass on the canonical record either. ai-sdk-backend.ts:1869 sets it on trace.sendDiagnostics, a RunTraceEvent. ModelCallAttempt.errorClass is declared in the contract and populated by nobody, for any kind — so compaction is not asymmetric with main here. What is true is narrower: the old semantic_compact row carried an errorClass and no canonical record does. Implementing it properly means classifying at the tracker for all four kinds, which is a #1679 change rather than a routing one. Stating it deliberately dropped for now; happy to take it as the next slice if you'd rather have it sooner.

P2 — manual compactHistory loses metering. Correct. Leaving it as an explicit statement rather than adding runId to BackendCompactHistoryInput in this PR: the input is a core contract with Host and Desktop callers, and nothing reaches the manual path today, so the change would land untested by anything real. Called out in the PR description.

P3. Both dead imports gone (normalizeAiSdkUsage was mine, llmCallUsageFields predates this PR). Usage IPC comment corrected. contextWindow now flows to history_compact records. On step: history_compact keeps step: 0 deliberately — its tracker covers exactly one call, so zero is that call's only step, whereas semantic_compact's tracker spans a turn and its step is the send step the summarization interrupted. Left the summarizerModel pricing mismatch alone as pre-existing; agreed it is more visible now that pricingRates is recorded, and it belongs with whatever routes goal_evaluation.

Yes please on the goal_evaluation follow-up — that's the right next slice, and it is what keeps provenance.legacyRecords from reaching zero.

Verification: @maka/runtime 2684/2696 (3 suites are this machine's rg-as-shell-function noise), @maka/runtime-host 524/524, @maka/desktop 1101/1139 (38 are this worktree's missing Astryx peer dep, unchanged on the parent commit), format:check and check-console clean.

@Astro-Han

Copy link
Copy Markdown
Contributor

One blocker here, plus a pricing issue that I think belongs in this PR.

P1: manual compaction drops the accounting record.

The manual path is active. Desktop sessions:compact and CLI /compact both end up in RuntimeKernel.compactSession(). That method creates an AgentRun, so it already has run.runId, but BackendCompactHistoryInput does not carry it (runtime-kernel.ts:927-983, backend-types.ts:130-135).

AiSdkBackend.currentRunId is only set by send() (ai-sdk-backend.ts:631-637). A manual compactHistory() never sets it. When the summary finishes, modelCallAccounting() cannot resolve a run, and emitModelCallAttempt() quietly returns without writing anything (provider-request-telemetry.ts:498-502).

The old recordLlmCall path did not need a run id, so removing it makes manual history compaction unmetered. The PR description currently says nothing reaches this path, but both Desktop and CLI do.

I think the clean fix is to put runId on BackendCompactHistoryInput, pass run.runId from the kernel, and use that explicit identity when building accounting for the manual call. Please add a SessionManager.compactSession() test that checks for exactly one history_compact record with the correct run id.

P2: a separate semantic summarizer model is still priced as the main model.

The tracker gets summarizerModelId, and the record carries the model that actually handled the request. Cost resolution ignores that id and always looks up ${providerType}:${this.input.modelId} (ai-sdk-backend.ts:2112-2116).

With MAKA_CONTEXT_SEMANTIC_COMPACT_MODEL set, we can therefore store a record for the compact model with pricingRates and costUsd from the main model.

This was already wrong before this PR, but canonical records now preserve the conflicting model and rates together. createProviderRequestTracker() already receives the call's model id, so the cost lookup should use the same value. A test with different prices for the main and summarizer models would pin it.

ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 2, 2026
…all as its own model

Review round 2 on apache#1877. Both findings were correct, and the first was a live
bug I had argued my way out of.

**Manual compaction was unmetered, and my "nothing reaches this path" was
wrong.** Desktop `sessions:compact` and CLI `/compact` both reach
`RuntimeKernel.compactSession()`, which opens an `AgentRun` and then called
`compactHistory` without its id. Outside `send()` there is no `currentRunId`, so
the summarization settled nothing while the old `recordLlmCall` had metered it.

`BackendCompactHistoryInput.runId` is now **required**, not optional: the failure
mode is silence, and an optional field is one a caller can forget in exactly the
situation that produced this bug. The kernel passes `run.runId`; the compaction
threads it into the accounting identity for that one call. Seventeen test call
sites had to name a run, which is the type doing its job.

**A configured summarizer model was priced as the session model.** Cost
resolution looked up `${providerType}:${this.input.modelId}` regardless of which
model served the request, so with `MAKA_CONTEXT_SEMANTIC_COMPACT_MODEL` set we
stored one model's id beside another model's `pricingRates` — precisely what
recording the rates exists to prevent. `resolveModelCallCost` now takes the
call's model id, supplied through the same identity the tracker already carries.

**One bug the new test caught in my own fix:** the compaction dep was wired as
`(callKind) => this.modelCallAccounting(callKind)`, which silently dropped the
new identity argument. TypeScript accepts a narrower function, so the manual
compaction kept recording nothing and the types stayed green. Only the
end-to-end assertion showed it.

Tests, both as asked: `SessionManager.compactSession()` settles exactly one
`history_compact` record carrying the run the kernel opened, driven through a
real `AiSdkBackend` and a real summarizer; and a semantic compaction with a
distinct summarizer model records that model's own rates while the send's own
steps keep the session model's.

`@maka/runtime` 2686/2698 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep). All runtime-consuming packages build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404

ARE404 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Both correct. Fixed in b5280caa.

P1 — manual compaction drops the record. You're right, and I was wrong in the way that matters: I wrote "nothing reaches this path today" in the PR description without checking, after you'd flagged it the first time. compactSession()AgentRuncompactHistory without the run id is exactly as you describe, and Desktop and CLI both walk it.

runId is now required on BackendCompactHistoryInput rather than optional. The failure mode here is silence — no error, no missing record anyone would notice — and an optional field is one a caller can forget in precisely the situation that produced the bug. Seventeen test call sites had to name a run; that's the type earning its keep. The kernel passes run.runId, and the compaction threads it into the accounting identity for that one call instead of resolving from turn state that does not exist outside send().

The test you asked for caught a second bug inside my own fix. The compaction dependency was wired as (callKind) => this.modelCallAccounting(callKind). Adding the identity argument to the method did not change that arrow, and TypeScript accepts a function that ignores parameters — so the manual compaction still recorded nothing, and tsc stayed green. The end-to-end assertion is the only thing that showed it. Same shape as the dead accounting gate from round 2 of #1755: a change about a guarantee needs a test that exercises the guarantee.

P2 — summarizer model priced as the main model. Also correct, and it belongs in this PR for the reason you gave: the record now preserves the id and the rates together, so the contradiction became storable. resolveModelCallCost takes the call's model id, supplied through the identity the tracker already carries. Test uses deliberately absurd rates for the two models (1 vs 1000 per 1M) and asserts the semantic_compact record carries the summarizer's pricingRates while the send's own steps carry the session model's.

Verification: @maka/runtime 2686/2698 (3 suites are this machine's rg-as-shell-function noise), @maka/runtime-host 524/524, @maka/desktop 1101/1139 (38 are this worktree's missing Astryx peer dep). Every runtime-consuming package builds — I checked headless, cli, mcp, and computer-use individually this time rather than trusting a grep, after the last round's miss.

@ARE404

ARE404 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

The red e2e on this run is not from this branch — it is failing on main right now.

main @ 3ce9afc4 (#1924) fails the identical assertion: run 30757371754, project-management.spec.ts:27, locator('[data-project-id]').filter({ hasText: '未归属项目' }) never becomes visible. Same line, same locator as this PR's failure — and CI tests refs/pull/N/merge, so this branch is being tested against that same main.

Nothing in this PR touches the renderer or project grouping; typecheck and test are green here. I'll rebase once main is fixed so the merge commit runs clean — gh run rerun isn't available to me as an external contributor.

ARE404 and others added 6 commits August 3, 2026 10:04
First of the two compaction call kinds apache#1679 left unrouted. `history_compact`
was still writing a per-send row into the frozen `LlmCallRecord` table, which is
the last place a model call is metered outside `ModelCallAttempt`.

The summarizer already built a `ProviderRequestTracker` for capture and attempt
diagnostics; it now also carries accounting, so the call settles the same way a
main send does — one record per physical provider request, with `usageBasis` and
`costBasis` instead of a cost that cannot say whether it is real.

**Accounting is supplied per call, by the backend.** The host wires the
summarizer once at composition time and cannot know `runId`, which is per-turn
state only the backend holds. Rather than plumbing a turn-to-run resolver down
through the kernel and the backend factory context so the host could look up
something the backend already has, `AiSdkBackend.modelCallAccounting(callKind)`
builds the identity and the caller passes it in at the moment the call is made.
The main send path now uses the same factory, so the three call kinds share one
construction instead of repeating it.

The legacy writer goes out in the same commit: keeping both would have made
history compaction the double-metered path that apache#1755 removed everywhere else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The second of the two compaction call kinds apache#1679 left unrouted, and the last
model call anywhere that was metered by hand. `semantic_compact` had no tracker
at all: it called `generateCompactSummary` directly and then built an
`LlmCallRecord` around the returned usage.

It now settles the way every other provider request does. `generateCompactSummary`
takes an optional `ProviderRequestTracker` and wraps the model with a
`wrapGenerate` middleware — the exact mirror of what `startStream` already does
with `wrapStream`, and in the one place that already owns "attach a tracker to a
model". The success, failure, and abort paths all settle inside the tracker, so
the hand-rolled try/catch that recorded three ways goes with it.

The backend hands the summarizer a *built* tracker rather than the capture,
attempt, and id sinks it is made of: compaction has no business assembling
metering identity, and a half-wired tracker is what produces records nothing can
attribute. `createProviderRequestTracker` also absorbed the main send's own
tracker construction, so this kind was added without a second copy of it.

One trace per turn rather than per call, so a step that summarizes is a step of
that trace and a retried summarization is another attempt of the same logical
call. Built on first use — most turns never summarize, and an unused trace id is
a trace that never happened.

The legacy writer goes out in the same commit, along with the now-unused
`computeCostUsd` dep: cost is resolved at settlement by the seam's own
`resolveCost`, with a basis attached.

One behavioural difference worth stating: the old row copied the SDK's
normalized `cacheRead` through as a cache hit. The canonical record attributes
cache tokens only when the provider's own payload claims them, so a provider
that reports none now yields an absent field instead of a number no provider
ever said.

`@maka/runtime` 2648/2661 (the 4 failures are the documented local `rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With both compaction kinds routed, nothing in the send path writes a
`LlmCallRecord` any more. `AiSdkCompactionCapabilities.recordLlmCall` and its
`LlmTelemetryRecorder` type had no writer left above them, and the Host and
Desktop each still wired a recorder into a backend field nothing read — dead
plumbing that a future change could just as easily have brought back to life.
Five backend tests still collected the records into arrays nothing asserted on;
those go too.

**`recordLlmCall` itself stays, and apache#1679's plan to delete it here was wrong.**
`goal_evaluation` is the fourth call kind, and `createHostGoalEvaluator` still
writes it to the frozen table. It cannot follow the compaction kinds through
this seam as-is: a `ModelCallAttempt` is identified by `(sessionId, runId,
turnId)`, and the Host evaluates a goal against a `sessionId` alone — there is
no run or turn at that layer to attribute the call to. Giving it one is a design
question for the RFC, not something to smuggle into a routing change.

The practical consequence is worth stating plainly, because apache#1679 currently says
otherwise: `provenance.legacyRecords` will *not* fall to zero as the old table
ages out. Goal evaluations keep landing there, so the merged read path stays
load-bearing until that kind is routed too.

`@maka/runtime` 2648/2661 (4 = documented local `rg` noise), `@maka/runtime-host`
488/488, `@maka/desktop` 1090/1128 — identical to this worktree's baseline on the
parent commit (the 38 are its missing Astryx peer dep, not this change).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… it settles

Review round 1 on apache#1877. The P1 and the P2s that were about this PR's own
behaviour, plus every P3.

**Metering no longer depends on a diagnostic.** `createProviderRequestTracker`
returned undefined without `recordProviderRequestCapture`, so a deployment with
capture off — a reachable config; both hosts wire it conditionally — silently
stopped metering compaction, which the deleted `recordLlmCall` had metered
unconditionally. Capture is now optional inside the tracker: `preparedCapture`
is pure, so `requestHash`, `requestBytes`, and `segments` survive without a
sink, and only the artifact join keys (`captureId`, `captureArtifactId`) go
absent. The tracker is built when there is either sink to feed. The Host now
wires the summarizer's tracking unconditionally too, and the Desktop wires it at
all — it never had, so the accounting the backend computed for a Desktop history
compaction was handed to a summarizer that had nowhere to settle it.

**The "one place owns attaching a tracker" claim is now true rather than
asserted.** `withProviderGenerateTracking` is shared by `generateCompactSummary`
and `buildLlmHistorySummarizer`; `ProviderMiddlewareGenerateInput` is declared
once. Also fixed: history_compact records carried no `contextWindow` while
semantic_compact did.

**Tests for what was only claimed:**

- `usageBasis: 'missing'` — the branch had no test anywhere. A call the provider
  reported no usage for records `missing`, not zero tokens, and stays unpriced
  whatever the resolver would have said.
- Metering with capture switched off, asserting the attempt still carries the
  locally-computed request shape and no artifact ids.
- Cache attribution end-to-end: the semantic mock now ships a `raw` provider
  payload, so the assertion moved from "absent because this mock claims nothing"
  to `cacheReadInputTokens: 2` through the provider branch. The no-`raw` rule
  stays pinned at the unit level.
- A dry-run (`validate_only`) semantic compaction really is a billed call: the
  summarizer runs to completion and only then is its block refused. Confirmed by
  the test, which is why it is worth pinning — a mode named "dry run" that bills
  is what a later reader would assume otherwise.
- Mid-turn history compaction settles a canonical record end-to-end through the
  backend glue, with the real summarizer against a mock provider, asserting the
  live `runId` resolves — a stubbed resolver cannot show that.

P3 cleanups: two dead imports, and the Usage IPC comment that still claimed the
frozen table receives compaction calls.

`@maka/runtime` 2684/2696 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep, unchanged from the parent commit).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fallout from making capture optional on `ProviderRequestAttemptRecord`: the
headless provider-request trace analyzer joins attempts to captures by id, and
the id is no longer guaranteed to be there.

Behaviour is unchanged for every trace this tool actually reads. It analyses a
capture ledger, so an attempt that cannot name a capture is incomplete by its
own definition and fails with the same diagnostic it already produced — and the
decoder above it already rejects such records as `invalid_attempt` before the
join runs. Only the type needed narrowing.

Caught by CI, not by me: I checked the blast radius of the optional fields in
`core`, `storage`, `runtime-host`, and `desktop`, and did not grep `headless`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…all as its own model

Review round 2 on apache#1877. Both findings were correct, and the first was a live
bug I had argued my way out of.

**Manual compaction was unmetered, and my "nothing reaches this path" was
wrong.** Desktop `sessions:compact` and CLI `/compact` both reach
`RuntimeKernel.compactSession()`, which opens an `AgentRun` and then called
`compactHistory` without its id. Outside `send()` there is no `currentRunId`, so
the summarization settled nothing while the old `recordLlmCall` had metered it.

`BackendCompactHistoryInput.runId` is now **required**, not optional: the failure
mode is silence, and an optional field is one a caller can forget in exactly the
situation that produced this bug. The kernel passes `run.runId`; the compaction
threads it into the accounting identity for that one call. Seventeen test call
sites had to name a run, which is the type doing its job.

**A configured summarizer model was priced as the session model.** Cost
resolution looked up `${providerType}:${this.input.modelId}` regardless of which
model served the request, so with `MAKA_CONTEXT_SEMANTIC_COMPACT_MODEL` set we
stored one model's id beside another model's `pricingRates` — precisely what
recording the rates exists to prevent. `resolveModelCallCost` now takes the
call's model id, supplied through the same identity the tracker already carries.

**One bug the new test caught in my own fix:** the compaction dep was wired as
`(callKind) => this.modelCallAccounting(callKind)`, which silently dropped the
new identity argument. TypeScript accepts a narrower function, so the manual
compaction kept recording nothing and the types stayed green. Only the
end-to-end assertion showed it.

Tests, both as asked: `SessionManager.compactSession()` settles exactly one
`history_compact` record carrying the run the kernel opened, driven through a
real `AiSdkBackend` and a real summarizer; and a semantic compaction with a
distinct summarizer model records that model's own rates while the send's own
steps keep the session model's.

`@maka/runtime` 2686/2698 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep). All runtime-consuming packages build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404
ARE404 force-pushed the are404/feat-compact-call-routing branch from b5280ca to bfdf958 Compare August 3, 2026 02:06
@Astro-Han

Copy link
Copy Markdown
Contributor

I rechecked the latest commit. The two fixes are sound: manual compaction now carries the run opened by the kernel, and a separate semantic summarizer is priced as its own model.

One live path is still missing the canonical record.

The remaining gap

RuntimeKernel exposes ctx.recordModelCallAttempt, but the CLI factory never passes it to AiSdkBackend. The CLI history summarizer is also created without providerRequestTracking.

So /compact gets the new runId, but there is still no tracker or canonical sink to use it:

sequenceDiagram
    participant CLI
    participant Kernel as RuntimeKernel
    participant Backend as AiSdkBackend
    participant Summary as History summarizer
    participant Ledger as AgentRun ledger

    CLI->>Kernel: /compact
    Kernel->>Kernel: open AgentRun
    Kernel->>Backend: compactHistory(runId, turnId)

    Note over Backend: recordModelCallAttempt<br/>was not passed by CLI
    Backend->>Backend: accounting = undefined

    Backend->>Summary: summarize()
    Note over Summary: providerRequestTracking<br/>was not configured
    Summary->>Summary: untracked provider call

    Summary-->>Backend: summary
    Backend-->>Kernel: compact result
    Note over Ledger: no history_compact<br/>ModelCallAttempt
Loading

The new SessionManager.compactSession() test misses this because its factory supplies both dependencies itself:

                         Test factory    Real CLI
recordModelCallAttempt        yes           no
providerRequestTracking       yes           no

That test proves the runtime path works when everything is wired. It does not exercise the CLI factory that users actually reach.

I still see this as a blocker for the claim that CLI /compact is fixed.

There is a similar, older gap in Harbor. Its controller exposes recordModelCallAttempt, but the Harbor AiSdkBackend composition does not pass it through. Harbor semantic compaction can therefore produce diagnostic attempts without a canonical record. That is not a regression from this PR, but it conflicts with #1679's intended end state.

I think the split ownership is the real problem

Today tracker ownership depends on the call kind:

flowchart LR
    subgraph Roots["Product composition"]
        Desktop
        Host["Runtime Host"]
        CLI
        Harbor
    end

    subgraph Calls["Model-call paths"]
        Main["main<br/>Backend creates tracker"]
        Semantic["semantic_compact<br/>Backend creates tracker"]
        History["history_compact<br/>Host builds tracker inputs,<br/>Backend supplies accounting"]
    end

    Tracker["ProviderRequestTracker"]
    Record["ModelCallAttempt"]

    Desktop --> Main
    Desktop --> Semantic
    Desktop --> History

    Host --> Main
    Host --> Semantic
    Host --> History

    CLI --> Main
    CLI --> History

    Harbor --> Main
    Harbor --> Semantic

    Main --> Tracker
    Semantic --> Tracker
    History -.->|"only if that product<br/>wired every input"| Tracker
    Tracker --> Record
Loading

This has now failed in several different ways: capture originally gated metering, Desktop history tracking was absent, manual compaction had no run identity, an intermediate callback dropped the new identity argument, and CLI still lacks both pieces of wiring.

Another local CLI patch would close today's hole. It would not remove the state in which a new composition root can silently omit half of accounting.

Would you be open to finishing the tracker ownership cleanup in this PR?

The narrow version would be:

  • Let AiSdkBackend create the history_compact tracker through the existing createProviderRequestTracker, just as it already does for main and semantic_compact.
  • Pass that complete tracker through HistoryCompactSummaryInput.
  • Remove BuildLlmHistorySummarizerOptions.providerRequestTracking and the duplicated Host/Desktop construction of tracker inputs.
  • Leave each product responsible for supplying its canonical sink, usually ctx.recordModelCallAttempt.
  • Cover the real CLI /compact factory and Harbor semantic-compaction factory with composition tests.

The resulting boundary is simpler:

flowchart TD
    Desktop
    Host["Runtime Host"]
    CLI
    Harbor

    Sink["Canonical sink<br/>plus optional capture diagnostics"]
    Backend["AiSdkBackend"]
    Factory["One tracker factory"]
    Main["main"]
    History["history_compact"]
    Semantic["semantic_compact"]
    Tracker["ProviderRequestTracker"]
    Authority["AgentRun<br/>ModelCallAttempt"]
    Usage["Usage projection"]

    Desktop --> Sink
    Host --> Sink
    CLI --> Sink
    Harbor --> Sink

    Sink --> Backend
    Backend --> Factory

    Factory --> Main
    Factory --> History
    Factory --> Semantic

    Main --> Tracker
    History --> Tracker
    Semantic --> Tracker

    Tracker --> Authority
    Authority --> Usage
Loading

This matches the revised #1679 design: every real provider request kind crosses the same tracker seam; capture remains optional; run and model identity belong to the call; the old compaction writer can be removed without leaving a product-specific hole.

I am not suggesting a unified Runtime Host or a broader accounting rewrite. goal_evaluation, global errorClass, Pi aggregate accounting, outbox recovery, and Inspector UI work can stay out of this PR.

If you want to keep the diff smaller, passing the two missing fields in CLI and adding a real CLI composition test would fix the immediate blocker. My preference is the backend-owned history tracker now. This PR already owns that routing change, and the split has caused enough silent misses that making the invariant structural seems cheaper than debugging the next composition root.

After that, I would stop. I do not see a reason to add another abstraction beyond one tracker factory and real composition tests.

中文意见

我重新检查了最新提交。之前两个问题已经正确修复:手动压缩携带 Kernel 创建的真实 runId,独立 semantic summarizer 也使用自己的模型价格。

但真实 CLI 入口仍然没有 canonical record。

RuntimeKernel 已经提供 ctx.recordModelCallAttempt,CLI 创建 AiSdkBackend 时没有转交。CLI 的 history summarizer 也没有配置 providerRequestTracking。因此 /compact 虽然拿到了新的 runId,却没有 tracker 和 canonical sink 来使用它,最终仍然是 0 条 history_compact ModelCallAttempt

新增的 SessionManager.compactSession() 测试没有覆盖这个边界。测试 factory 自己注入了 recordModelCallAttemptproviderRequestTracking,所以它证明的是“正确接线后 Runtime 能工作”,而不是“真实 CLI factory 已经正确接线”。

Harbor 也有类似的既有缺口。controller 已经提供 recordModelCallAttempt,但 Harbor composition 没有传给 AiSdkBackend。启用 semantic compaction 后,可以产生 diagnostic attempt,却没有 canonical record。它不是本 PR 引入的回归,但与 #1679 的最终目标不一致。

我认为反复遗漏的根因是 tracker ownership 分裂:

  • main 由 Backend 创建 tracker;
  • semantic_compact 由 Backend 创建 tracker;
  • history_compact 由 Host/Desktop/CLI 组装 tracker 原料,Backend 只补 accounting。

这个拆分已经先后导致 capture 依赖、Desktop 漏接、manual run identity 缺失、中间回调丢参数,以及现在的 CLI 漏接。

建议考虑在当前 PR 中做一次窄范围收口:

  • 使用已有的 createProviderRequestTracker,让 Backend 创建 history_compact tracker。
  • 通过 HistoryCompactSummaryInput 传递完整 tracker。
  • 删除 BuildLlmHistorySummarizerOptions.providerRequestTracking 和 Host/Desktop 重复的 tracker 原料组装。
  • 每个产品入口只负责提供 canonical sink,例如 ctx.recordModelCallAttempt
  • 增加真实 CLI /compact 和 Harbor semantic compaction 的 composition tests。

这不是要求统一 Runtime Host,也不是扩大 accounting 设计。goal_evaluation、全局 errorClass、Pi aggregate、outbox recovery 和 Inspector UI 都可以继续留在本 PR 之外。

如果希望保持最小 diff,只补 CLI 缺失的两个字段并增加真实 CLI composition 测试,也能解决当前 blocker。但我更倾向在本 PR 内把 history tracker ownership 收回 Backend。这个 PR 本来就在修改这条路径,而当前拆分已经造成了多次静默遗漏。

完成这一点后就应该停。我不建议继续增加其他抽象。

… tracker

Review round 3 on apache#1877. The blocker was real: the CLI composition never passed
`ctx.recordModelCallAttempt` to `AiSdkBackend` and never configured the
summarizer's tracking, so `/compact` got the new `runId` and still had nothing to
settle into. Harbor had the same gap for `semantic_compact`. Both are fixed here,
but the reviewer's structural point is the one worth acting on: this is the fifth
way the same split has failed, so the split goes.

`history_compact` now gets its tracker from the same
`AiSdkBackend.createProviderRequestTracker` that `main` and `semantic_compact`
already use, and `HistoryCompactSummaryInput` carries a *ready* tracker instead
of an accounting identity. `BuildLlmHistorySummarizerOptions.providerRequestTracking`
is gone, and with it the Host's and Desktop's hand-assembled copies of the
tracker's inputs — including the `contextWindow` plumbing added one round ago,
which the backend now supplies for free because it knows the model.

What a product owes accounting is now exactly one thing: the canonical sink.
Nothing else can be half-wired, because nothing else is a product's to wire.
`AiSdkCompactionDeps.modelCallAccounting` went with it — one factory, one seam.

**Composition tests, because the last round's test proved the wrong thing.** The
`SessionManager.compactSession()` test supplied both dependencies itself, so it
demonstrated that the runtime works when everything is wired, not that the CLI
wires it. The two new tests build backends through the real CLI and Harbor
factories and assert the sink reaches the caller's recorder. Both were confirmed
to fail with their production wiring reverted.

CLI's gap turned out to be wider than the review described: with no sink passed
at all, `main` sends were unmetered too, and had been since the frozen table's
writer was removed — CLI never wired `recordLlmCall` either, so this composition
root has simply never been inside accounting.

Left out deliberately, as agreed: `goal_evaluation` routing, a global
`errorClass`, and anything Inspector-shaped.

`@maka/runtime` 2700/2712 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 563/563, `maka-agent` (CLI) 467/467, `@maka/headless`
harbor-cell 102/102, `@maka/desktop` 1156/1194 (38 = this worktree's missing
Astryx peer dep).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404

ARE404 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Took the backend-owned version. 6503ad18.

The blocker was real, and wider than described. CLI passes no sink at all — recordModelCallAttempt appears zero times in runtime-bootstrap.ts — so it isn't only /compact that records nothing, main sends don't either. Checking the history: CLI never wired recordLlmCall before this work either (git log -S finds nothing under packages/cli/src), so this composition root has simply never been inside accounting. Nothing regressed; it was never connected, and my runId fix alone could not have shown up there.

On the ownership split — you're right, and the count is what convinced me. Capture gating metering, Desktop's missing tracking, manual compaction's missing run, the callback that dropped the identity argument, CLI's missing sink: five failures, all the same shape, all "a composition root supplied some of what accounting needs." A sixth patch would have been the wrong answer.

So:

  • history_compact now gets its tracker from the same createProviderRequestTracker as main and semantic_compact.
  • HistoryCompactSummaryInput carries a ready tracker, not an accounting identity.
  • BuildLlmHistorySummarizerOptions.providerRequestTracking is deleted, along with the Host's and Desktop's hand-assembled copies of its inputs — including the contextWindow plumbing I added last round, which the backend now supplies for free because it already knows the model.
  • AiSdkCompactionDeps.modelCallAccounting is gone too. One factory, one seam.
  • A product now owes accounting exactly one thing: the canonical sink. There is nothing else left to half-wire.

CLI and Harbor both pass it now.

Your point about the last round's test landed. SessionManager.compactSession() supplied both dependencies itself, so it proved the runtime works when wired — not that anything wires it. The two new tests build backends through the real CLI and Harbor factories and assert the sink reaches the caller's own recorder. I reverted each production wiring and confirmed the matching test fails, so they are pinning the composition rather than passing by construction.

Verification: @maka/runtime 2700/2712 (3 suites are this machine's rg-as-shell-function noise), @maka/runtime-host 563/563, CLI 467/467, harbor-cell 102/102, @maka/desktop 1156/1194 (38 are this worktree's missing Astryx peer dep). @maka/headless's full workspace test script can't run here — its build trips on pre-existing undici typings in provider-auth-proxy.ts unrelated to this branch — so I ran the harbor suite directly; CI covers the rest.

Agreed on where to stop: goal_evaluation, global errorClass, Pi aggregates, outbox recovery, and Inspector work all stay out.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following this through. The latest revision resolves the remaining wiring and ownership gaps from the earlier reviews:

  • tracker construction now has a single owner in the backend;
  • main, semantic-compaction, and history-compaction calls share the same canonical accounting path;
  • CLI and Harbor now forward the canonical sink;
  • the added composition and runtime tests cover the previously missing paths.

I also did a fresh-eye re-review of the final diff and found no remaining P0–P3 issues. The current boundary is both simpler and more reliable: product roots provide the sink, while the backend owns the run/model/cost context needed to build the tracker.

CI is green. Looks good to merge.

@Astro-Han
Astro-Han merged commit 7cb5136 into apache:main Aug 3, 2026
5 checks passed
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Astro-Han pushed a commit that referenced this pull request Aug 3, 2026
…#1956)

* feat(inspector): project a per-session causal trace over both ledgers

First slice of #1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (#1687, #1755, #1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(inspector): correct six attribution and coverage gaps in the trace

Review on #1956. All six were right; four are cases the tests never drove and
two were contract surface that promised more than the code delivered.

**Coverage claimed a proof it cannot have.** `complete` said every call settled;
the canonical contract says present settlements cannot prove completeness. The
state is now `no_known_gap` — the absence of evidence of a gap. A detectable
shortfall is also new: `tokenUsage.runtimeSteps` states how many tool-loop steps
one aggregate stands for, and fewer main calls on record than that is a
disagreement between the two ledgers, reported per turn as a floor on what is
missing rather than a count of it.

**Attempts are deduplicated by `attemptId`.** An aborted attempt and its later
settlement are appended under one id; the ledger dedupes on write, a stream read
does not. Without `dedupeModelCallAttempts` the trace invented a retry and
double-counted a priced settlement, which would have put a session total out of
step with Settings → Usage over the very same records. Grouping now goes through
core's `groupModelCallAttempts`, which dedupes on the way in.

**Step-less turns had non-finite bounds.** Usage-only and text-only turns project
no steps, and folding an empty list gives ±Infinity — which JSON renders as
`null`. Bounds now come from the ledger facts the turn is made of.

**A handled tool failure no longer fails the turn.** Any failed step marked the
whole turn failed, including a tool error the model recovered from before
finishing normally. The ledger's terminal status decides whether the turn failed;
the failed step only locates the cause once that is established.

**The compaction step is emitted rather than merely declared.** `TraceCompactionStep`
existed and nothing produced it. Written checkpoints are system text events, and
they are a different fact from the `history_compact` model call: one is the
boundary the next request replays from, the other is the spend. Both now appear.

**`recoveryMode` was a policy wearing the name of an outcome.** Every dispatch
declares one, including ordinary first executions. It is now `recoveryPolicy`,
and an actual `recovered` decision is joined from `actions.toolRecovery` by
`operationId` — correlated rather than positional, because the decision is
appended by the recovery writer and not by the dispatch it settles.

Seven regression tests, one per finding plus the no-known-gap state.

`@maka/core` 703/703, `@maka/runtime` 2742/2757 — the 6 local failures are this
machine's `rg`-as-a-shell-function and macOS path noise; CI ran the same base
green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants