feat(core): add canonical ModelCallAttempt accounting contract - #1687
Conversation
|
The failing The failure is a Host process death, not an assertion. This run failed Which test fails varies per run. #1664 — an unrelated change — is currently red on the same file with Reproduced locally on this branch, and it moves. Running the full This PR can't reach that code path. It only touches Locally on this branch: I can't re-run the job ( |
Astro-Han
left a comment
There was a problem hiding this comment.
Review
Approve. Contract-only PR1 matches #1679: one record per physical provider attempt, strict subtype codec, pure projections, and model_call_attempt_recorded on the AgentRun seam. Leaving provider-request attempts as diagnostics is the right split. Explicit logicalCallId and the usage/cost bases are the pieces that shape was missing. Deferring aggregate is fine for v1.
No P0/P1. No live writer, so none of the notes below block merge. Closing the P2 before a writer depends on a looser codec is still the cheap window.
P2
priced without costUsd is accepted. The codec only forbids unpriced rows that carry an amount. Free is meant to be priced + costUsd: 0, but a priced row with no amount still decodes. Coverage then increments pricedAttempts while sumModelCallCostUsd skips the row (priced && costUsd !== undefined), so "all priced, total $0" looks like free. Add the dual reject next to the unpriced guard, allow zero, and mirror the existing unpriced-with-cost test. Do it here or before PR2.
P3
- Projection tests:
sum/coverage/groupdedupe internally; only the bare helper is exercised with a replay. One multi-id stream with a replayedattemptIdthrough sum+coverage locks that path. - Numeric gates: production already rejects negative / non-finite
costUsdand tokens; tests never pass-1/NaN/Infinity. pricingRates:isFiniteNumberlets negative rates and extra nested keys through. Weaker than top-level exact shape and weaker than pricing normalize elsewhere. Sum still keys offcostUsd, so this is audit quality only.- Empty join keys: only
logicalCallId/attemptIdrequire non-empty. EmptyproviderId/modelId/ session-run-turn strings pass. Attribution hygiene, not sum math. - Docs: payload identity is the portable SoT and must match the AgentRun envelope when written as an event; once writers land, this type is metering SoT and provider-attempt stays diagnostic.
- Optional audit fields: keep
pricingRevision/pricingRatesoptional on priced rows for v1; no need to force a full rate freeze on every record.
CI
Red test job is runtime-host unclean host shutdown under load. Diff is @maka/core only. Looks like the existing flake unless a re-run says otherwise.
Checked, no issue
New event instead of upgrading provider_request_attempt_recorded. Explicit logicalCallId. Subtype codec outside decodeAgentRunEvent. Free = priced + zero. Coverage is not a completeness proof.
Addresses the P2 and P3 notes on apache#1687. The codec forbade an unpriced record carrying an amount but still accepted a priced record with no amount. Coverage counted that row as priced while `sumModelCallCostUsd` skipped it, so a set of such rows read as "every call priced, total $0" — indistinguishable from genuinely free, which is the exact confusion the basis fields exist to prevent. `costBasis` and `costUsd` now travel together in both directions; zero stays legal and remains the only way to record a call that cost nothing. Also tightens the surrounding validation, all audit and attribution quality: - `pricingRates` is held to an exact object shape with non-negative rates. It previously accepted negative rates and unrecognized nested keys, which makes a recorded amount unexplainable and defeats storing the basis. - `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must be non-empty, matching the existing treatment of `logicalCallId` and `attemptId`. - Documents that the payload identity is the portable source of truth and must agree with the AgentRun envelope when the record is written as an event. Tests: symmetric priced-without-cost rejection and a priced-zero acceptance, negative and non-finite gates on costs and tokens, empty attribution keys, the strengthened rate shapes, and a replayed `attemptId` carried through `sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the bare dedupe helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks for the approve — I took the P2 and the cheap P3s now, since you're right that the window before a writer depends on a looser codec is the cheap one. Pushed as 2187a32. P2 — priced without P3 taken:
P3 left as-is:
On the metering-SoT wording — I'd rather land that line together with the writer in PR 2, so the doc comment doesn't claim an authority nothing populates yet. Say the word if you'd prefer it stated now. Tests grew 16 → 21. |
Addresses the P2 and P3 notes on apache#1687. The codec forbade an unpriced record carrying an amount but still accepted a priced record with no amount. Coverage counted that row as priced while `sumModelCallCostUsd` skipped it, so a set of such rows read as "every call priced, total $0" — indistinguishable from genuinely free, which is the exact confusion the basis fields exist to prevent. `costBasis` and `costUsd` now travel together in both directions; zero stays legal and remains the only way to record a call that cost nothing. Also tightens the surrounding validation, all audit and attribution quality: - `pricingRates` is held to an exact object shape with non-negative rates. It previously accepted negative rates and unrecognized nested keys, which makes a recorded amount unexplainable and defeats storing the basis. - `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must be non-empty, matching the existing treatment of `logicalCallId` and `attemptId`. - Documents that the payload identity is the portable source of truth and must agree with the AgentRun envelope when the record is written as an event. Tests: symmetric priced-without-cost rejection and a priced-zero acceptance, negative and non-finite gates on costs and tokens, empty attribution keys, the strengthened rate shapes, and a replayed `attemptId` carried through `sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the bare dedupe helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2187a32 to
faf4b60
Compare
Introduces the accounting record that PR 2 will write from the provider request seam, per the design settled in apache#1679. Contract only: no behavior change, no writer, no read path. `ModelCallAttempt` is one record per physical provider request attempt. It extends the shape `ProviderRequestAttemptRecord` already carries with the three things it lacks — call kind, resolved cost, and a logical-call link — and adds a strict subtype codec, because the generic AgentRun event decoder only checks that `data` is a record. For an accounting record an unvalidated field silently becomes a wrong number in a cost report. Two invariants are enforced by the codec rather than left to convention: - `costUsd` is absent unless `costBasis` is `'priced'`. Zero means the call was genuinely free, never that the price was unknown. - `usageBasis: 'missing'` cannot carry token fields. "The provider never reported usage" and "we have usage but no price" are separate failures and must not collapse into one counter. `logicalCallId` is explicit rather than reconstructed from `(traceId, step)`: `traceId` is per-tracker-instance, so it does not group a turn's calls, and a compound key every consumer has to rebuild is the implicit contract this record exists to remove. Retries are attempts of one logical call; terminality is derived in projection, not stored. `ModelCallCoverage` classifies the records present and documents that it is not a completeness proof — nothing records an expected dispatch count, so an attempt lost between dispatch and settlement is undetectable. `sumModelCallCostUsd` returns coverage alongside the total for the same reason: a bare number cannot express "plus an unknown amount from unpriced calls". Registers `model_call_attempt_recorded` in `AGENT_RUN_EVENT_TYPES`, so the record rides the existing AgentRun event seam — no new table, store, or writer, and nothing for the apache#1649 persistence cutover to coordinate with. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the P2 and P3 notes on apache#1687. The codec forbade an unpriced record carrying an amount but still accepted a priced record with no amount. Coverage counted that row as priced while `sumModelCallCostUsd` skipped it, so a set of such rows read as "every call priced, total $0" — indistinguishable from genuinely free, which is the exact confusion the basis fields exist to prevent. `costBasis` and `costUsd` now travel together in both directions; zero stays legal and remains the only way to record a call that cost nothing. Also tightens the surrounding validation, all audit and attribution quality: - `pricingRates` is held to an exact object shape with non-negative rates. It previously accepted negative rates and unrecognized nested keys, which makes a recorded amount unexplainable and defeats storing the basis. - `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must be non-empty, matching the existing treatment of `logicalCallId` and `attemptId`. - Documents that the payload identity is the portable source of truth and must agree with the AgentRun envelope when the record is written as an event. Tests: symmetric priced-without-cost rejection and a priced-zero acceptance, negative and non-finite gates on costs and tokens, empty attribution keys, the strengthened rate shapes, and a replayed `attemptId` carried through `sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the bare dedupe helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
faf4b60 to
7000efa
Compare
|
Rebased onto The runtime-host flake is resolved. The current red is Evidence that it isn't from this branch:
Local on the rebased commit: Happy to open a separate issue for the CLI breakage if it isn't already being tracked — it's blocking every PR that rebases onto current |
Astro-Han
left a comment
There was a problem hiding this comment.
Re-review
Approve again on 7000efa8b.
The fix commit closes the P2 (priced must carry costUsd, zero still legal) and the P3 items we called out (projection-stream dedupe tests, numeric poison samples, tighter pricingRates, non-empty attribution keys, payload-identity docs). Dual pairing is in the codec with symmetric tests.
Nothing left is merge-blocking. Leftover notes are optional P3 only: unused isOptionalFiniteNumber import, float tokens, more exhaustive optional-field tests, status enum docs. Fine to land as-is; further tightening can wait for the writer PR.
CI test still red on NO_REAL_CONNECTION:model_not_enabled in CLI bootstrap tests. This PR only touches @maka/core accounting contract files and does not exercise that path.
First half of the metering work in apache#1679: the provider request seam now produces the canonical accounting record landed in apache#1687. The old writers are untouched here and come out in the same PR before it opens. `ProviderRequestTracker` gains an optional `accounting` input. It is one unit rather than several independent fields because a `ModelCallAttempt` without session, run, and call kind is unattributable — there is no useful state where half of it is wired. Absent, the tracker stays purely diagnostic, which is what the capture-only paths and their tests rely on. Three behaviours the review on apache#1679 called for: - **Settlement never throws.** `finalize` runs inside the stream's `pull` handler, so a rejection there reaches `controller.error` and fails an otherwise-complete model response. The dispatch-time gate is `assertReady`, checked before the provider is called and alongside the existing capture gate rather than as a second one. A sink failure after dispatch means the call happened and was billed but went unrecorded, which is reported rather than raised. - **Cancellation and settlement are separate events.** An abort carrying no usage records provisionally without closing the attempt, so a `finish` arriving afterwards still settles it. Both writes share one `attemptId` and the record dedupes on that key keeping the last, so a cancelled call that really consumed tokens is no longer frozen as permanently token-less. - **Retries are attempts of one logical call.** `logicalCallId` is assigned per step and reused across retries, so the grouping is explicit in the record instead of reconstructed from `(traceId, step)` by every consumer. Cost resolves at settlement time and carries the rates it was computed against, so a stored figure stays auditable when pricing later changes. An unresolvable price records `costBasis: 'unpriced'` with no amount — never zero, which is reserved for calls that genuinely cost nothing. `usageBasis` separately reports whether the provider returned usage at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First half of the metering work in apache#1679: the provider request seam now produces the canonical accounting record landed in apache#1687. The old writers are untouched here and come out in the same PR before it opens. `ProviderRequestTracker` gains an optional `accounting` input. It is one unit rather than several independent fields because a `ModelCallAttempt` without session, run, and call kind is unattributable — there is no useful state where half of it is wired. Absent, the tracker stays purely diagnostic, which is what the capture-only paths and their tests rely on. Three behaviours the review on apache#1679 called for: - **Settlement never throws.** `finalize` runs inside the stream's `pull` handler, so a rejection there reaches `controller.error` and fails an otherwise-complete model response. The dispatch-time gate is `assertReady`, checked before the provider is called and alongside the existing capture gate rather than as a second one. A sink failure after dispatch means the call happened and was billed but went unrecorded, which is reported rather than raised. - **Cancellation and settlement are separate events.** An abort carrying no usage records provisionally without closing the attempt, so a `finish` arriving afterwards still settles it. Both writes share one `attemptId` and the record dedupes on that key keeping the last, so a cancelled call that really consumed tokens is no longer frozen as permanently token-less. - **Retries are attempts of one logical call.** `logicalCallId` is assigned per step and reused across retries, so the grouping is explicit in the record instead of reconstructed from `(traceId, step)` by every consumer. Cost resolves at settlement time and carries the rates it was computed against, so a stored figure stays auditable when pricing later changes. An unresolvable price records `costBasis: 'unpriced'` with no amount — never zero, which is reserved for calls that genuinely cost nothing. `usageBasis` separately reports whether the provider returned usage at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…1755) * feat(runtime): emit canonical ModelCallAttempt from the provider seam First half of the metering work in #1679: the provider request seam now produces the canonical accounting record landed in #1687. The old writers are untouched here and come out in the same PR before it opens. `ProviderRequestTracker` gains an optional `accounting` input. It is one unit rather than several independent fields because a `ModelCallAttempt` without session, run, and call kind is unattributable — there is no useful state where half of it is wired. Absent, the tracker stays purely diagnostic, which is what the capture-only paths and their tests rely on. Three behaviours the review on #1679 called for: - **Settlement never throws.** `finalize` runs inside the stream's `pull` handler, so a rejection there reaches `controller.error` and fails an otherwise-complete model response. The dispatch-time gate is `assertReady`, checked before the provider is called and alongside the existing capture gate rather than as a second one. A sink failure after dispatch means the call happened and was billed but went unrecorded, which is reported rather than raised. - **Cancellation and settlement are separate events.** An abort carrying no usage records provisionally without closing the attempt, so a `finish` arriving afterwards still settles it. Both writes share one `attemptId` and the record dedupes on that key keeping the last, so a cancelled call that really consumed tokens is no longer frozen as permanently token-less. - **Retries are attempts of one logical call.** `logicalCallId` is assigned per step and reused across retries, so the grouping is explicit in the record instead of reconstructed from `(traceId, step)` by every consumer. Cost resolves at settlement time and carries the rates it was computed against, so a stored figure stays auditable when pricing later changes. An unresolvable price records `costBasis: 'unpriced'` with no amount — never zero, which is reserved for calls that genuinely cost nothing. `usageBasis` separately reports whether the provider returned usage at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(core): project Usage aggregates from the canonical attempt ledger Second commit of the metering slice in #1679. Pure aggregation over `ModelCallAttempt`, serving the shapes the Usage authority reads today — `summary`, `buckets`, `logs` — so the read path can move off the per-send `LlmCallRecord` table in the next commit. Nothing is wired yet. One behavioural difference from the old table is deliberate, and is the reason this projection exists rather than a schema-compatible one. `totalCostUsd` sums only records whose price was resolvable, and every result carries the coverage that qualifies it. The old schema had no way to say "this call cost something we could not price", so it stored zero, and unpriced spend was indistinguishable from a free call. A total presented without its coverage repeats that claim. Two mapping decisions worth stating: - `interrupted` projects to `aborted`, not `error`. Both mean the call stopped short without the provider reporting a failure, and folding it into `error` would inflate the error rate with user cancellations. - Log rows carry `logicalCallId` as `callId` and keep session and turn attribution, so a row remains traceable back to the conversation that caused it rather than only to a model. Selection dedupes on `attemptId` before aggregating, so a re-appended settlement — the abort-then-late-usage path from the previous commit — counts once. Tool telemetry is untouched: `toolLogs` reads tool invocations, which this ledger does not describe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(metering): read Usage from the canonical ledger and drop the send-level meter Third and last commit of the metering slice in #1679, and the atomic one: the Usage authority moves onto `ModelCallAttempt` and the writer it replaces comes out in the same change, so no commit leaves two independent meters running. **Where canonical records land.** The AgentRun stream is the durable log — `model_call_attempt_recorded`, `durable: true`, because a record lost to a crashed flush is spend nothing else can reconstruct. It is not the read model: `AgentRunStore.readEvents(sessionId, runId)` answers "what happened in this run", and no Usage question is shaped that way. So each attempt also lands in `usage_model_call_attempts`, a new table beside the old one rather than inside it — `usage_llm_calls` cannot express `usageBasis` or `costBasis`, and writing canonical records through that schema is the dishonest-zero problem again. **The read path sums two sources.** The old table is frozen, never migrated, and still receives the `semantic_compact` and `history_compact` calls that have not been routed through the canonical seam yet. Every LLM-sourced result therefore carries `UsageProvenance`: the coverage of its canonical half, how many rows came from the frozen table, and how many stored records failed to decode. A total on the wire without that cannot be read honestly, so it is part of the wire. `legacyRecords` reaches zero on its own as the old table ages out of the range. **What comes out.** The terminal `recordLlmCall` in the streaming backend and the `usage_recorded` RunTraceEvent. Both measured the same provider requests the seam now settles per request instead of per send. `token_usage` SessionEvents and the RuntimeEvent per-turn aggregates stay — they feed replay and recovery and are not accounting. **What the deletion nearly took with it.** That record was also the only durable home for the send's terminal context diagnostics. The exhausted and aborted paths emit no `token_usage` event, so their compaction decisions, their final request shape, and the accumulated usage of the steps that did complete would have gone with it. They move to a `send_diagnostics_recorded` run-trace event, which carries no cost and meters nothing. The request-shape hashes needed no rescue — `model_stream_started` already carries them for step 0, and the new event carries the final shape a same-turn tool load produces. Two gaps the end-to-end acceptance found in the seam from the first commit: canonical records carried neither `connectionSlug` nor the connection's provider type, so they were attributable to a provider and model but not to a connection, and `moonshot.chat` would have split one provider across two bucket keys against the historical rows. The Desktop runs a second, complete metering stack against its own store. Deleting the send-level writer globally would have stopped its Usage page accruing, so it reads and writes through the same ledger. That is wiring, not authority: nothing here gives the embedded writer ownership, election, or admission. Acceptance is end-to-end and now green against a real provider wire: provider call → attempt record → projection → Usage surface. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(metering): make the Usage ledger a projection of one authority Addresses the review on #1755. **P1 — the Usage table was an independent write, not a recoverable read model.** The finding is right, and it undercut the claim the previous commit made about itself. Both hosts wrote each `ModelCallAttempt` to the AgentRun stream and to `usage_model_call_attempts` through `Promise.allSettled`, with nothing ever replaying one into the other. Calling the stream "the log of record" does not make it one: either write could land alone, and the Host's answer was to request a drain it could not recover from while the Desktop swallowed the failure and kept dispatching. That is two sources of truth, which is the thing this whole change set exists to remove. There is now one commit point. The AgentRun append is awaited first and `AgentRun.recordModelCallAttempt` rejects on failure instead of resolving, so a caller can tell whether the authority actually holds the record; the ledger is written only after it does. Rejecting is safe because the seam still swallows it — settlement runs inside the stream's `pull` handler, and a completed, billed response must never be failed by its own bookkeeping. A failed projection is recoverable. The run is marked in `usage_model_call_reprojection`, and the Usage read path re-derives marked runs from the stream before answering, bounded per query so a backlog drains across reads instead of inside one. The projection is idempotent — it upserts on `attemptId` — so repairing a run the table already holds is a no-op, and one undecodable event does not block the rest of its run. The marker is an optimization, not the correctness argument. Losing it costs a targeted repair, not the records: the authority holds every attempt, so a full re-projection recovers a run whose marker was never written. What a pass cannot repair is reported as `provenance.pendingRepairs` rather than silently missing from the totals — spend that is recoverable is a different claim from spend that is counted. The pre-dispatch gate moves with it. It now keys off the authority alone: a stale projection is recoverable and must not block a send, but an authority that cannot accept the record means the next dispatch produces spend nothing will ever hold. **P2 — one bucket-key function.** The two sources derived "the same hour" differently — an epoch-hour ordinal against an ISO hour — so `mergeUsageBuckets` saw two keys and split one hour in half without failing anywhere. Both now call `usageBucketKey`, and a mixed-source hourly query is a contract test. `day` happened to agree already; that it did was luck, not design. **P2 — a log row keeps its cost basis.** `projectModelCallUsageLogs` mapped an unpriced attempt to `costUsd: 0`, so a genuinely free call and a call whose price could not be resolved were indistinguishable per row — page-level coverage says how many were unpriced, not which. This reproduced, at row granularity, exactly the ambiguity the coverage breakdown was added to remove. `UsageLogRow` and the wire projection now carry `costBasis`, `costUsd` is absent for an unpriced row, and the codec rejects a row that claims both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(metering): read the accounting gate's own flag and keep repair honest Second review round on #1755. **The accounting gate was dead code.** `recordModelCallAttempt` set `accountingAuthorityFailed`, and `assertModelCallAccountingReady` checked `telemetryDrainRequested` — a flag that tracks the frozen legacy table, which no longer meters main sends at all. Nothing read the new flag, so a host whose authority had stopped accepting records kept dispatching provider calls it could never account for. The gate the previous commit introduced never actually closed; it does now. **The pending marker is written before the projection, not after it fails.** A marker written from a catch block cannot cover the case where the catch never runs — the process exiting between the authority append and the projection — which left a committed record this table would never learn about. Marking first turns it into an intent record: a crash anywhere after it still leaves a run the repair finds. That does not close the window entirely, and the docs no longer say it does. A process that dies between committing to the authority and writing the marker still leaves a record Usage will not see, because nothing sweeps the whole stream. The claim that "a full re-projection recovers a run whose marker was never written" described a sweep that was never implemented; the comment and the evidence-spine doc now state the real limit instead. **An unreadable authority event no longer disappears with the marker.** The repair decoded what it could, cleared the run, and dropped the undecodable count — so a real call fell out of the totals and out of the pending count in the same step, leaving nothing to say it existed. Clearing the marker is still right (it will not decode next pass either, and holding the run would stall every later repair behind it), so the count travels out on the result and folds into `provenance.unreadableRecords` alongside stored rows that will not decode. Two review items are deliberately left as follow-ups, and the PR description says so rather than implying they are covered: the Desktop Usage page still reads the session-derived `settings.usageStats` path and has no consumer for the canonical endpoints, and the Daily Review archives a merged total without carrying its provenance into the saved summary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
First slice of apache#1625, and the one that was blocked: the RFC's headline — per-step latency and cost — was not derivable from `RuntimeEvent`, which is why this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work landed, so `ModelCallAttempt` now carries one record per physical provider request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a cost frozen at call time. The projection this issue always wanted is now writable honestly. Pure and synchronous: both ledgers are handed in already read, so the caller owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in `packages/core`, builder in `packages/runtime`, per the split agreed on the issue. Three properties are deliberate: - **Retries are nested, not flattened.** Attempts of one logical call share a `logicalCallId` by contract, so "this call was retried twice" is a grouping rather than something a reader reconstructs from four steps that happen to share an id. - **An absent price stays absent.** A step whose attempts were never priced has no `costUsd`, and a session of only such calls totals to no price rather than to zero — the distinction the canonical record exists to keep. - **Coverage is stated, not implied.** A backend that emits no canonical records produces a trace that says so. The pi backend is exactly this case: it emits `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be indistinguishable from a session that did nothing. Aggregate usage with no record behind it is the signal, reported per session as `absent` or `partial` with the turns named. Failure attribution points at what failed *first*, not at the terminal error: a turn that ends in an error usually ends there because of an earlier tool failure, and naming the last event names the symptom. No UI, no search API, no cost authority of its own — a per-session total sums the same `ModelCallAttempt` records Settings → Usage aggregates, read at a different scope. `@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local `rg` noise). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First slice of apache#1625, and the one that was blocked: the RFC's headline — per-step latency and cost — was not derivable from `RuntimeEvent`, which is why this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work landed, so `ModelCallAttempt` now carries one record per physical provider request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a cost frozen at call time. The projection this issue always wanted is now writable honestly. Pure and synchronous: both ledgers are handed in already read, so the caller owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in `packages/core`, builder in `packages/runtime`, per the split agreed on the issue. Three properties are deliberate: - **Retries are nested, not flattened.** Attempts of one logical call share a `logicalCallId` by contract, so "this call was retried twice" is a grouping rather than something a reader reconstructs from four steps that happen to share an id. - **An absent price stays absent.** A step whose attempts were never priced has no `costUsd`, and a session of only such calls totals to no price rather than to zero — the distinction the canonical record exists to keep. - **Coverage is stated, not implied.** A backend that emits no canonical records produces a trace that says so. The pi backend is exactly this case: it emits `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be indistinguishable from a session that did nothing. Aggregate usage with no record behind it is the signal, reported per session as `absent` or `partial` with the turns named. Failure attribution points at what failed *first*, not at the terminal error: a turn that ends in an error usually ends there because of an earlier tool failure, and naming the last event names the symptom. No UI, no search API, no cost authority of its own — a per-session total sums the same `ModelCallAttempt` records Settings → Usage aggregates, read at a different scope. `@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local `rg` noise). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…#1956) * feat(inspector): project a per-session causal trace over both ledgers First slice of #1625, and the one that was blocked: the RFC's headline — per-step latency and cost — was not derivable from `RuntimeEvent`, which is why this issue paused for the accounting work (#1687, #1755, #1877). That work landed, so `ModelCallAttempt` now carries one record per physical provider request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a cost frozen at call time. The projection this issue always wanted is now writable honestly. Pure and synchronous: both ledgers are handed in already read, so the caller owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in `packages/core`, builder in `packages/runtime`, per the split agreed on the issue. Three properties are deliberate: - **Retries are nested, not flattened.** Attempts of one logical call share a `logicalCallId` by contract, so "this call was retried twice" is a grouping rather than something a reader reconstructs from four steps that happen to share an id. - **An absent price stays absent.** A step whose attempts were never priced has no `costUsd`, and a session of only such calls totals to no price rather than to zero — the distinction the canonical record exists to keep. - **Coverage is stated, not implied.** A backend that emits no canonical records produces a trace that says so. The pi backend is exactly this case: it emits `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be indistinguishable from a session that did nothing. Aggregate usage with no record behind it is the signal, reported per session as `absent` or `partial` with the turns named. Failure attribution points at what failed *first*, not at the terminal error: a turn that ends in an error usually ends there because of an earlier tool failure, and naming the last event names the symptom. No UI, no search API, no cost authority of its own — a per-session total sums the same `ModelCallAttempt` records Settings → Usage aggregates, read at a different scope. `@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local `rg` noise). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(inspector): correct six attribution and coverage gaps in the trace Review on #1956. All six were right; four are cases the tests never drove and two were contract surface that promised more than the code delivered. **Coverage claimed a proof it cannot have.** `complete` said every call settled; the canonical contract says present settlements cannot prove completeness. The state is now `no_known_gap` — the absence of evidence of a gap. A detectable shortfall is also new: `tokenUsage.runtimeSteps` states how many tool-loop steps one aggregate stands for, and fewer main calls on record than that is a disagreement between the two ledgers, reported per turn as a floor on what is missing rather than a count of it. **Attempts are deduplicated by `attemptId`.** An aborted attempt and its later settlement are appended under one id; the ledger dedupes on write, a stream read does not. Without `dedupeModelCallAttempts` the trace invented a retry and double-counted a priced settlement, which would have put a session total out of step with Settings → Usage over the very same records. Grouping now goes through core's `groupModelCallAttempts`, which dedupes on the way in. **Step-less turns had non-finite bounds.** Usage-only and text-only turns project no steps, and folding an empty list gives ±Infinity — which JSON renders as `null`. Bounds now come from the ledger facts the turn is made of. **A handled tool failure no longer fails the turn.** Any failed step marked the whole turn failed, including a tool error the model recovered from before finishing normally. The ledger's terminal status decides whether the turn failed; the failed step only locates the cause once that is established. **The compaction step is emitted rather than merely declared.** `TraceCompactionStep` existed and nothing produced it. Written checkpoints are system text events, and they are a different fact from the `history_compact` model call: one is the boundary the next request replays from, the other is the spend. Both now appear. **`recoveryMode` was a policy wearing the name of an outcome.** Every dispatch declares one, including ordinary first executions. It is now `recoveryPolicy`, and an actual `recovered` decision is joined from `actions.toolRecovery` by `operationId` — correlated rather than positional, because the decision is appended by the recovery writer and not by the dispatch it settles. Seven regression tests, one per finding plus the no-known-gap state. `@maka/core` 703/703, `@maka/runtime` 2742/2757 — the 6 local failures are this machine's `rg`-as-a-shell-function and macOS path noise; CI ran the same base green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
PR 1 of the canonical model-call accounting work in #1679. Contract only — no behavior change, no writer, no read path, independently mergeable.
What this adds
ModelCallAttempt: one record per physical provider request attempt, plus a strict subtype codec and pure projection helpers.It extends the shape
ProviderRequestAttemptRecordalready carries with the three things it lacks —callKind, resolved cost, and a logical-call link. The codec validates field shape becausedecodeAgentRunEventonly checks thatdatais a record; for an accounting record an unvalidated field silently becomes a wrong number in a cost report.Invariants enforced by the codec
costUsdis absent unlesscostBasisis'priced'. Zero means genuinely free, never unknown price. TodaycomputeCostreturns0when no pricing is configured, so the existing table already mixes the two — this record cannot.usageBasis: 'missing'cannot carry token fields. Per the review on [RFC] Canonical model-call accounting: one ModelCallAttempt record per real provider request #1679: "the provider never reported usage" and "we have usage but no price" are different failures and must not collapse into one counter.completedAtcannot precedestartedAt; ordinals are non-negative integers; unknown fields are rejected.Design decisions carried from #1679
logicalCallIdis explicit, not reconstructed from(traceId, step).traceIdis per-tracker-instance, so it does not group a turn's calls. Retries are attempts of one logical call; terminality is derived in projection (settledAttempt), not stored.granularity: 'aggregate'is not in this contract. Deferred with Q1 —PiAgentBackendis harbor-only (headless/harbor-cell.ts:769), which does not justify shaping the core contract around it yet.pricingRevisionandpricingRatespersisted alongside the amount so the figure stays auditable.ModelCallCoverageis documented as not a completeness proof. Nothing records an expected dispatch count, so an attempt lost between dispatch and settlement is undetectable.sumModelCallCostUsdreturns coverage with the total for the same reason — a bare number cannot express "plus an unknown amount from unpriced calls".ts, documented ondedupeModelCallAttempts; attempt events carry the settlement time and are appended asynchronously.Placement
model_call_attempt_recordedis registered inAGENT_RUN_EVENT_TYPES, so the record rides the existing AgentRun event seam — no new table, store, or writer, and nothing for the #1649 persistence cutover to coordinate with.Tests
16 new tests in
packages/core/src/__tests__/model-call-attempt.test.tscovering codec acceptance and rejection, both invariants, the free-vs-unpriced distinction,attemptIdidempotency, retry grouping and settlement derivation, and coverage counting. Full@maka/coresuite: 1224 passing, 0 failing. Biome lint and format clean.Not in this PR
The seam upgrade,
semantic_compactrouting, double-metering removal, and the authority read path — sequenced in #1679.