Skip to content

feat(core): add canonical ModelCallAttempt accounting contract - #1687

Merged
Astro-Han merged 2 commits into
apache:mainfrom
ARE404:are404/feat-model-call-accounting
Jul 31, 2026
Merged

feat(core): add canonical ModelCallAttempt accounting contract#1687
Astro-Han merged 2 commits into
apache:mainfrom
ARE404:are404/feat-model-call-accounting

Conversation

@ARE404

@ARE404 ARE404 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

PR 1 of the canonical model-call accounting work in #1679. Contract only — no behavior change, no writer, no read path, independently mergeable.

What this adds

ModelCallAttempt: one record per physical provider request attempt, plus a strict subtype codec and pure projection helpers.

It extends the shape ProviderRequestAttemptRecord already carries with the three things it lacks — callKind, resolved cost, and a logical-call link. The codec validates field shape because decodeAgentRunEvent only checks that data is a record; for an accounting record an unvalidated field silently becomes a wrong number in a cost report.

Invariants enforced by the codec

  • costUsd is absent unless costBasis is 'priced'. Zero means genuinely free, never unknown price. Today computeCost returns 0 when no pricing is configured, so the existing table already mixes the two — this record cannot.
  • usageBasis: 'missing' cannot carry token fields. Per the review on [RFC] Canonical model-call accounting: one ModelCallAttempt record per real provider request #1679: "the provider never reported usage" and "we have usage but no price" are different failures and must not collapse into one counter.
  • completedAt cannot precede startedAt; ordinals are non-negative integers; unknown fields are rejected.

Design decisions carried from #1679

  • logicalCallId is explicit, not reconstructed from (traceId, step). traceId is per-tracker-instance, so it does not group a turn's calls. Retries are attempts of one logical call; terminality is derived in projection (settledAttempt), not stored.
  • granularity: 'aggregate' is not in this contract. Deferred with Q1 — PiAgentBackend is harbor-only (headless/harbor-cell.ts:769), which does not justify shaping the core contract around it yet.
  • Cost is frozen at record time, with pricingRevision and pricingRates persisted alongside the amount so the figure stays auditable.
  • ModelCallCoverage is documented as not a completeness proof. Nothing records an expected dispatch count, so an attempt lost between dispatch and settlement is undetectable. sumModelCallCostUsd returns coverage with the total for the same reason — a bare number cannot express "plus an unknown amount from unpriced calls".
  • Ordering is append order, never ts, documented on dedupeModelCallAttempts; attempt events carry the settlement time and are appended asynchronously.

Placement

model_call_attempt_recorded is registered in AGENT_RUN_EVENT_TYPES, so the record rides the existing AgentRun event seam — no new table, store, or writer, and nothing for the #1649 persistence cutover to coordinate with.

Tests

16 new tests in packages/core/src/__tests__/model-call-attempt.test.ts covering codec acceptance and rejection, both invariants, the free-vs-unpriced distinction, attemptId idempotency, retry grouping and settlement derivation, and coverage counting. Full @maka/core suite: 1224 passing, 0 failing. Biome lint and format clean.

Not in this PR

The seam upgrade, semantic_compact routing, double-metering removal, and the authority read path — sequenced in #1679.

@ARE404

ARE404 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

The failing test check looks like a pre-existing flake in packages/runtime-host/src/__tests__/execution-host.test.js rather than something this PR causes. Evidence, in case it's useful beyond this PR:

The failure is a Host process death, not an assertion. This run failed steering becomes durable and ordered followups automatically start the next root with execution Host stopped uncleanly: code 1 (414/415 passing). The fixture spawns real Host child processes and reports this when shutdown doesn't settle.

Which test fails varies per run. #1664 — an unrelated change — is currently red on the same file with production Host settles dispatched Client Capabilities before publishing Ready / execution Host stopped uncleanly: SIGTERM.

Reproduced locally on this branch, and it moves. Running the full @maka/runtime-host suite locally fails #1664's test, not the one CI failed here. Both pass in isolation — production Host settles… takes 475 ms alone and times out under the full suite. That pattern reads as shutdown-timeout under load rather than a logic failure.

This PR can't reach that code path. It only touches @maka/core: a new module, its exports, and one string appended to AGENT_RUN_EVENT_TYPES. That array has three usages repo-wide — the declaration, the AgentRunEventType derivation, and the .includes() membership check in decodeAgentRunEvent. Nothing iterates it or derives counts from it, so adding a member only widens what the decoder accepts.

Locally on this branch: @maka/core 1224/1224 passing, @maka/runtime-host green except the flake above, and biome lint/format clean.

I can't re-run the job (Must have admin rights to Repository), so a maintainer re-run would confirm it. Happy to open a separate issue for the flake with the reproduction above if that's useful — I didn't find an existing one.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Approve. Contract-only PR1 matches #1679: one record per physical provider attempt, strict subtype codec, pure projections, and model_call_attempt_recorded on the AgentRun seam. Leaving provider-request attempts as diagnostics is the right split. Explicit logicalCallId and the usage/cost bases are the pieces that shape was missing. Deferring aggregate is fine for v1.

No P0/P1. No live writer, so none of the notes below block merge. Closing the P2 before a writer depends on a looser codec is still the cheap window.

P2

priced without costUsd is accepted. The codec only forbids unpriced rows that carry an amount. Free is meant to be priced + costUsd: 0, but a priced row with no amount still decodes. Coverage then increments pricedAttempts while sumModelCallCostUsd skips the row (priced && costUsd !== undefined), so "all priced, total $0" looks like free. Add the dual reject next to the unpriced guard, allow zero, and mirror the existing unpriced-with-cost test. Do it here or before PR2.

P3

  • Projection tests: sum / coverage / group dedupe internally; only the bare helper is exercised with a replay. One multi-id stream with a replayed attemptId through sum+coverage locks that path.
  • Numeric gates: production already rejects negative / non-finite costUsd and tokens; tests never pass -1 / NaN / Infinity.
  • pricingRates: isFiniteNumber lets negative rates and extra nested keys through. Weaker than top-level exact shape and weaker than pricing normalize elsewhere. Sum still keys off costUsd, so this is audit quality only.
  • Empty join keys: only logicalCallId / attemptId require non-empty. Empty providerId / modelId / session-run-turn strings pass. Attribution hygiene, not sum math.
  • Docs: payload identity is the portable SoT and must match the AgentRun envelope when written as an event; once writers land, this type is metering SoT and provider-attempt stays diagnostic.
  • Optional audit fields: keep pricingRevision / pricingRates optional on priced rows for v1; no need to force a full rate freeze on every record.

CI

Red test job is runtime-host unclean host shutdown under load. Diff is @maka/core only. Looks like the existing flake unless a re-run says otherwise.

Checked, no issue

New event instead of upgrading provider_request_attempt_recorded. Explicit logicalCallId. Subtype codec outside decodeAgentRunEvent. Free = priced + zero. Coverage is not a completeness proof.

Comment thread packages/core/src/model-call-attempt.ts
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Jul 31, 2026
Addresses the P2 and P3 notes on apache#1687.

The codec forbade an unpriced record carrying an amount but still accepted a
priced record with no amount. Coverage counted that row as priced while
`sumModelCallCostUsd` skipped it, so a set of such rows read as "every call
priced, total $0" — indistinguishable from genuinely free, which is the exact
confusion the basis fields exist to prevent. `costBasis` and `costUsd` now
travel together in both directions; zero stays legal and remains the only way
to record a call that cost nothing.

Also tightens the surrounding validation, all audit and attribution quality:

- `pricingRates` is held to an exact object shape with non-negative rates.
  It previously accepted negative rates and unrecognized nested keys, which
  makes a recorded amount unexplainable and defeats storing the basis.
- `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must
  be non-empty, matching the existing treatment of `logicalCallId` and
  `attemptId`.
- Documents that the payload identity is the portable source of truth and must
  agree with the AgentRun envelope when the record is written as an event.

Tests: symmetric priced-without-cost rejection and a priced-zero acceptance,
negative and non-finite gates on costs and tokens, empty attribution keys, the
strengthened rate shapes, and a replayed `attemptId` carried through
`sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the
bare dedupe helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404

ARE404 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the approve — I took the P2 and the cheap P3s now, since you're right that the window before a writer depends on a looser codec is the cheap one. Pushed as 2187a32.

P2 — priced without costUsd. Fixed, and you identified it precisely: coverage counted the row as priced while sumModelCallCostUsd skipped it, so "every call priced, total $0" was indistinguishable from genuinely free. costBasis and costUsd now travel together in both directions, zero stays legal, and there's a symmetric test plus a priced-zero acceptance test.

P3 taken:

  • pricingRates now held to an exact object shape with non-negative rates — it accepted negative rates and unrecognized nested keys before. Audit quality, as you noted, but an unexplainable amount defeats storing the basis at all.
  • Non-empty now required on traceId, sessionId, runId, turnId, providerId, modelId, matching logicalCallId/attemptId.
  • Numeric gates now tested: -1, NaN, Infinity across costs and tokens.
  • Replay path: a replayed attemptId is now carried through sumModelCallCostUsd and summarizeModelCallCoverage on a multi-id stream, not just the bare dedupe helper.
  • Documented that the payload identity is the portable source of truth and must agree with the AgentRun envelope when written as an event.

P3 left as-is: pricingRevision/pricingRates stay optional on priced rows for v1, per your last note.

@maka/core is 1229/1229 with the new tests, biome lint and format clean.

On the metering-SoT wording — I'd rather land that line together with the writer in PR 2, so the doc comment doesn't claim an authority nothing populates yet. Say the word if you'd prefer it stated now.

Tests grew 16 → 21.

ARE404 added a commit to ARE404/maka-agent that referenced this pull request Jul 31, 2026
Addresses the P2 and P3 notes on apache#1687.

The codec forbade an unpriced record carrying an amount but still accepted a
priced record with no amount. Coverage counted that row as priced while
`sumModelCallCostUsd` skipped it, so a set of such rows read as "every call
priced, total $0" — indistinguishable from genuinely free, which is the exact
confusion the basis fields exist to prevent. `costBasis` and `costUsd` now
travel together in both directions; zero stays legal and remains the only way
to record a call that cost nothing.

Also tightens the surrounding validation, all audit and attribution quality:

- `pricingRates` is held to an exact object shape with non-negative rates.
  It previously accepted negative rates and unrecognized nested keys, which
  makes a recorded amount unexplainable and defeats storing the basis.
- `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must
  be non-empty, matching the existing treatment of `logicalCallId` and
  `attemptId`.
- Documents that the payload identity is the portable source of truth and must
  agree with the AgentRun envelope when the record is written as an event.

Tests: symmetric priced-without-cost rejection and a priced-zero acceptance,
negative and non-finite gates on costs and tokens, empty attribution keys, the
strengthened rate shapes, and a replayed `attemptId` carried through
`sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the
bare dedupe helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404
ARE404 force-pushed the are404/feat-model-call-accounting branch from 2187a32 to faf4b60 Compare July 31, 2026 09:13
ARE404 and others added 2 commits July 31, 2026 17:24
Introduces the accounting record that PR 2 will write from the provider
request seam, per the design settled in apache#1679. Contract only: no behavior
change, no writer, no read path.

`ModelCallAttempt` is one record per physical provider request attempt. It
extends the shape `ProviderRequestAttemptRecord` already carries with the
three things it lacks — call kind, resolved cost, and a logical-call link —
and adds a strict subtype codec, because the generic AgentRun event decoder
only checks that `data` is a record. For an accounting record an unvalidated
field silently becomes a wrong number in a cost report.

Two invariants are enforced by the codec rather than left to convention:

- `costUsd` is absent unless `costBasis` is `'priced'`. Zero means the call
  was genuinely free, never that the price was unknown.
- `usageBasis: 'missing'` cannot carry token fields. "The provider never
  reported usage" and "we have usage but no price" are separate failures and
  must not collapse into one counter.

`logicalCallId` is explicit rather than reconstructed from `(traceId, step)`:
`traceId` is per-tracker-instance, so it does not group a turn's calls, and a
compound key every consumer has to rebuild is the implicit contract this
record exists to remove. Retries are attempts of one logical call; terminality
is derived in projection, not stored.

`ModelCallCoverage` classifies the records present and documents that it is
not a completeness proof — nothing records an expected dispatch count, so an
attempt lost between dispatch and settlement is undetectable. `sumModelCallCostUsd`
returns coverage alongside the total for the same reason: a bare number cannot
express "plus an unknown amount from unpriced calls".

Registers `model_call_attempt_recorded` in `AGENT_RUN_EVENT_TYPES`, so the
record rides the existing AgentRun event seam — no new table, store, or writer,
and nothing for the apache#1649 persistence cutover to coordinate with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the P2 and P3 notes on apache#1687.

The codec forbade an unpriced record carrying an amount but still accepted a
priced record with no amount. Coverage counted that row as priced while
`sumModelCallCostUsd` skipped it, so a set of such rows read as "every call
priced, total $0" — indistinguishable from genuinely free, which is the exact
confusion the basis fields exist to prevent. `costBasis` and `costUsd` now
travel together in both directions; zero stays legal and remains the only way
to record a call that cost nothing.

Also tightens the surrounding validation, all audit and attribution quality:

- `pricingRates` is held to an exact object shape with non-negative rates.
  It previously accepted negative rates and unrecognized nested keys, which
  makes a recorded amount unexplainable and defeats storing the basis.
- `traceId`, `sessionId`, `runId`, `turnId`, `providerId`, and `modelId` must
  be non-empty, matching the existing treatment of `logicalCallId` and
  `attemptId`.
- Documents that the payload identity is the portable source of truth and must
  agree with the AgentRun envelope when the record is written as an event.

Tests: symmetric priced-without-cost rejection and a priced-zero acceptance,
negative and non-finite gates on costs and tokens, empty attribution keys, the
strengthened rate shapes, and a replayed `attemptId` carried through
`sumModelCallCostUsd` and `summarizeModelCallCoverage` rather than only the
bare dedupe helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404
ARE404 force-pushed the are404/feat-model-call-accounting branch from faf4b60 to 7000efa Compare July 31, 2026 09:24
@ARE404

ARE404 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto bfab3625 (current main, so it now includes #1690 and #1692). Two things changed, and the red test job is now a different failure than the one discussed above.

The runtime-host flake is resolved. steering becomes durable and ordered followups automatically start the next root passes on this run. #1690 fixed it — thanks @M4n5ter.

The current red is main's own breakage, not this PR. The test job now fails the CLI bootstrap suite:

✖ uses the default ready connection and requested model
✖ uses an explicit connection and forwards one-shot limits and invocation results
  Error: NO_REAL_CONNECTION:model_not_enabled

Evidence that it isn't from this branch:

  • main is red on bfab3625 itself — the exact commit this PR is rebased onto (run 30619541068), and has been red on every run since 08:49, while 07:13 (4153915b) was green.
  • The only commit in that green→red window is 7a5335d6 (fix(models): harden connection lifecycle and management UX #1691, fix(models): harden connection lifecycle and management UX). I haven't chased the mechanism inside it, so I'd rather flag the window than guess at the cause — but packages/cli/src/__tests__/connection-target.test.ts expects missing_api_key and missing_default_connection, and the reason now surfacing through resolveReadyTargetForConnection is model_not_enabled.
  • Reproduced locally on this branch at packages/cli/dist/__tests__/connection-target.test.js:708, with the same error.
  • This PR's diff is 5 files, all @maka/core, and touches nothing under cli, connection resolution, or model readiness.

Local on the rebased commit: @maka/core 1230/1230, @maka/runtime-host 416/416, typecheck and e2e green in CI.

Happy to open a separate issue for the CLI breakage if it isn't already being tracked — it's blocking every PR that rebases onto current main, not just this one.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review

Approve again on 7000efa8b.

The fix commit closes the P2 (priced must carry costUsd, zero still legal) and the P3 items we called out (projection-stream dedupe tests, numeric poison samples, tighter pricingRates, non-empty attribution keys, payload-identity docs). Dual pairing is in the codec with symmetric tests.

Nothing left is merge-blocking. Leftover notes are optional P3 only: unused isOptionalFiniteNumber import, float tokens, more exhaustive optional-field tests, status enum docs. Fine to land as-is; further tightening can wait for the writer PR.

CI test still red on NO_REAL_CONNECTION:model_not_enabled in CLI bootstrap tests. This PR only touches @maka/core accounting contract files and does not exercise that path.

@Astro-Han
Astro-Han merged commit 6380921 into apache:main Jul 31, 2026
2 of 3 checks passed
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 1, 2026
First half of the metering work in apache#1679: the provider request seam now
produces the canonical accounting record landed in apache#1687. The old writers are
untouched here and come out in the same PR before it opens.

`ProviderRequestTracker` gains an optional `accounting` input. It is one unit
rather than several independent fields because a `ModelCallAttempt` without
session, run, and call kind is unattributable — there is no useful state where
half of it is wired. Absent, the tracker stays purely diagnostic, which is what
the capture-only paths and their tests rely on.

Three behaviours the review on apache#1679 called for:

- **Settlement never throws.** `finalize` runs inside the stream's `pull`
  handler, so a rejection there reaches `controller.error` and fails an
  otherwise-complete model response. The dispatch-time gate is `assertReady`,
  checked before the provider is called and alongside the existing capture gate
  rather than as a second one. A sink failure after dispatch means the call
  happened and was billed but went unrecorded, which is reported rather than
  raised.
- **Cancellation and settlement are separate events.** An abort carrying no
  usage records provisionally without closing the attempt, so a `finish`
  arriving afterwards still settles it. Both writes share one `attemptId` and
  the record dedupes on that key keeping the last, so a cancelled call that
  really consumed tokens is no longer frozen as permanently token-less.
- **Retries are attempts of one logical call.** `logicalCallId` is assigned per
  step and reused across retries, so the grouping is explicit in the record
  instead of reconstructed from `(traceId, step)` by every consumer.

Cost resolves at settlement time and carries the rates it was computed against,
so a stored figure stays auditable when pricing later changes. An unresolvable
price records `costBasis: 'unpriced'` with no amount — never zero, which is
reserved for calls that genuinely cost nothing. `usageBasis` separately reports
whether the provider returned usage at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 1, 2026
First half of the metering work in apache#1679: the provider request seam now
produces the canonical accounting record landed in apache#1687. The old writers are
untouched here and come out in the same PR before it opens.

`ProviderRequestTracker` gains an optional `accounting` input. It is one unit
rather than several independent fields because a `ModelCallAttempt` without
session, run, and call kind is unattributable — there is no useful state where
half of it is wired. Absent, the tracker stays purely diagnostic, which is what
the capture-only paths and their tests rely on.

Three behaviours the review on apache#1679 called for:

- **Settlement never throws.** `finalize` runs inside the stream's `pull`
  handler, so a rejection there reaches `controller.error` and fails an
  otherwise-complete model response. The dispatch-time gate is `assertReady`,
  checked before the provider is called and alongside the existing capture gate
  rather than as a second one. A sink failure after dispatch means the call
  happened and was billed but went unrecorded, which is reported rather than
  raised.
- **Cancellation and settlement are separate events.** An abort carrying no
  usage records provisionally without closing the attempt, so a `finish`
  arriving afterwards still settles it. Both writes share one `attemptId` and
  the record dedupes on that key keeping the last, so a cancelled call that
  really consumed tokens is no longer frozen as permanently token-less.
- **Retries are attempts of one logical call.** `logicalCallId` is assigned per
  step and reused across retries, so the grouping is explicit in the record
  instead of reconstructed from `(traceId, step)` by every consumer.

Cost resolves at settlement time and carries the rates it was computed against,
so a stored figure stays auditable when pricing later changes. An unresolvable
price records `costBasis: 'unpriced'` with no amount — never zero, which is
reserved for calls that genuinely cost nothing. `usageBasis` separately reports
whether the provider returned usage at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Astro-Han pushed a commit that referenced this pull request Aug 1, 2026
…1755)

* feat(runtime): emit canonical ModelCallAttempt from the provider seam

First half of the metering work in #1679: the provider request seam now
produces the canonical accounting record landed in #1687. The old writers are
untouched here and come out in the same PR before it opens.

`ProviderRequestTracker` gains an optional `accounting` input. It is one unit
rather than several independent fields because a `ModelCallAttempt` without
session, run, and call kind is unattributable — there is no useful state where
half of it is wired. Absent, the tracker stays purely diagnostic, which is what
the capture-only paths and their tests rely on.

Three behaviours the review on #1679 called for:

- **Settlement never throws.** `finalize` runs inside the stream's `pull`
  handler, so a rejection there reaches `controller.error` and fails an
  otherwise-complete model response. The dispatch-time gate is `assertReady`,
  checked before the provider is called and alongside the existing capture gate
  rather than as a second one. A sink failure after dispatch means the call
  happened and was billed but went unrecorded, which is reported rather than
  raised.
- **Cancellation and settlement are separate events.** An abort carrying no
  usage records provisionally without closing the attempt, so a `finish`
  arriving afterwards still settles it. Both writes share one `attemptId` and
  the record dedupes on that key keeping the last, so a cancelled call that
  really consumed tokens is no longer frozen as permanently token-less.
- **Retries are attempts of one logical call.** `logicalCallId` is assigned per
  step and reused across retries, so the grouping is explicit in the record
  instead of reconstructed from `(traceId, step)` by every consumer.

Cost resolves at settlement time and carries the rates it was computed against,
so a stored figure stays auditable when pricing later changes. An unresolvable
price records `costBasis: 'unpriced'` with no amount — never zero, which is
reserved for calls that genuinely cost nothing. `usageBasis` separately reports
whether the provider returned usage at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(core): project Usage aggregates from the canonical attempt ledger

Second commit of the metering slice in #1679. Pure aggregation over
`ModelCallAttempt`, serving the shapes the Usage authority reads today —
`summary`, `buckets`, `logs` — so the read path can move off the per-send
`LlmCallRecord` table in the next commit. Nothing is wired yet.

One behavioural difference from the old table is deliberate, and is the reason
this projection exists rather than a schema-compatible one. `totalCostUsd` sums
only records whose price was resolvable, and every result carries the coverage
that qualifies it. The old schema had no way to say "this call cost something
we could not price", so it stored zero, and unpriced spend was indistinguishable
from a free call. A total presented without its coverage repeats that claim.

Two mapping decisions worth stating:

- `interrupted` projects to `aborted`, not `error`. Both mean the call stopped
  short without the provider reporting a failure, and folding it into `error`
  would inflate the error rate with user cancellations.
- Log rows carry `logicalCallId` as `callId` and keep session and turn
  attribution, so a row remains traceable back to the conversation that caused
  it rather than only to a model.

Selection dedupes on `attemptId` before aggregating, so a re-appended
settlement — the abort-then-late-usage path from the previous commit — counts
once.

Tool telemetry is untouched: `toolLogs` reads tool invocations, which this
ledger does not describe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(metering): read Usage from the canonical ledger and drop the send-level meter

Third and last commit of the metering slice in #1679, and the atomic one: the
Usage authority moves onto `ModelCallAttempt` and the writer it replaces comes
out in the same change, so no commit leaves two independent meters running.

**Where canonical records land.** The AgentRun stream is the durable log —
`model_call_attempt_recorded`, `durable: true`, because a record lost to a
crashed flush is spend nothing else can reconstruct. It is not the read model:
`AgentRunStore.readEvents(sessionId, runId)` answers "what happened in this
run", and no Usage question is shaped that way. So each attempt also lands in
`usage_model_call_attempts`, a new table beside the old one rather than inside
it — `usage_llm_calls` cannot express `usageBasis` or `costBasis`, and writing
canonical records through that schema is the dishonest-zero problem again.

**The read path sums two sources.** The old table is frozen, never migrated, and
still receives the `semantic_compact` and `history_compact` calls that have not
been routed through the canonical seam yet. Every LLM-sourced result therefore
carries `UsageProvenance`: the coverage of its canonical half, how many rows came
from the frozen table, and how many stored records failed to decode. A total on
the wire without that cannot be read honestly, so it is part of the wire.
`legacyRecords` reaches zero on its own as the old table ages out of the range.

**What comes out.** The terminal `recordLlmCall` in the streaming backend and the
`usage_recorded` RunTraceEvent. Both measured the same provider requests the
seam now settles per request instead of per send. `token_usage` SessionEvents and
the RuntimeEvent per-turn aggregates stay — they feed replay and recovery and are
not accounting.

**What the deletion nearly took with it.** That record was also the only durable
home for the send's terminal context diagnostics. The exhausted and aborted paths
emit no `token_usage` event, so their compaction decisions, their final request
shape, and the accumulated usage of the steps that did complete would have gone
with it. They move to a `send_diagnostics_recorded` run-trace event, which
carries no cost and meters nothing. The request-shape hashes needed no rescue —
`model_stream_started` already carries them for step 0, and the new event carries
the final shape a same-turn tool load produces.

Two gaps the end-to-end acceptance found in the seam from the first commit:
canonical records carried neither `connectionSlug` nor the connection's provider
type, so they were attributable to a provider and model but not to a connection,
and `moonshot.chat` would have split one provider across two bucket keys against
the historical rows.

The Desktop runs a second, complete metering stack against its own store. Deleting
the send-level writer globally would have stopped its Usage page accruing, so it
reads and writes through the same ledger. That is wiring, not authority: nothing
here gives the embedded writer ownership, election, or admission.

Acceptance is end-to-end and now green against a real provider wire: provider
call → attempt record → projection → Usage surface.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(metering): make the Usage ledger a projection of one authority

Addresses the review on #1755.

**P1 — the Usage table was an independent write, not a recoverable read model.**

The finding is right, and it undercut the claim the previous commit made about
itself. Both hosts wrote each `ModelCallAttempt` to the AgentRun stream and to
`usage_model_call_attempts` through `Promise.allSettled`, with nothing ever
replaying one into the other. Calling the stream "the log of record" does not
make it one: either write could land alone, and the Host's answer was to request
a drain it could not recover from while the Desktop swallowed the failure and
kept dispatching. That is two sources of truth, which is the thing this whole
change set exists to remove.

There is now one commit point. The AgentRun append is awaited first and
`AgentRun.recordModelCallAttempt` rejects on failure instead of resolving, so a
caller can tell whether the authority actually holds the record; the ledger is
written only after it does. Rejecting is safe because the seam still swallows it
— settlement runs inside the stream's `pull` handler, and a completed, billed
response must never be failed by its own bookkeeping.

A failed projection is recoverable. The run is marked in
`usage_model_call_reprojection`, and the Usage read path re-derives marked runs
from the stream before answering, bounded per query so a backlog drains across
reads instead of inside one. The projection is idempotent — it upserts on
`attemptId` — so repairing a run the table already holds is a no-op, and one
undecodable event does not block the rest of its run.

The marker is an optimization, not the correctness argument. Losing it costs a
targeted repair, not the records: the authority holds every attempt, so a full
re-projection recovers a run whose marker was never written. What a pass cannot
repair is reported as `provenance.pendingRepairs` rather than silently missing
from the totals — spend that is recoverable is a different claim from spend that
is counted.

The pre-dispatch gate moves with it. It now keys off the authority alone: a
stale projection is recoverable and must not block a send, but an authority that
cannot accept the record means the next dispatch produces spend nothing will
ever hold.

**P2 — one bucket-key function.**

The two sources derived "the same hour" differently — an epoch-hour ordinal
against an ISO hour — so `mergeUsageBuckets` saw two keys and split one hour in
half without failing anywhere. Both now call `usageBucketKey`, and a
mixed-source hourly query is a contract test. `day` happened to agree already;
that it did was luck, not design.

**P2 — a log row keeps its cost basis.**

`projectModelCallUsageLogs` mapped an unpriced attempt to `costUsd: 0`, so a
genuinely free call and a call whose price could not be resolved were
indistinguishable per row — page-level coverage says how many were unpriced, not
which. This reproduced, at row granularity, exactly the ambiguity the coverage
breakdown was added to remove. `UsageLogRow` and the wire projection now carry
`costBasis`, `costUsd` is absent for an unpriced row, and the codec rejects a
row that claims both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(metering): read the accounting gate's own flag and keep repair honest

Second review round on #1755.

**The accounting gate was dead code.** `recordModelCallAttempt` set
`accountingAuthorityFailed`, and `assertModelCallAccountingReady` checked
`telemetryDrainRequested` — a flag that tracks the frozen legacy table, which no
longer meters main sends at all. Nothing read the new flag, so a host whose
authority had stopped accepting records kept dispatching provider calls it could
never account for. The gate the previous commit introduced never actually
closed; it does now.

**The pending marker is written before the projection, not after it fails.** A
marker written from a catch block cannot cover the case where the catch never
runs — the process exiting between the authority append and the projection —
which left a committed record this table would never learn about. Marking first
turns it into an intent record: a crash anywhere after it still leaves a run the
repair finds.

That does not close the window entirely, and the docs no longer say it does. A
process that dies between committing to the authority and writing the marker
still leaves a record Usage will not see, because nothing sweeps the whole
stream. The claim that "a full re-projection recovers a run whose marker was
never written" described a sweep that was never implemented; the comment and the
evidence-spine doc now state the real limit instead.

**An unreadable authority event no longer disappears with the marker.** The
repair decoded what it could, cleared the run, and dropped the undecodable
count — so a real call fell out of the totals and out of the pending count in
the same step, leaving nothing to say it existed. Clearing the marker is still
right (it will not decode next pass either, and holding the run would stall
every later repair behind it), so the count travels out on the result and folds
into `provenance.unreadableRecords` alongside stored rows that will not decode.

Two review items are deliberately left as follow-ups, and the PR description
says so rather than implying they are covered: the Desktop Usage page still
reads the session-derived `settings.usageStats` path and has no consumer for the
canonical endpoints, and the Daily Review archives a merged total without
carrying its provenance into the saved summary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Astro-Han pushed a commit that referenced this pull request Aug 3, 2026
…#1956)

* feat(inspector): project a per-session causal trace over both ledgers

First slice of #1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (#1687, #1755, #1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.

Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.

Three properties are deliberate:

- **Retries are nested, not flattened.** Attempts of one logical call share a
  `logicalCallId` by contract, so "this call was retried twice" is a grouping
  rather than something a reader reconstructs from four steps that happen to
  share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
  no `costUsd`, and a session of only such calls totals to no price rather than
  to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
  produces a trace that says so. The pi backend is exactly this case: it emits
  `token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
  indistinguishable from a session that did nothing. Aggregate usage with no
  record behind it is the signal, reported per session as `absent` or `partial`
  with the turns named.

Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.

No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.

`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(inspector): correct six attribution and coverage gaps in the trace

Review on #1956. All six were right; four are cases the tests never drove and
two were contract surface that promised more than the code delivered.

**Coverage claimed a proof it cannot have.** `complete` said every call settled;
the canonical contract says present settlements cannot prove completeness. The
state is now `no_known_gap` — the absence of evidence of a gap. A detectable
shortfall is also new: `tokenUsage.runtimeSteps` states how many tool-loop steps
one aggregate stands for, and fewer main calls on record than that is a
disagreement between the two ledgers, reported per turn as a floor on what is
missing rather than a count of it.

**Attempts are deduplicated by `attemptId`.** An aborted attempt and its later
settlement are appended under one id; the ledger dedupes on write, a stream read
does not. Without `dedupeModelCallAttempts` the trace invented a retry and
double-counted a priced settlement, which would have put a session total out of
step with Settings → Usage over the very same records. Grouping now goes through
core's `groupModelCallAttempts`, which dedupes on the way in.

**Step-less turns had non-finite bounds.** Usage-only and text-only turns project
no steps, and folding an empty list gives ±Infinity — which JSON renders as
`null`. Bounds now come from the ledger facts the turn is made of.

**A handled tool failure no longer fails the turn.** Any failed step marked the
whole turn failed, including a tool error the model recovered from before
finishing normally. The ledger's terminal status decides whether the turn failed;
the failed step only locates the cause once that is established.

**The compaction step is emitted rather than merely declared.** `TraceCompactionStep`
existed and nothing produced it. Written checkpoints are system text events, and
they are a different fact from the `history_compact` model call: one is the
boundary the next request replays from, the other is the spend. Both now appear.

**`recoveryMode` was a policy wearing the name of an outcome.** Every dispatch
declares one, including ordinary first executions. It is now `recoveryPolicy`,
and an actual `recovered` decision is joined from `actions.toolRecovery` by
`operationId` — correlated rather than positional, because the decision is
appended by the recovery writer and not by the dispatch it settles.

Seven regression tests, one per finding plus the no-known-gap state.

`@maka/core` 703/703, `@maka/runtime` 2742/2757 — the 6 local failures are this
machine's `rg`-as-a-shell-function and macOS path noise; CI ran the same base
green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants