fix(runtime): history compaction accepts truncated/unstructured LLM summaries without validation

[me2seeks]

Problem

A history compaction wrote a checkpoint whose LLM summary was a 138-token free-form fragment that (a) ignored the mandated structured format, (b) ended mid-sentence, and (c) omitted all load-bearing context (which files were changed, current worktree state, test status). The continuing model then confabulated implementation details (it described a TypeScript CLI change as non-existent Rust files), which looked like foreign content leaking into the session.

Incident evidence (session fbdb3fd3, checkpoint hcheckpoint-981ceab88738d4819971747aeb292203):

• Compaction folded 742 RuntimeEvents / ~235,503 estimated tokens into a 138-token summary (~1700:1).
• The summary text: 确认服务端语义后，决定：…现在看 desktop 的 retry 循环结尾： — free-form, ends with a colon mid-sentence, and follows none of the SUMMARIZATION_SYSTEM_PROMPT sections (## Goal / ## Progress / ## Key Decisions / ## Next Steps / ## Critical Context).
• The run ledger shows 3 failedOpen attempts with failOpenReason: provider_error immediately before the write that succeeded — the accepted response was produced right after repeated provider failures and is consistent with a degraded/truncated provider completion.

Root cause

history-compact-summarizer.ts + ai-sdk-compaction.ts accept any non-empty string as a checkpoint summary:

1. The only rejection gate is empty_summary (!summary?.trim()). There is no structural validation (required sections), no minimum-size sanity check relative to the folded span, and no truncated-output detection beyond finishReason === 'length' — which does not catch providers that return partial text with a stop finish reason (observed with the session's own model serving as summarizer).
2. The summarizer reuses the session model (resolveModel()). A weak instruction-follower can silently produce a free-form note instead of the structured checkpoint, and the pipeline records it as a valid checkpoint with decision: 'replaced', replacing 235k tokens of history.

No cross-session data is involved in this incident: buildHistoryCompactCheckpoint rejects multi-session event sets, and the checkpoint's coverage references this session's own event projection. The "foreign" content was reference material deliberately fetched into the session (a codex-rs source study requested by the user) that the thin summary failed to contextualize, leaving the continuation model to hallucinate around it.

Proposal

1. Structural validation of summaries: require the mandated section headers (## Goal, ## Progress, ## Next Steps at minimum) and reject/flag summaries that end mid-sentence (e.g. trailing colon/open code fence). Failure → HistoryCompactSummarizerError('malformed_summary') → fail-open (keep history, retry next turn) instead of persisting the fragment.
2. Size sanity bound: reject summaries smaller than a floor relative to the folded span (or an absolute floor such as ~200 tokens when folding >10k tokens), so a 235k→138 collapse cannot be accepted silently.
3. Truncation robustness: treat missing/unknown finish reasons on suspiciously short output as output_length (retryable), and record the summarizer's finishReason/usage in compaction diagnostics.
4. Deterministic facts appendix: attach model-independent anchors to the checkpoint — files touched (from tool-call args), task ledger snapshot, git worktree/branch — so the continuation has ground truth even when the prose summary is thin.
5. Consider a dedicated (stronger or cheaper-but-reliable) summarizer model rather than always reusing the session model, or at least a retry with elevated instructions when validation fails.

Acceptance

• A degraded provider response (short, section-less, mid-sentence) can never replace folded history; the turn fails open and retries.
• Compaction diagnostics record summarizer finishReason, output tokens, and validation outcome.
• Regression test: a summarizer stub returning the incident's 138-token fragment fails validation and the checkpoint is not written.

Generated-by: Maka

[me2seeks]

Reproduction confirmed

Reproduced on three levels:

E1 — deterministic (pipeline gate). Driving planMidTurnCapacityCompaction with a stub summarizer returning the incident's exact fragment yields decision: 'compacted' and a written checkpoint. Boundary sweep:

REJECTED  empty string / whitespace only        (existing empty_summary gate)
ACCEPTED  single word "done"
ACCEPTED  incident fragment (mid-sentence colon)
ACCEPTED  unclosed code fence
ACCEPTED  section-less prose

E2 — real model (DeepSeek deepseek-chat, real incident events as input). Two consecutive calls both returned ~160 tokens of raw tool-call markup (<｜｜DSML｜｜tool_calls>…) instead of the structured summary — zero required sections present — and the current gate accepts it. So a real weak-model response silently replaces folded history.

Related defect found during reproduction (Bug B): replayPlanItemsToModelMessages in history-compact-summarizer.ts emits each parallel tool_call as its own assistant message. Strict OpenAI-compatible providers reject this: DeepSeek returns 400 "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'". The primary replay path's materializer explicitly merges a step's tool calls into one provider message (see comments in model-history.ts); the summarizer path diverges. This matches the incident run ledger: 3 provider_error fail-opens immediately before the degraded write succeeded. Filed separately as #3030.

Generated-by: Maka