Skip to content

ingest_codebase doc snapshots have no freshness policy — store served README/CLAUDE.md 2–4 months behind HEAD, including a figure the repo had already retracted #381

Description

@cdeust

Observed (A/B bench rev.2, B-Cortex probe session, 2026-08-07)

In a probe session answering documentation questions about the Cortex repo itself, the document store served verbatim-real but superseded doc snapshots:

  • README content matching revision f80bf539 (2026-06-11): "26 mechanisms · 49 tools" — live HEAD says "36 · 52".
  • CLAUDE.md content matching v2.6.0 (2026-03-29): layer counts 11/108/21/33/4 — measured live: 26/230/98/138/0; server/ is now empty (viz stack moved to cortex-viz).
  • The "108 modules" figure was explicitly retracted by docs/module-inventory.md 9 days before the probe — the store still served it.
  • Storage default answered as PostgreSQL; live default is SQLite (PRIVACY.md).

Zero fabrication — every claim traced to a real historical revision. The failure mode is a time-lagged mirror presented as current.

Evidence: harness-comparison bench, results/COMPARISON-rev2.md (B-Cortex verdict detail + error-pattern section) and results/probes/B-Cortex.json. The code-graph outage in that session (†) is unrelated: the doc store was reachable and answered.

Root cause (policy gap, not a bug in any one function)

Doc snapshots captured at ingest time (ingest_codebase / seeding) are stored as memories with no source-revision stamp and no invalidation or re-verification on subsequent ingests of the same repo. Recall then serves them indistinguishably from current facts.

Expected — any of these closes the gap, in order of strength

  1. Re-verify/re-ingest on ingest_codebase: when re-ingesting a repo, diff stored doc-derived memories against the live files they came from; supersede (version-chain) the stale ones.
  2. Provenance stamping at serve time: memories derived from repo files carry {source_path, revision/mtime, captured_at} and recall surfaces it, so a consumer can see "captured 2026-06-11" next to a 2026-08 question.
  3. Staleness scoring: core/staleness.py (file-reference staleness) already exists — wire it into the recall path for doc-derived memories.

Related: #359 (wiki rot — same disease, wiki surface), #345 (write-time provenance hints), #368 (no provenance dimension in WRRF ranking). This issue is the doc-snapshot/memory instance with hard bench evidence.

Why it matters

This was the single dominant error pattern of the ai-architect.tools harness across the whole bench (stale counts on 3 of 4 repos, stale line numbers, one retracted figure re-served). Fixing the freshness policy converts most of the harness's "partial" verdicts to "correct" — it is the highest-leverage retrieval-quality fix identified by the comparison.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions