feat(wiki): validate and normalize frontmatter at write time in write_governed_page - #109
Merged
Merged
Conversation
…_governed_page write_governed_page persisted caller-supplied markdown verbatim without re-deriving or validating frontmatter -- how the `title: title: "..."` corruption (repaired reactively at read-time by commit 53712df/PR #104) got onto disk in the first place. Adds a write-time gate (core/wiki_frontmatter_validation.py) that round-trips every full-page write through parse_page + render_page so persisted bytes are always canonical, closing the corruption class instead of known signatures. Only the one shape parse_page cannot repair -- an unclosed frontmatter fence, which would silently swallow the body into misparsed keys -- is rejected (raises, becomes a ToolError via safe_handler, the repo's standing idiom). append mode's content is a fragment, not a full page, and is never normalized. Headless-authoring call sites in consolidation/page_io.py catch the new exception explicitly so a malformed page can't crash the batch drain cycle. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Jul 14, 2026
8 write paths bypass write_governed_page — governance and write-time normalization not enforced
#110
Closed
cdeust
added a commit
that referenced
this pull request
Jul 14, 2026
Move Function / Split Module (Fowler) — wiki_pages.py (460 lines) split along its three cohesive concerns into new core modules, with wiki_pages.py becoming the composed public-API entry point (all prior exports preserved, same names/signatures — this is the module's public contract per issue #106's coordination requirement with in-flight PR #109). - wiki_frontmatter.py (164 lines) — PageDocument, parse_page, render_page, frontmatter parsing/rendering internals. - wiki_page_builders.py (230 lines) — build_adr/spec/file_doc/note/lesson/ convention/reference, ADR_STATUSES, maturity_label, sources/related section helpers. - wiki_index.py (79 lines) — build_index (INDEX.md generation). - wiki_pages.py (54 lines) — re-exports the full prior public API. Before: 460 lines, 1 file. After: 527 lines across 4 files, none exceeding 230 lines (§4.1/§4.3). Tests: 466 passed / 1 pre-existing failure (missing psycopg in this env, unrelated to wiki_pages) before and after — 0 regressions, 0 tests modified. ruff format --check + ruff check: pass on all 4 files. Co-authored-by: Claude <noreply@anthropic.com>
cdeust
added a commit
that referenced
this pull request
Jul 14, 2026
…on all write paths Issue #110: 8 call sites bypassed write_governed_page, so PR #109's write-time frontmatter normalization only protected the interactive wiki_write tool. Move normalize_frontmatter into wiki_store.write_page itself (the true choke point every caller shares) so no path — governed or direct — can persist a non-canonical frontmatter shape. Deduplicate the now-redundant normalize call in write_governed_page, correct its docstring's false "only function that may call write_page" claim, and add an architecture test enumerating the audited whitelist of direct callers so a new unreviewed bypass fails loudly instead of shipping silently. Also fixes a regression introduced mid-change: routing normalization through write_page's own try/except let UnclosedFrontmatterError (a ValueError subclass) get silently swallowed into an error dict instead of propagating. Co-Authored-By: Claude <noreply@anthropic.com>
cdeust
added a commit
that referenced
this pull request
Jul 14, 2026
…on all write paths (#112) Issue #110: 8 call sites bypassed write_governed_page, so PR #109's write-time frontmatter normalization only protected the interactive wiki_write tool. Move normalize_frontmatter into wiki_store.write_page itself (the true choke point every caller shares) so no path — governed or direct — can persist a non-canonical frontmatter shape. Deduplicate the now-redundant normalize call in write_governed_page, correct its docstring's false "only function that may call write_page" claim, and add an architecture test enumerating the audited whitelist of direct callers so a new unreviewed bypass fails loudly instead of shipping silently. Also fixes a regression introduced mid-change: routing normalization through write_page's own try/except let UnclosedFrontmatterError (a ValueError subclass) get silently swallowed into an error dict instead of propagating. Co-authored-by: Claude <noreply@anthropic.com>
cdeust
added a commit
that referenced
this pull request
Jul 14, 2026
…dening Cut immediately after v4.14.0 to carry a fix that landed on main just after that tag: - fix(mcp): safe_handler raises ToolError from handler failure instead of returning a schema-violating error dict, so a failing MCP tool call surfaces its real diagnostic instead of the generic "Output validation error" every client previously saw regardless of cause (49f29e9, c7dfc24). - fix(wiki): strip quoted/duplicated-label frontmatter scalar values that broke FS↔PG sync (#104, 53712df). - fix(wiki): wiki_migrate --dry-run made transactionally neutral (#108, b60d268). - feat(wiki): write_governed_page validates and normalizes frontmatter at write time instead of only at read time (#109, 3eb43f2). - refactor(wiki): wiki_pages.py extracted into cohesive collaborators, no behaviour change (#111, 28145f0). Marketplace: cortex 4.14.0 -> 4.14.1 (metadata + pin + version_note). Pre-tag guard (exact tree): LongMemEval MRR 0.9167/R@10 0.982 (matches v4.14.0 0.9166/0.982); LoCoMo 3-run mean MRR 0.7984/R@10 0.9142 (v4.14.0 was 0.8005/0.9131 — delta MRR -0.0021 is smaller than v4.14.0's own documented +-0.0025 single-run variance, not a regression; R@10 improved); BEAM 0.5406 (3 runs, spread 0.0001) vs v4.14.0's 0.5471. BEAM deviation investigated: same-day A/B + bisect + control rerun (9 runs total) show an intra-day production-DB variance band of 0.5391-0.5445 spanned by BOTH trees — a same-day v4.14.0 control run measured 0.5445, then on rerun ~2.5h later 0.5391 (SAME code, SAME tree), and a bisect at 53712df/49f29e98 landed inside that band with no monotonic step at any commit boundary. Code effect excluded on two independent grounds: (1) the bench import closure (BenchmarkDB -> mcp_server.core.memory_ingest + mcp_server.core.pg_recall) is untouched by any 4.14.1 commit — the full mcp_server diff between v4.14.0 and this tag is confined to wiki_*, handlers/consolidation/ page_io.py, handlers/wiki_migrate.py, handlers/wiki_write.py, and tool_error_handler.py, none of which is in that import graph; (2) the SAME v4.14.0 tree reproduces both the high and low ends of the band on rerun. BEAM is not gated by reproduce.sh's floor check (proxy metric, within-system comparison only) so this does not block the tag. Floor-gate result (reproduce.sh::check_floors logic, applied directly to the JSONs since no runner prints it itself; tolerance 0.005): LongMemEval R@10 0.9820 vs floor 0.9820 PASS; LongMemEval MRR 0.9167 vs floor 0.9140 PASS; LoCoMo R@10 0.9142 vs floor 0.9150 PASS; LoCoMo MRR 0.7984 vs floor 0.8050 FAIL (-0.0066, exceeds the 0.005 tolerance) — flagged, not hidden: fails the codebase's hardcoded absolute floor while passing the release's actual stopping rule (no regression beyond noise vs the v4.14.0 baseline). The absolute floor has not been re-baselined since the 4.11-4.13 measurement era and LoCoMo MRR has trended down release-over-release with no corresponding retrieval-code change; recommend recalibrating the floor in a future session against an isolated benchmark DB rather than the live growing production store used here. Reranker active. Results committed under benchmarks/results/repro/20260714-v4.14.1-pretag/ (11 files: 5 primary runs, beam-100K-rep2/rep3, and 4 investigation runs from the bisect: beam-control-v4140, beam-bisect-53712df8, beam-bisect-49f29e98, beam-bisect-da3f2104-rep2). Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #107
Design decision
Write-time normalization (option retenue dans l'issue) : chaque écriture full-page passant par
write_governed_pageest round-trippée viaparse_page/render_page, donc les octets persistés sont toujours la forme canonique. Cela ferme la classe entière de corruption (donttitle: title: "...", 62 lignes réparées par la migration du 2026-07-14) au lieu des seules signatures connues. La tolérance read-time du fix #104 reste en défense en profondeur.Implementation
mcp_server/core/wiki_frontmatter_validation.py(normalize_frontmatter) — importe uniquementwiki_pages, aucun I/O.wiki_pages.pynon touché (refactor wiki_pages.py (452 l.) au-delà de la trajectoire de taille — extraire des collaborateurs #106 en cours à part).write_governed_page(mcp_server/handlers/wiki_write.py) normalise tout writemode != "append"; seul un fence de frontmatter jamais fermé (structurellement inexploitable) est rejeté viaUnclosedFrontmatterError→ToolErrorparsafe_handler.mcp_server/handlers/consolidation/page_io.py: gestion d'exception aux call-sites existants.Tests
safe_handler, modeappendjamais normalisé.Hors scope (signalé, à traiter séparément)
8 chemins d'écriture appellent
wiki_store.write_pagedirectement en contournantwrite_governed_page(ingest_prd, ingest_findings_writers, wiki_rename, ingest_codebase_pages, wiki_adr, wiki_link, wiki_compile, append_section), contredisant la docstring « the ONLY function that may call write_page ». Follow-up recommandé.🤖 Generated with Claude Code