test(ast): pin the language extractor walkers — 241 unpinned mutants → 6 documented equivalents (#369) - #373

cdeust · 2026-08-06T14:46:09Z

Closes the walker half of #369. Refs #372.

Result

	before	after
not-killed mutants, seven walkers	241	6 (all documented equivalents)
killed, five extractor modules	571	847

_walk_java, _walk_kotlin, _walk_csharp, _walk_ruby, _walk_php, _extract_swift_node and _walk_type are now at zero undocumented survivors. The six that remain are all in _walk_for_calls and each carries a rationale at the use site.

Why it mattered

These extractors decide what enters the code graph. A wrong qualified name, a dropped nested type or a method attributed to the wrong scope degrades ingest_codebase, the wiki reference pages and unified_search with no failing test and no signal. test_ast_extractors.py covered Python, JS and Go; the other six languages had essentially nothing. #370 had to lean on a purpose-built differential harness against the old implementation precisely because the suite could not have caught an ordering regression.

Approach

Each language asserts the exact ordered (name, kind) list, not membership. A membership assertion cannot distinguish a correct walker from one that emits the right names in the wrong scope, and order is contract here: the walkers are depth-first pre-order and consumers see that order.

Every expectation was measured against the implementation and then read back against the source to confirm it is intended contract rather than a bug being frozen.

Three surprises, pinned rather than silently accepted

Behaviour	Cause
Swift `enum` and `struct` report kind `class`	tree-sitter-swift parses all three as `class_declaration`, so `_SWIFT_KIND_MAP`'s `enum_declaration` / `struct_declaration` entries never match
Kotlin `object O` emits no entry for `O`, but `object Named : Base()` does	the supertype is what makes the grammar produce a `type_identifier`
Ruby `def self.x` is dropped entirely	parses as `singleton_method`, which the walker does not handle

These are pinned as current behaviour. Changing any of them now requires editing a failing assertion, which is the point.

The mutation gate could report a false clean — fixed

scripts/mutation_check.sh printed none — 0 surviving mutants 🎉 on a five-file run whose own progress line ended at 🙁 423, with mutmut results returning a 915-line non-empty listing. A gate that reports clean when it is not is worse than no gate, because the ledger row it produces gets believed.

It now derives two independent counts — the run's 🙁 tally and the results listing — and refuses to render a verdict at all when they disagree, rather than taking the friendlier number. Fail closed. The cleanup trap wipes mutants/ and .mutmut-cache on exit, so a disagreement cannot be investigated afterwards; it has to be caught inside the run.

Verified live at the exact scope that produced the false green: the script now exits 1 and reports 175 GENUINE surviving mutant(s) — a real test gap.

On the `decorated_definition` arm

I first removed it as dead code, on the grounds that it is byte-identical to the else that follows. That was wrong, and the reasoning error is worth recording: behavioural equivalence tells you a branch is currently redundant and says nothing about why it exists.

It mirrors _extract_python_children's dispatch, where the same branch is load-bearing — that function handles three node types with no catch-all, so without it decorated definitions vanish from definitions entirely. And it is the seam for behaviour that was never filled in:

@app.route("/api")
@retry(times=3)
def handler():
    work()

extract_calls_per_function -> {'handler': ['work']}

route and retry produce no edge anywhere. Deleting the arm would have foreclosed that rather than answered it. Arm restored, reasoning at the site, gap filed as #372 with the design question attached.

Completion Ledger

§	Item	Evidence
A1	Happy paths	`pytest -k "ast or extractor or codebase"` → 1181 passed, 5968 deselected
A2	Edge cases	Per language: empty source, unnamed/anonymous construct (Java anonymous class), nested type, type-without-name fallthrough, method outside any type, namespace-level method, object with and without supertype
A3	Failure paths	No new failure path. The unreachable `else ""` name fallback is documented as such at the use site
A4	Trust boundary	Fixtures include fragments and partial files, which is what Cortex actually indexes
A5	Invariants	Qualifier rule stated once in `TestScopeQualification` and asserted for Java, C# and Ruby rather than implied six times
A6	Idempotency	Pure functions over a parsed tree
B1–B3	Concurrency	N/A
C1–C3	Resources / perf	N/A — tests plus comments; the one source change is comment-only
D1–D3	Security	N/A
E1	API compatibility	No signature or behaviour change. `git diff origin/main -- mcp_server/core/ast_extractors.py` is comment-only
E2	Downstream consumers	Verified by differential rather than inspection: 1344 files, old vs new `extract_calls_per_function` including dict key order, 0 mismatches
E3–E4	Persisted data, cross-platform	N/A
F1–F2	Observability	The mutation gate's signal is the subject; it now fails closed instead of emitting a false green
G1	Path→test ledger	6 language walkers → 6 exact-list tests + 10 targeted branch tests; `_walk_for_calls` → 5 call-extraction tests
G2	Regression test fails pre-fix	Not applicable as a bug fix, but each of the 19 intermediate survivors was traced to the fixture that kills it, verified by re-running mutation after each addition (241 → 19 → 6)
G3	Determinism	Suite run three times across the change; 1181 each time
G4	Negative assertions	Anonymous class contributes nothing; Ruby singleton method absent; Kotlin `O` absent; default-argument calls absent from the call map; empty map for a source with no definitions
G5	Full gate	`ruff check .` → All checks passed! · `ruff format --check .` → 1276 files already formatted · `pyright mcp_server/` → 0 errors, 0 warnings, 0 informations
H1	Standards	No layer change; comment-only source diff
H2	Readability	Each pinned surprise says why the behaviour is what it is, not just that it is
H3	Conventions	Matches the existing extractor test style
H4	CHANGELOG	N/A — tests, comments and a developer-tooling fix; no consumer-observable behaviour change
H5	Commit hygiene	One commit; formatting applied by `ruff format` to touched files only
H6	CI green	Required before merge
H7	Boy-scout (§14)	Two defects found en route and neither waved through: the gate's false green is fixed here; the decorator-call gap is filed as #372 with a design question, not deferred as prose

What this does not close

#369 also covers the non-walker functions in the same five files — _extract_rust_node, extract_go_definitions, _extract_python_class, _extract_python_func and others. The scoped runner is file-granular, so it still exits 1 with 177 survivors across those. That is the remaining tail of #369 and the issue stays open for it; this PR does not claim otherwise.

Not-killed mutants in the seven walkers: 241 -> 6, all six documented equivalents. Killed across the five extractor modules: 571 -> 847. The Java, Kotlin, C#, Ruby, PHP and Swift extractors had almost no behavioural tests — test_ast_extractors.py covered Python, JS and Go only — so a wrong qualified name or a dropped nested type would have reached ingest_codebase, the wiki reference pages and unified_search with no failing test. The recursion fix in #370 had to lean on a purpose-built differential harness precisely because the suite could not have caught an ordering change. Each language asserts the exact ordered (name, kind) list, not membership: a membership assertion cannot tell a correct walker from one emitting the right names in the wrong scope. Every expectation was measured against the implementation and then read back against the source, rather than predicted. Three measured behaviours are surprising and are pinned as current behaviour so changing them takes a failing test rather than an accident: - Swift enum/struct report kind "class"; tree-sitter-swift parses all three as class_declaration, so two _SWIFT_KIND_MAP entries never match - Kotlin `object O` emits no type entry, but `object Named : Base()` does — the supertype is what makes the grammar emit a type_identifier - Ruby `def self.x` is dropped; it parses as singleton_method Also fixes a mutation gate that could report a false clean. On a five-file run scripts/mutation_check.sh printed "none — 0 surviving mutants" while mutmut's own progress line ended at 423 survived and mutmut results returned a 915-line listing. It now cross-checks the two independent counts and refuses to render any verdict when they disagree, instead of taking the friendlier number. Its cleanup trap wipes mutants/ and .mutmut-cache, so a disagreement is uninvestigable afterwards and has to be caught in the run. ast_extractors.py changes are comment-only: equivalence notes at the use site for the six remaining mutants. The decorated_definition arm of _walk_for_calls is one of them, and is deliberately kept — it mirrors _extract_python_children's dispatch, where the same branch is load-bearing (that function has no catch-all), and it is the seam for behaviour that was never filled in: calls inside a decorator produce no edge at all. Filed as #372 with the design question attached. Verified: pytest -k "ast or extractor or codebase" -> 1181 passed differential vs merged main, 1344 files -> 0 mismatches ruff check / format --check -> clean pyright mcp_server/ -> 0 errors, 0 warnings scoped mutation, seven walkers -> 6 survivors, all documented Refs #369, #372 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Finishes what #373 left. Unpinned mutants across the five extractor modules: 639 -> 51, and mutants no test executes at all: 462 -> 0. Killed 1435 of 1486. #373 closed the seven scope-tracking walkers and named the rest as a tail. A named tail is still an unfinished PR, so this covers everything else in those modules: all eleven import extractors, C, C++, Go, Rust, Python and JS/TS definitions, and callee-basename resolution. Three new test modules, all asserting exact ordered output and all measured against the implementation before being asserted rather than predicted: test_ast_extractor_imports.py the eleven import extractors test_ast_extractor_definitions.py C/C++/Go/Rust/Python/JS + callee basename test_ast_extractor_edges.py fields, constants and character sets The third exists because shape assertions structurally cannot see three classes of drift, which mutation made visible: - ImportInfo.names was asserted nowhere, so the whole name-collection path was free to change silently; the resolver binds references through it - constants at their boundary: the [:120] signature cut and the len(base) < 100 callee cap. No ordinary fixture separates 120 from 121 - character sets in string cleanup: rstrip(";"), strip('"<>'), replace("using", "", 1). An ordinary module path cannot tell the real set from a wider one Hence deliberately awkward fixtures: paths ending in X, a callee named maXker, a name of exactly 100 characters, a[b[c]](), (a or b)(), and bytes that are not valid UTF-8 (Cortex indexes whatever is on disk, and a broken decode handler would raise LookupError out of the indexer on the first such file — valid input never consults it). Seven rough edges are now pinned as CURRENT behaviour with the cause named, so changing one takes a failing test: C emits a struct twice when a typedef names it; C++ drops `struct` while C keeps it; Rust drops functions inside `mod {}` and keeps `use a::{B,C}` / `as` clauses verbatim; Python drops `import os.path as p`; JS ignores require() and new; PHP group-use loses its prefix. Each is a plausible resolution defect, none is fixed here. The 51 survivors are classified, not tolerated, with the argument at the use site. Provably equivalent: rsplit(sep,n)[-1] and split(sep,n)[0] do not depend on n; None vs "" as an initial scope (both falsy, every use is a truthiness test); _declarator_name's two branches have identical bodies. Unreachable, verified from the callers: _extract_js_node's method arm (only reached from _extract_js_class, which always supplies a name), Go's params and receiver fallbacks, Python's name/params fallbacks. Kept with a filed follow-up: the decorated_definition arm (#372). Source changes are documentation only — proved by comparing the parsed AST of each file against HEAD with docstrings stripped: identical for all three. Verified: pytest -k "ast or extractor or codebase" -> 1265 passed ruff check / format --check -> clean pyright mcp_server/ -> 0 errors, 0 warnings scoped mutation -> 1435 killed, 0 uncovered, 51 equivalent Refs #369, #372 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Finishes what #373 left. Unpinned mutants across the five extractor modules: 639 -> 51, and mutants no test executes at all: 462 -> 0. Killed 1435 of 1486. #373 closed the seven scope-tracking walkers and named the rest as a tail. A named tail is still an unfinished PR, so this covers everything else in those modules: all eleven import extractors, C, C++, Go, Rust, Python and JS/TS definitions, and callee-basename resolution. Three new test modules, all asserting exact ordered output and all measured against the implementation before being asserted rather than predicted: test_ast_extractor_imports.py the eleven import extractors test_ast_extractor_definitions.py C/C++/Go/Rust/Python/JS + callee basename test_ast_extractor_edges.py fields, constants and character sets The third exists because shape assertions structurally cannot see three classes of drift, which mutation made visible: - ImportInfo.names was asserted nowhere, so the whole name-collection path was free to change silently; the resolver binds references through it - constants at their boundary: the [:120] signature cut and the len(base) < 100 callee cap. No ordinary fixture separates 120 from 121 - character sets in string cleanup: rstrip(";"), strip('"<>'), replace("using", "", 1). An ordinary module path cannot tell the real set from a wider one Hence deliberately awkward fixtures: paths ending in X, a callee named maXker, a name of exactly 100 characters, a[b[c]](), (a or b)(), and bytes that are not valid UTF-8 (Cortex indexes whatever is on disk, and a broken decode handler would raise LookupError out of the indexer on the first such file — valid input never consults it). Seven rough edges are now pinned as CURRENT behaviour with the cause named, so changing one takes a failing test: C emits a struct twice when a typedef names it; C++ drops `struct` while C keeps it; Rust drops functions inside `mod {}` and keeps `use a::{B,C}` / `as` clauses verbatim; Python drops `import os.path as p`; JS ignores require() and new; PHP group-use loses its prefix. Each is a plausible resolution defect, none is fixed here. The 51 survivors are classified, not tolerated, with the argument at the use site. Provably equivalent: rsplit(sep,n)[-1] and split(sep,n)[0] do not depend on n; None vs "" as an initial scope (both falsy, every use is a truthiness test); _declarator_name's two branches have identical bodies. Unreachable, verified from the callers: _extract_js_node's method arm (only reached from _extract_js_class, which always supplies a name), Go's params and receiver fallbacks, Python's name/params fallbacks. Kept with a filed follow-up: the decorated_definition arm (#372). Source changes are documentation only — proved by comparing the parsed AST of each file against HEAD with docstrings stripped: identical for all three. Verified: pytest -k "ast or extractor or codebase" -> 1265 passed ruff check / format --check -> clean pyright mcp_server/ -> 0 errors, 0 warnings scoped mutation -> 1435 killed, 0 uncovered, 51 equivalent Refs #369, #372 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

cdeust merged commit 90ea57a into main Aug 6, 2026
21 checks passed

cdeust deleted the test/369-pin-extractor-walkers branch August 6, 2026 15:04

Navigation Menu

test(ast): pin the language extractor walkers — 241 unpinned mutants → 6 documented equivalents (#369) - #373