Skip to content

fix(launcher-deps): defer dist-info prune until whole tmp_dir commits (issue #149) - #150

Merged
cdeust merged 1 commit into
mainfrom
fix/issue-149
Jul 15, 2026
Merged

fix(launcher-deps): defer dist-info prune until whole tmp_dir commits (issue #149)#150
cdeust merged 1 commit into
mainfrom
fix/issue-149

Conversation

@cdeust

@cdeust cdeust commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the Python-3.10-only flake in test_pip_install_rollback_on_mid_commit_failure (#149).

Mechanism (confirmed by deterministic forced-ordering reproducer, not just reruns):
pip_install's per-entry commit loop in scripts/launcher_deps_install.py called
_fs.prune_superseded_dist_info(deps_dir, entry) immediately after EACH individual
entry committed, inside the loop. os.listdir() order is unspecified by the Python
stdlib and differs by OS/filesystem/container image (this explains "py3.10-only in
CI, always green in 3.11-3.13" — the runners differ in directory-enumeration order,
not in pip/importlib behavior). When the *.dist-info entry happened to be
enumerated before the package-directory entry, the prune step permanently deleted
the OLD (still-valid) dist-info right after the new one committed — then the
package-directory entry's own commit failed (simulated locked .pyd on Windows;
the mid-commit-failure scenario the test exercises) and rolled back to the
ORIGINAL package files, leaving deps_dir in a genuinely inconsistent state:
package files reverted to the old version, but the old version's metadata gone.
This was a real defect in the rollback logic, not a test artifact — it just needed
the right os.listdir ordering to surface, which is why it was flaky rather than
always-failing.

Fix: collect committed *.dist-info entry names during the loop; only run the
destructive prune once the whole tmp_dir has committed successfully. The one
irreversible step now waits for confirmation of overall success regardless of
os.listdir enumeration order — closing the race by construction, not by luck of
ordering.

Also: split pip_install's commit loop into _commit_resolved_entries +
_entry_already_satisfied (this is the exact concern that carried the bug), and
extracted _remove_path to fix two pre-existing NESTING_TOO_DEEP violations in
commit_entry that the pre-commit craftsmanship hook flagged once this file was
touched. pip_install itself remains over the 50-line function budget
(pre-existing debt, mostly PEP-668-retry + subprocess-invocation logic unrelated
to #149) — documented as an out-of-scope §10 rationale rather than expanded into
this fix's blast radius.

Verification

  • Deterministic forced-os.listdir-order reproducer: fails against pre-fix code,
    passes post-fix.
  • New permanent regression test
    test_pip_install_rollback_preserves_dist_info_regardless_of_commit_order
    monkeypatches os.listdir to force the dangerous ordering — confirmed it FAILS
    against the pre-fix code (mutant-kill) and passes post-fix.
  • 60x loop of both the original flaky test and the new regression test under
    Python 3.10.18 (provisioned locally via uv venv --python 3.10, since this
    machine doesn't have 3.10 on PATH by default): 0/120 failures.
  • Full tests_py/scripts/test_launcher_deps.py: 32/32 green under 3.10.
  • Full suite (uv run pytest, Python 3.13.7, this repo's normal env): 5179
    passed
    , 2 pre-existing failures in tests_py/hooks/test_hook_receipts.py
    (unrelated — reproduced in isolation, independent of this change; not touched
    by this diff).
  • ruff check + ruff format --check: clean.

Fixes #149

🤖 Generated with Claude Code

Root cause of issue #149's Python-3.10-only flake: pip_install's
per-entry commit loop pruned superseded *.dist-info siblings
immediately after each individual entry committed. os.listdir() order
is unspecified by the stdlib and differs by OS/filesystem, so when the
dist-info entry happened to enumerate before the package-directory
entry, the prune permanently deleted the still-valid OLD dist-info
right before the package-directory commit failed and rolled back --
leaving deps_dir with reverted package files but no metadata for
either version. Confirmed with a forced os.listdir-order reproducer
(fails pre-fix, passes post-fix) and a new permanent regression test
that pins the dangerous ordering deterministically.

Fix: collect committed *.dist-info names during the loop and only run
the destructive prune once the whole tmp_dir has committed
successfully, so the one irreversible step waits for confirmation of
overall success regardless of listdir enumeration order.

Also split pip_install's commit loop into _commit_resolved_entries and
_entry_already_satisfied (Move 5 -- this is the concern that carried
the bug), and extracted _remove_path to fix two pre-existing
nesting-depth violations in commit_entry surfaced by the pre-commit
craftsmanship gate. pip_install itself remains over the 50-line budget
(pre-existing, mostly PEP-668-retry + subprocess-invocation, unrelated
to #149); documented as an out-of-scope §10 rationale rather than
expanded into this fix's blast radius.

Verified: 60x loop of both the original flaky test and the new
regression test under Python 3.10.18, 0/120 failures. Full suite:
5179 passed, 2 pre-existing unrelated failures
(tests_py/hooks/test_hook_receipts.py, reproduced in isolation
independent of this change). ruff check + ruff format --check clean.

Fixes #149

Co-Authored-By: Claude <noreply@anthropic.com>
@cdeust
cdeust merged commit 5226bec into main Jul 15, 2026
14 checks passed
@cdeust
cdeust deleted the fix/issue-149 branch July 15, 2026 14:07
cdeust added a commit that referenced this pull request Jul 17, 2026
…ps bump (#154)

* chore(release): prepare v4.14.3 (fix write-gate, launcher-deps; deps bump; refactor split)

Bump version 4.14.2 -> 4.14.3 across pyproject.toml, .claude-plugin/plugin.json,
.claude-plugin/marketplace.json (SemVer patch: 4 fix/chore/refactor commits since
v4.14.2, no new features -- #147/#148, #149/#150, #152, #134/#153).

CHANGELOG.md: new [4.14.3] entry per Keep a Changelog, dated 2026-07-17, with a
PENDING placeholder in the Verified section for the mandatory pre-tag
benchmarks/reproduce.sh non-regression evidence (user mandate 2026-07-10) --
this commit prepares the release, it does not run the bench or cut the tag.

marketplace.json version_note for the cortex plugin entry updated to match,
same PENDING placeholder. zetetic-team-subagents cross-reference (2.28.1) left
unchanged: no open release PR found on cdeust/zetetic-team-subagents and its
latest tag (v2.28.1) already matches the pinned version.

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(release): commit v4.14.3 pre-tag bench evidence and adjudication

Pre-tag guard on the exact release tree 4e3a202: LongMemEval MRR
0.9166/R@10 0.9820 PASS; LoCoMo 3-run mean MRR 0.7998/R@10 0.9135 —
MRR mean sits 0.0002 below the floor-tolerance threshold and was
adjudicated as sampling noise by explicit maintainer decision
(delta vs 4.14.2 mean −0.0011 < SEM ~0.0013; rep1 0.8013 vs 0.8015;
reranker loaded in all MANIFESTs; no released commit in the LoCoMo
harness dependency graph — ingestion via BenchmarkDB, not the #148
write path). BEAM 0.5417, inside the documented noise band.
Evidence: benchmarks/results/repro/20260717-v4.14.3-pretag/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BxpvWMSYo2FLCw4c1xw7iJ

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: test_pip_install_rollback_on_mid_commit_failure fails intermittently on Python 3.10 only

1 participant