chore(headless): bump the pinned Codex CLI to 0.146.0 - #2057
Merged
Conversation
The three-arm DeepSeek V4 Flash benchmark pins Claude Code at 2.1.220 — the current latest — while Codex sat at 0.144.6, published 2026-07-18 and two stable releases behind by the time #1983 merged. That asymmetry is a property of the comparison, not of either agent: one competitor gets the build its vendor ships today and the other gets a 17-day-old one. Repin Codex to 0.146.0 (2026-07-29). The archive integrity matches the registry's own dist.integrity for 0.146.0-linux-x64, and all six vendored file digests were recomputed from that archive; `engines.node` is unchanged at >=16, so the pinned Node 22.23.1 runtime stays. The composition fingerprint carries the competitor version, so the two Codex manifest fingerprints move with the pin and the Kimi Code and OpenCode ones are byte-identical — that split is the assertion, not collateral churn.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The synchronized three-arm benchmark pins Claude Code at
2.1.220— the version its vendor ships today — and Codex at0.144.6, published 2026-07-18. By the time #1983 merged on 2026-08-03,0.145.0(07-21) and0.146.0(07-29) had both shipped. The pin was chosen in mid-July and never refreshed before merge.Pinning is correct; the fingerprint is what makes a published run reproducible. The defect is that the two pins were taken at different times, so the version skew is a property of the comparison rather than of either agent. A reader of the report cannot tell whether a Codex result reflects Codex or reflects a 17-day-old build, and the arm that loses the coin flip is the competitor's.
Repin Codex to
0.146.0.engines.nodeis unchanged at>=16, so the pinned Node 22.23.1 runtime and everyarchivePath/stripComponentsentry stay as they are; only the version, URL, archive integrity, and the six vendored digests move.The composition fingerprint carries the competitor version string, so the
Terminal-Bench CodexandDeepSWE Codexmanifest fingerprints move with the pin whileTerminal-Bench Kimi Code,Terminal-Bench OpenCode, andDeepSWE Kimi Codestay byte-identical. That split is the assertion inharness composition preserves historical manifest fingerprints— if a non-Codex fingerprint had moved, this bump would have leaked outside its blast radius.Refs #1970.
Verification
npm run test -w @maka/headless— 1337 pass / 0 fail / 1 skip.npm run lint,npm run format:check,git diff --check— clean.archiveIntegritycross-checked against the registry's owndist.integrityfor0.146.0-linux-x64; the same method reproduces the outgoing0.144.6value byte-for-byte from the registry, so the procedure is validated against a known-good pin rather than trusted blind.prepareCodexToolchain()against the live registry: downloaded the archive, verified all six vendored digests plus the pinned Node runtime, and materialized a toolchain whosecodex-package.jsonreports0.146.0.Rollout
Any in-flight run whose manifest froze the
0.144.6composition fingerprint cannot resume across this change — that is the fingerprint doing its job. The full 89-task three-arm run for #1970 has not started; it will be launched on a fresh run id after this merges.