Skip to content

chore(headless): bump the pinned Codex CLI to 0.146.0 - #2057

Merged
Astro-Han merged 1 commit into
mainfrom
chore/headless-bump-codex-0.146.0
Aug 3, 2026
Merged

chore(headless): bump the pinned Codex CLI to 0.146.0#2057
Astro-Han merged 1 commit into
mainfrom
chore/headless-bump-codex-0.146.0

Conversation

@Astro-Han

Copy link
Copy Markdown
Contributor

Summary

The synchronized three-arm benchmark pins Claude Code at 2.1.220 — the version its vendor ships today — and Codex at 0.144.6, published 2026-07-18. By the time #1983 merged on 2026-08-03, 0.145.0 (07-21) and 0.146.0 (07-29) had both shipped. The pin was chosen in mid-July and never refreshed before merge.

Pinning is correct; the fingerprint is what makes a published run reproducible. The defect is that the two pins were taken at different times, so the version skew is a property of the comparison rather than of either agent. A reader of the report cannot tell whether a Codex result reflects Codex or reflects a 17-day-old build, and the arm that loses the coin flip is the competitor's.

Repin Codex to 0.146.0. engines.node is unchanged at >=16, so the pinned Node 22.23.1 runtime and every archivePath / stripComponents entry stay as they are; only the version, URL, archive integrity, and the six vendored digests move.

The composition fingerprint carries the competitor version string, so the Terminal-Bench Codex and DeepSWE Codex manifest fingerprints move with the pin while Terminal-Bench Kimi Code, Terminal-Bench OpenCode, and DeepSWE Kimi Code stay byte-identical. That split is the assertion in harness composition preserves historical manifest fingerprints — if a non-Codex fingerprint had moved, this bump would have leaked outside its blast radius.

Refs #1970.

Verification

  • npm run test -w @maka/headless — 1337 pass / 0 fail / 1 skip.
  • npm run lint, npm run format:check, git diff --check — clean.
  • archiveIntegrity cross-checked against the registry's own dist.integrity for 0.146.0-linux-x64; the same method reproduces the outgoing 0.144.6 value byte-for-byte from the registry, so the procedure is validated against a known-good pin rather than trusted blind.
  • End-to-end prepareCodexToolchain() against the live registry: downloaded the archive, verified all six vendored digests plus the pinned Node runtime, and materialized a toolchain whose codex-package.json reports 0.146.0.

Rollout

Any in-flight run whose manifest froze the 0.144.6 composition fingerprint cannot resume across this change — that is the fingerprint doing its job. The full 89-task three-arm run for #1970 has not started; it will be launched on a fresh run id after this merges.

The three-arm DeepSeek V4 Flash benchmark pins Claude Code at 2.1.220 —
the current latest — while Codex sat at 0.144.6, published 2026-07-18 and
two stable releases behind by the time #1983 merged. That asymmetry is a
property of the comparison, not of either agent: one competitor gets the
build its vendor ships today and the other gets a 17-day-old one.

Repin Codex to 0.146.0 (2026-07-29). The archive integrity matches the
registry's own dist.integrity for 0.146.0-linux-x64, and all six vendored
file digests were recomputed from that archive; `engines.node` is
unchanged at >=16, so the pinned Node 22.23.1 runtime stays.

The composition fingerprint carries the competitor version, so the two
Codex manifest fingerprints move with the pin and the Kimi Code and
OpenCode ones are byte-identical — that split is the assertion, not
collateral churn.
@Astro-Han
Astro-Han marked this pull request as ready for review August 3, 2026 20:24
@Astro-Han
Astro-Han merged commit a3fa270 into main Aug 3, 2026
10 checks passed
@Astro-Han
Astro-Han deleted the chore/headless-bump-codex-0.146.0 branch August 3, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant