Skip to content

fix(scripts): stop check-prerequisites text mode crashing on a legacy stdout code page - #3890

Merged
mnriem merged 3 commits into
github:mainfrom
jawwad-ali:fix/check-prereq-ascii-fallback
Aug 6, 2026
Merged

fix(scripts): stop check-prerequisites text mode crashing on a legacy stdout code page#3890
mnriem merged 3 commits into
github:mainfrom
jawwad-ali:fix/check-prereq-ascii-fallback

Conversation

@jawwad-ali

Copy link
Copy Markdown
Contributor

Problem

_check_file / _check_dir in scripts/python/check_prerequisites.py hard-code the U+2713 / U+2717 glyphs and print() them to sys.stdout:

marker = "✓" if path.is_file() else "✗"
print(f"  {marker} {description}")

On Windows sys.stdout falls back to the ANSI code page whenever stdout is not a console — a pipe or a file redirect, which is exactly how an agent or a workflow step invokes these scripts. U+2713 is unencodable in cp1252.

Reproduction on current main (81bf741)

stdout encoding: cp1252
UnicodeEncodeError: 'charmap' codec can't encode character '✓' in position 2

Text mode therefore aborts right after printing AVAILABLE_DOCS:, so the caller gets a header with no document lines under it — and a non-zero exit for a repository that is perfectly healthy.

Fix

Downgrade to ASCII only when stdout cannot encode the glyph, so a UTF-8 console is unaffected:

glyph = "✓" if ok else "✗"
try:
    glyph.encode(getattr(sys.stdout, "encoding", None) or "utf-8")
except (LookupError, UnicodeEncodeError):
    return "[OK]" if ok else "[FAIL]"
return glyph

[OK] / [FAIL] is not invented — it is the ASCII rendering these markers already have in this repo:

Site Code
scripts/powershell/common.ps1:243 Write-Output " [OK] $Description"
scripts/powershell/common.ps1:246 Write-Output " [FAIL] $Description"
tests/parity_helpers.py:130-131 text.replace(" ✓ ", " [OK] ").replace(" ✗ ", " [FAIL] ")

The PowerShell twin emits the ASCII form natively, and the parity helper maps the glyphs onto it — so the test suite already treats the two forms as equivalent output. Nothing downstream distinguishes them.

Breaking risk: a UTF-8-capable stdout still gets the glyphs, byte-identical to today. The only case that changes is one that previously raised and truncated the report. Callers parsing the marker are already required to accept both forms, per normalize_status_text.

Scope note

scripts/python/setup_tasks.py:58-65 carries a byte-identical block and crashes the same way. I kept this PR to one script — happy to extend it to the twin here or as a follow-up, whichever you prefer.

Verification

  • Fail-before / pass-after: the new test (running the real script with PYTHONIOENCODING=cp1252) fails on unpatched src and passes with the fix. File: 1 failed → 12 passed, 8 skipped.
  • Scoped regression: failure set identical to the clean-main baseline captured on 81bf741 (0 pre-existing in scope).
  • uvx ruff@0.15.0 check src tests → clean

Note this test file is also touched by my open #3785 (a comment-only change to scripts/python/common.py), so expect a trivial append conflict if that merges first — happy to rebase.


Written with assistance from Claude Code. Bug found, reproduced, and verified by me on current main.

… code page

_check_file/_check_dir hard-code U+2713/U+2717 and print() them to
sys.stdout. On Windows sys.stdout falls back to the ANSI code page whenever
stdout is not a console — which is every time an agent or a workflow step
captures the output — and U+2713 is unencodable in cp1252:

  stdout encoding: cp1252
  UnicodeEncodeError: 'charmap' codec can't encode character '✓'

So text mode aborted right after printing "AVAILABLE_DOCS:", losing every
per-document line.

Fall back to ASCII when stdout cannot encode the glyph. "[OK]"/"[FAIL]" is
the rendering these markers already have in-tree: Test-FileExists in
scripts/powershell/common.ps1 emits exactly those, and
normalize_status_text in tests/parity_helpers.py maps the glyphs onto them,
so the twins already treat the two forms as equivalent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents prerequisite text output from crashing on legacy stdout encodings.

Changes:

  • Adds encoding-aware ASCII status-marker fallbacks.
  • Adds a cp1252 subprocess regression test.
Show a summary per file
File Description
scripts/python/check_prerequisites.py Selects encodable status markers.
tests/test_check_prerequisites_python_parity.py Tests legacy code-page output.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread tests/test_check_prerequisites_python_parity.py Outdated

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address Copilot feedback

jawwad-ali and others added 2 commits August 6, 2026 10:35
Review catch: the fixture left every reported document absent (the empty
contracts/ also reports missing), so the test only ever called
_status_marker(False). The assertion was `"[OK]" in out or "[FAIL]" in out`,
which "[FAIL]" alone satisfied.

Proved the hole by mutation: replacing the fallback body with a bare
`return "[FAIL]"` — deleting the success branch outright — left the test
GREEN.

Add research.md so one document is present, and assert both markers
explicitly. The strengthened test now kills all three mutations:

  fallback always "[FAIL]"  -> FAILS (was passing)
  fallback always "[OK]"    -> FAILS
  no fallback at all        -> FAILS (the original bug)
  unmutated                 -> 12 passed, 8 skipped

Missing documents are still present in the fixture, so the failure path
stays covered too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit on this branch unintentionally reverted the source fix
while adding the strengthened test, so the branch carried the test without
the implementation it tests.

Cause: my local verification script reverted the file for its red run with
`git checkout upstream/main -- <file>`, which writes the INDEX as well as
the working tree. Restoring the working-tree copy afterwards left main's
version staged, and the next commit captured it.

Restores the fix from 275663b. Verified: 12 passed / 8 skipped, and the red
run (source reverted) produces 1 new-vs-baseline failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mnriem
mnriem requested a balanced review from Copilot August 6, 2026 13:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@mnriem
mnriem self-requested a review August 6, 2026 13:17
@mnriem
mnriem merged commit 3dff6f1 into github:main Aug 6, 2026
14 checks passed
@mnriem

mnriem commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants