feat(scripts): add check-encoding-safety.py (sibling of check-windows-footguns) - #66669
Closed
pnascimento9596 wants to merge 2 commits into
Closed
feat(scripts): add check-encoding-safety.py (sibling of check-windows-footguns)#66669pnascimento9596 wants to merge 2 commits into
pnascimento9596 wants to merge 2 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(scripts): add check-encoding-safety.py (sibling of check-windows-footguns)
Addresses #66668
What / why
The env-class cluster is not hypothetical: #66607 closed, #66609
merged to main via salvage #66825 (2026-07-18), authorship preserved
(51e1fb8) shows a UTF-8 BOM in
jobs.jsoncrashingload_jobs;three open fix PRs (#65124, #66475, #66483) cover the same
user-writable encoding/write class; accepted-class #57247 / #57249 is
the same
.envfamily Hermes users edit by hand. The salvage itselfwidened the fix to three additional readers (backup.py,
curator_backup.py, prompt_builder.py context scan). This checker still
flags residual prompt_builder sites on the hit list
(prompt_builder.py:1401 and prompt_builder.py:1858), demonstrating the
class is live and the checker's coverage is real. CONTRIBUTING already
documents Notepad/Windows BOM provenance and mandates
utf-8-sigforWindows-touched config.
This PR lands step 1 of a smaller-ask ladder: an opt-in, stdlib-only
AST checker (
scripts/check-encoding-safety.py) sibling ofscripts/check-windows-footguns.py. It flags reintroduction at reviewtime (R1 plain
utf-8on user-writable reads, R2 latin-1 fallback withoutBOM strip, R3
errors="replace"rewrite) without changing runtimebehavior or wiring CI/docs yet.
Known recall boundary: opaque path locals (e.g. the two readers #66825 widened to) are not bound to the allowlist - precision-first tradeoff, assignment tracing is a follow-on; details in #66668.
How to test
Footprint
Implementation footprint (vs upstream/main @ d59b79f): 16 files,
+1700/-0 - script (+1120), tests (+168), 14 fixtures. No CI,
pyproject, or docs in this PR.
Platforms tested
(22/22) + full-tree
--all(85/786) + C-proof runtime evidence in thelinked design issue.
scripts/check-windows-footguns.py scripts/check-encoding-safety.pyclean (exit 0); the lint script itselfhas no OS-specific code paths (AST/stdlib-only;
pathlib+ allowlistpath fragments). Not re-run on a native Windows host in this PR.
Non-goals
cron: UTF-8 BOM in jobs.json crashes load_jobs (RuntimeError) #66607 / fix(cron): accept UTF-8 BOM when reading jobs.json #66609; remaining C work is follow-on).
ladder steps, not day one.
Test plan (CI)
pytest tests/scripts/test_check_encoding_safety.py -q(22 passed)CI note (at push, run 29627150709): all required
checks pass ('All required checks pass' = SUCCESS;
ruff, Windows footguns, all 8 test slices green).
The single red mark is the non-required 'CI timing
report' job, which failed historically on fork PRs; passed since the 2026-07-19 run
(empty AUTOFIX_BOT_PAT in the workflow env - same
residual visible on #66609 and other open fork
PRs). Not specific to this change.