fix(doctor): pin utf-8/replace decoding for captured subprocesses - #49512
fix(doctor): pin utf-8/replace decoding for captured subprocesses#49512Bartok9 wants to merge 2 commits into
Conversation
Closes NousResearch#49499 Root cause: subprocess.run(..., text=True) decodes child output with the system locale codec. On non-UTF-8 Windows locales (Chinese CP936/GBK), a child that emits UTF-8 bytes triggers UnicodeDecodeError inside subprocess's reader thread, crashing 'hermes doctor' mid-run (the reported 'gbk codec can't decode byte 0xaa' traceback). Fix: add _run_text_subprocess() that pins encoding='utf-8' and errors='replace' (caller-overridable) and route the two text-capturing doctor probes (SSH connectivity, npm audit) through it. Undecodable bytes now degrade to the replacement character instead of raising, so doctor keeps running and prints a readable diagnostic. Tests: TestRunTextSubprocess covers the utf-8/replace default, text kwarg normalization, caller override, and an end-to-end run with bytes (0xaa 0xff) that raise UnicodeDecodeError under GBK but decode cleanly through the helper. 70 tests pass in tests/hermes_cli/test_doctor.py.
|
Duplicate of #49510 — both fix #49499 by pinning encoding="utf-8", errors="replace" on the same two text-mode subprocess sites (SSH probe + npm audit) in hermes_cli/doctor.py. #49510 was opened earlier (the canonical fix); this PR factors the same logic into a _run_text_subprocess() helper, but the behavior change is identical. |
The _run_text_subprocess wrapper passed timeout via **kwargs, which the static subprocess-timeout guardrail (tests/hermes_cli/test_subprocess_timeouts.py) cannot see, so CI failed on doctor.py:47. Make timeout a keyword-only param forwarded literally to subprocess.run so the AST check passes; callers already supply it.
|
Thanks @alt-glitch — you're right, this duplicates #49510 (opened earlier by @briandevans), which is the canonical fix for #49499. Same two text-mode subprocess sites, same @briandevans if it's useful, the only delta here was factoring the two call sites into a shared |
Summary
hermes doctorcrashed withUnicodeDecodeErroron non-UTF-8 Windows locales (CP936/GBK).Motivation
Closes #49499.
On Chinese Simplified Windows (default locale CP936/GBK),
hermes doctorcrashed during the connectivity/diagnostic checks:subprocess.run(..., text=True)decodes the child's output with the system locale codec. When a captured child emits UTF-8 bytes that are invalid under GBK, the decode raises inside subprocess's reader thread and takes down the command that's supposed to be the first troubleshooting step.Fix
Add a small helper
_run_text_subprocess()that pinsencoding="utf-8"anderrors="replace"(both caller-overridable), and route the two text-capturing doctor probes — the SSH connectivity check and thenpm auditcheck — through it. Undecodable bytes now degrade to the replacement character instead of crashing, exactly as the issue's "Suggested Fix" requested.The two byte-only captures (
docker info,gh auth status) don't decode text and are left unchanged.Verification
python3 -m pytest tests/hermes_cli/test_doctor.py— 70 passedReal behavior proof
0xaa 0xff) that are illegal under GBK._run_text_subprocess(...):tests/hermes_cli/test_doctor.py::TestRunTextSubprocess::test_survives_bytes_invalid_under_gbkspawns a child that writesb'ok\xaa\xff tail'and asserts the helper returns cleanly (returncode 0, output preserved) — it would raiseUnicodeDecodeErrorunder a GBK locale with the oldtext=Truepath.