Skip to content

fix(windows): utf-8 subprocess decode across the desktop backend — salvage #61978 + widen to sibling sites - #70850

Merged
teknium1 merged 3 commits into
mainfrom
hermes/hermes-dc3471e3
Jul 24, 2026
Merged

fix(windows): utf-8 subprocess decode across the desktop backend — salvage #61978 + widen to sibling sites#70850
teknium1 merged 3 commits into
mainfrom
hermes/hermes-dc3471e3

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Windows desktop backend no longer crashes with UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in subprocess._readerthread — every text-mode subprocess capture in the desktop-backend process now decodes as UTF-8 with lossy fallback.

Root cause: on Windows, subprocess with text=True and no explicit encoding= decodes child output with the ANSI code page (cp1252/GBK/…). cp1252 defines 251 of 256 bytes, so the bug stayed latent for months — until commit 84db32484f put a 🐛 emoji (UTF-8 f0 9f 90 9b) in a commit subject on main. Byte 0x90 is one of only five bytes undefined in cp1252, and the desktop update panel's git log HEAD..origin/main (_recent_upstream_commits) started pulling it through the stdlib reader thread on every Windows install behind that commit, crashing the backend (#52649, #53137, #63686, #61595, #58385).

Salvages PR #61978 by @Sahil-SS9 (server.py hardening + regression tests, cherry-picked with authorship preserved) and widens the fix to the sibling sites the crash actually came through.

Changes

  • tui_gateway/server.py (salvaged from fix: harden tui_gateway subprocess reads against Windows locale UnicodeDecodeError (#53137) #61978): encoding="utf-8", errors="replace" on the slash-worker Popen, pdftoppm, cli.exec, quick-command exec, and shell.exec — plus regression tests pinning the kwargs
  • hermes_cli/web_server.py: git-log update panel (the site that fired), memory-provider setup runner, WhatsApp bridge npm install, docker probe
  • hermes_cli/banner.py: all 5 git sites (startup update check runs in this process)
  • tui_gateway/host_supervisor.py: build-SHA probe, ps probe, compute-host Popen drain threads
  • tui_gateway/compute_host.py: build-SHA probe, ps rss probe

errors="replace" is the load-bearing half: even if a child ever emits genuinely non-UTF-8 bytes, decoding degrades to U+FFFD instead of raising in a daemon thread.

Validation

Check Result
Legacy path repro: cp1252 decode of the real trigger line raises 'charmap' ... byte 0x90 as reported
E2E: real git repo with 🐛 commit through _recent_upstream_commits + banner._git_stdout decodes correctly, kwargs verified
Invalid byte 0x90 through the new path degrades to U+FFFD, no exception
tests/tui_gateway/ + banner tests 457 passed, 0 failed

Closes #61978 (salvaged with credit). Fixes #52649, #53137 (and duplicates #63686, #61595, #58385).

Infographic

utf8-subprocess-decode-guard

Sahil-SS9 and others added 3 commits July 24, 2026 09:23
…backend sites

The salvaged #61978 covers tui_gateway/server.py. The crash reported on
Jul 24 came from a sibling site it doesn't touch: the desktop update
panel's _recent_upstream_commits() in hermes_cli/web_server.py runs
git log with text=True and no encoding. Commit 84db324 put a bug
emoji (UTF-8 f0 9f 90 9b) in a subject on main; byte 0x90 is undefined
in cp1252, so every Windows desktop install behind that commit crashed
in subprocess._readerthread during the update check (#52649).

Guard every text=True capture site in the desktop-backend process with
encoding='utf-8', errors='replace':
- hermes_cli/web_server.py: git log update panel, memory-provider setup
  runner, WhatsApp bridge npm install, docker probe
- hermes_cli/banner.py: all 5 git sites (update check runs at startup)
- tui_gateway/host_supervisor.py: build-sha probe, ps probe, compute-host
  Popen drain threads
- tui_gateway/compute_host.py: build-sha probe, ps rss probe
@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on a618210

all good!

@teknium1
teknium1 merged commit ead23aa into main Jul 24, 2026
37 checks passed
@teknium1
teknium1 deleted the hermes/hermes-dc3471e3 branch July 24, 2026 16:50
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/cli CLI entry point, hermes_cli/, setup wizard platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows labels Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows: Gateway crashes with UnicodeDecodeError (cp950) in _readerthread — causes zombie cycle every ~580s

3 participants