Skip to content

fix(tui_gateway): set utf-8 encoding for subprocess pipes on Windows - #67922

Closed
escalate007 wants to merge 1 commit into
NousResearch:mainfrom
escalate007:fix/windows-gbk-encoding
Closed

fix(tui_gateway): set utf-8 encoding for subprocess pipes on Windows#67922
escalate007 wants to merge 1 commit into
NousResearch:mainfrom
escalate007:fix/windows-gbk-encoding

Conversation

@escalate007

Copy link
Copy Markdown

Problem

On Chinese Windows systems (GBK system encoding), subprocess.Popen with text=True defaults to the system locale for pipe decoding. When the Hermes backend writes UTF-8 bytes to stderr, the _drain_stderr thread crashes with UnicodeDecodeError: 'gbk' codec can't decode byte. This makes the desktop/TUI backend unresponsive — the user clicks send after uploading a file and nothing happens.

Root cause

tui_gateway/server.py line 339: text=True without an explicit encoding=. Python uses the locale encoding (GBK on zh-CN Windows), but the backend process outputs UTF-8.

Fix

Add encoding='utf-8' to the Popen call, after text=True. This is a one-line change, safe on all platforms — Python ignores unknown encoding parameters on non-Windows systems where UTF-8 is already the default.

Related logs

[gateway-crash] thread Thread-14 (_drain_stderr) raised UnicodeDecodeError:
'gbk' codec can't decode byte 0xa0 in position 7: illegal multibyte sequence

And also Python's internal _readerthread in subprocess.py: UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 140: illegal multibyte sequence

On Chinese Windows systems, subprocess.Popen with text=True defaults to
GBK encoding for pipe reads. When the Hermes backend writes UTF-8 bytes
to stderr, the _drain_stderr thread crashes with UnicodeDecodeError,
making the desktop UI unresponsive after file uploads.

Fix by adding encoding='utf-8' to the Popen call.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/tui Terminal UI (ui-tui/ + tui_gateway/) platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows needs-decision Awaiting maintainer decision before any implementation labels Jul 20, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to open #52700 and report #52649. This patch fixes the slash-worker Popen only; #52700 also covers git subprocesses and uses errors="replace", so the narrower scope needs a maintainer consolidation choice.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded by PR #70850 (merged), which covers this Popen site plus every other text=True capture in the desktop backend. Note the merged version also passes errors="replace" alongside encoding="utf-8" — without it, a child emitting genuinely invalid UTF-8 would still raise in the drain thread. Credit to @Sahil-SS9's earlier #61978 for the server.py hardening that was cherry-picked as the base; thanks for the report and fix regardless.

@teknium1 teknium1 closed this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants