Skip to content

fix(browser): prevent UnicodeDecodeError on Windows with non-UTF-8 subprocess output - #50182

Closed
allin2 wants to merge 1 commit into
NousResearch:mainfrom
allin2:fix/browser-tool-unicode-error
Closed

fix(browser): prevent UnicodeDecodeError on Windows with non-UTF-8 subprocess output#50182
allin2 wants to merge 1 commit into
NousResearch:mainfrom
allin2:fix/browser-tool-unicode-error

Conversation

@allin2

@allin2 allin2 commented Jun 21, 2026

Copy link
Copy Markdown

Description

Fix UnicodeDecodeError in browser_tool.py when running on Windows with Chinese locale (GBK/CP936).

Problem

When agent-browser.cmd fails on Windows with a Chinese locale, the error message from cmd.exe is encoded in the system default charset (GBK), not UTF-8. Reading stderr with strict utf-8 codec raises:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb2 in position 9: invalid start byte

This masks the original error (e.g., node not found) and makes all browser tools completely unusable on Windows Chinese systems.

Root Cause

In _run_browser_command() (lines 2113-2116), stdout and stderr temp files are read with encoding="utf-8" without error handling:

with open(stdout_path, "r", encoding="utf-8") as f:
    stdout = f.read()
with open(stderr_path, "r", encoding="utf-8") as f:
    stderr = f.read()

When agent-browser.cmd fails (e.g., node not found), cmd.exe outputs the error in GBK encoding. Byte 0xB2 at position 9 is not a valid UTF-8 start byte, triggering UnicodeDecodeError.

Fix

Add errors="replace" to the open() calls:

with open(stdout_path, "r", encoding="utf-8", errors="replace") as f:
    stdout = f.read()
with open(stderr_path, "r", encoding="utf-8", errors="replace") as f:
    stderr = f.read()

This replaces non-UTF-8 bytes with the Unicode replacement character (U+FFFD) instead of crashing, so the actual subprocess error message is visible to users and agents.

Impact

Scenario Before After
Normal UTF-8 output Works Works (unchanged)
GBK error on Windows UnicodeDecodeError Error message visible
Browser tools on Windows Chinese All broken All work

Other open(encoding="utf-8") calls in this file

  • Line 927: Already wrapped in try/except Exception - safe
  • Line 1304: open(..., "w") - write-only, no decoding risk
  • Line 2050: Inside try/except OSError - safe
  • Line 3665: Reads /proc/1/cgroup (Linux-only, always UTF-8) - safe

…bprocess output

When agent-browser.cmd fails on Windows with a Chinese locale (GBK/CP936),
the error message from cmd.exe is encoded in the system default charset,
not UTF-8. Reading stderr with strict 'utf-8' codec raises:

    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb2 in position 9

This masks the original error (e.g. 'node' not found) and makes all
browser_* tools unusable.

Fix by adding errors='replace' to the open() calls that read stdout/stderr
temp files. This replaces non-UTF-8 bytes with the U+FFFD replacement
character instead of crashing, so the actual subprocess error is visible
to users and agents.

Other open(encoding='utf-8') calls in this file are either already inside
try/except blocks (line 927) or read files guaranteed to be UTF-8
(/proc/1/cgroup, config files).

Closes: #<issue-number>
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/browser Browser automation (CDP, Playwright) P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jun 21, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #47516 — same fix (adding errors="replace" to the open(stdout_path/stderr_path, "r", encoding="utf-8") reads in tools/browser_tool.py) for the same bug (#47456, GBK/non-UTF-8 subprocess output on Windows). #47516 is the earlier open PR with the identical mechanism at the identical edit site.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for identifying the Windows locale failure. The strict UTF-8 reads are still present on current main at tools/browser_tool.py:2477 and tools/browser_tool.py:2479, so the underlying bug remains real.

Problems

  • The two-site change is incomplete on current main: _read_command_output_files() still reads the same temp files strictly at tools/browser_tool.py:322 (used by the timeout path at :2461), and Chrome fallback still does so at :1120.
  • This branch is a whole-file rewrite (+3891/-3891) and GitHub reports it DIRTY; current main has substantially moved since its base.
  • No regression test is included; tests/tools/test_browser_open_timeout.py:81 currently exercises only valid UTF-8 output.

Suggested changes

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Closing as resolved by PR #70875 (merged, commit 0f732cb), which closed out this bug class codebase-wide: every text=True subprocess call now passes encoding="utf-8", errors="replace", and a CI linter rule (scripts/check-windows-footguns.py) rejects any future unguarded site. The sites this PR targeted are all guarded on current main — verified per-file. Credit for the class fix goes to @Stoltemberg (#55339, the original sweep) and @jinglun010-cpu (#60741 + the #60751 linter); thanks for your fix as well — the volume of independent PRs on this bug is what escalated it to a class-wide close-out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows tool/browser Browser automation (CDP, Playwright) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants