browser_exec: pin the browser-use CLI pipes to UTF-8

[pucedoteth]

Fixes #87152

Problem

browser_exec in tools/browser_use_cli.py runs the CLI with text=True and no encoding:

proc = subprocess.run(
    cmd,
    input=code,
    capture_output=True,
    text=True,
    timeout=timeout,
    env=env,
    **popen_extra,
)

text=True on its own decodes the child's stdout/stderr with locale.getencoding(). On Windows that is the ANSI code page — cp1252 on US/Western installs, cp932/cp936/cp949 on CJK ones — never UTF-8.

The child, meanwhile, is emitting UTF-8: hermes_bootstrap.py sets PYTHONUTF8=1 and PYTHONIOENCODING=utf-8 in os.environ on Windows, and the browser-use CLI inherits them. So the two ends of the pipe disagree by construction.

What that produces, verified locally:

sample = "Página — Ferramenta ✓".encode("utf-8")
  cp1252   -> decoded (mojibake, no crash)
  cp932    -> UnicodeDecodeError: illegal multibyte sequence at byte 10
  cp949    -> UnicodeDecodeError: illegal multibyte sequence at byte 8
  cp936    -> UnicodeDecodeError: illegal multibyte sequence at byte 10
  cp1252 b'\x81' -> UnicodeDecodeError (character maps to <undefined>)
  cp1252 b'\x8d' -> UnicodeDecodeError (character maps to <undefined>)
  cp1252 b'\x8f' -> UnicodeDecodeError (character maps to <undefined>)
  cp1252 b'\x90' -> UnicodeDecodeError (character maps to <undefined>)
  cp1252 b'\x9d' -> UnicodeDecodeError (character maps to <undefined>)

So a page title, URL, or log line with a non-ASCII character either comes back to the model as mojibake, or blows up. And the blow-up is not contained — the call site only handles subprocess.TimeoutExpired and OSError:

UnicodeDecodeError is a ValueError: True
UnicodeDecodeError is an OSError  : False

UnicodeDecodeError is a ValueError, so it escapes browser_exec entirely rather than being turned into a tool_error(...) the agent can recover from.

Fix

Pin both ends to UTF-8:

encoding="utf-8",
errors="replace",

This is exactly what the uv tool install browser-use call at the top of this same file already does (line ~353), so it brings the two subprocess.run calls in the module into agreement.

errors="replace" also means the decode can no longer raise at all, so the surrounding exception handling does not need widening.

It fixes the input direction too: input=code is encoded with the same codec, so a script containing a non-ASCII selector or string no longer risks UnicodeEncodeError on cp1252.

Tests

Two tests added to tests/tools/test_browser_use_cli.py::TestBrowserExec:

• test_pipes_are_pinned_to_utf8 — asserts the kwargs actually handed to subprocess.run. Deterministic on every platform. On main it fails with assert None == 'utf-8'.
• test_non_ascii_output_round_trips — a fake CLI emitting Página → café must survive the pipe. This one only bites on Windows CI, since POSIX locales are already UTF-8.

I could only install a subset of the project deps locally, so 8 tests in this file fail in my environment. That set is identical before and after the change (baseline 8 failed, 84 passed → with this PR 8 failed, 86 passed, the two extra passes being the new tests), so full CI is the real check.

Not addressed here

The same text=True-without-encoding pattern appears at 7 other call sites, which I left alone to keep this focused on the filed issue:

tools/code_execution_tool.py:1894
tools/computer_use/cua_backend.py:453
tools/computer_use/cua_backend.py:477
tools/self_repo_guard.py:569
tools/transcription_tools.py:1610
tools/transcription_tools.py:2030
tools/transcription_tools.py:2040

Happy to fold them in here or open a follow-up, whichever you prefer.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #87162. Both PRs pin browser_exec text-pipe decoding to UTF-8 with replacement handling and add equivalent non-ASCII regression coverage.

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

browser_exec: pin the browser-use CLI pipes to UTF-8

1. tools/browser_use_cli.py:637-645 — errors="replace" converts undecodable bytes to U+FFFD instead of surfacing an error. That fixes the UnicodeDecodeError-escapes-OSError-handler problem, but a genuine CLI encoding regression would now come back as silent mojibake in page titles/log lines. The comment documents the choice well; consider logging a warning when replacement actually occurs so the lossy path is observable.

2. Pinning encoding="utf-8" relies on the CLI inheriting PYTHONIOENCODING/PYTHONUTF8 from hermes_bootstrap. If the CLI is ever spawned from a path that doesn't set those (user's own install, non-managed binary), output would be mis-decoded again. Deriving the codec from the same env the CLI uses (or asserting PYTHONUTF8 is set before spawning) would make the coupling explicit rather than implied.

3. Test coverage is good — the kwargs-pinning assertion plus a real non-ASCII round-trip (Página → café) exercises both the plumbing and the actual behavior.

[pucedoteth]

Closing as a duplicate of #87162, which landed first and carries the same fix (encoding="utf-8", errors="replace" on the subprocess.run call in browser_exec) plus an extra assertion that text stays True. Deferring to that one — no reason for both to sit in the queue.

@Enough1122 thanks for the review; the substantive points apply equally to #87162, so I'll leave them there rather than split the discussion.