Skip to content

fix(windows): base64 clipboard reads + UTF-8 subprocess decodes (PowerShell codepage class) - #81963

Merged
teknium1 merged 4 commits into
mainfrom
fix/powershell-codepage-utf8
Aug 8, 2026
Merged

fix(windows): base64 clipboard reads + UTF-8 subprocess decodes (PowerShell codepage class)#81963
teknium1 merged 4 commits into
mainfrom
fix/powershell-codepage-utf8

Conversation

@teknium1

@teknium1 teknium1 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

PowerShell codepage cluster: clipboard reads mangled emoji/CJK to ?? because Get-Clipboard output crossed the console codepage; several Windows subprocess captures decoded with locale defaults and could crash.

Changes

Validation

Result
clipboard vitest (CJK/emoji) green
working_diff + goal gates pytest green
#57434 already fixed on main (c89481d)

Infographic

clipboard-quest

annguyenNous and others added 4 commits August 8, 2026 11:48
…nt ANSI codepage corruption

PowerShell's Get-Clipboard -Raw outputs text in the system's ANSI codepage
(e.g. CP1252, CP936), not UTF-8. When Node.js reads this with encoding: 'utf8',
non-ASCII characters (CJK, emoji, accented chars) are corrupted.

The write path already solved this by base64-encoding UTF-8 bytes and passing
them via -Command argument (see comment at line 94-98). This fix applies the
same approach to the read path:

- Change PowerShell read command to base64-encode the clipboard content
  using [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(...))
- Add base64 flag to read command type for PowerShell entries
- Decode base64 result in readClipboardText when flag is set

Also adds child.unref() to spawned clipboard child processes in the write
path to prevent delaying process.exit() when the app exits mid-clipboard-write.
_run() used text=True without an encoding, so Windows decoded git's
UTF-8 output with the locale code page (cp932) and raised
UnicodeDecodeError on non-ASCII filenames or diff content, breaking
the "Never raises on git failure" contract in its docstring. Match
the utf-8 + errors="replace" policy checkpoint_manager's _run_git
already uses. Legacy cp932-encoded blob content degrades to
replacement characters instead of crashing; a test pins that
trade-off so it stays a documented choice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s codepage

A gate runs whatever command the operator configured, so its output is
arbitrary bytes. run_gate captured it with text=True and no encoding, which
decodes with locale.getpreferredencoding() under errors="strict".

One byte the decoder rejects — a test runner's checkmarks or CJK on a
non-UTF-8 Windows console, a stray binary byte anywhere in the stream — kills
subprocess's reader thread. proc.stdout comes back None, the `or ""` fallback
turns that into an empty tail, and an unhandled traceback is dumped to stderr.
The gate's pass/fail verdict still lands on the exit code, but the output tail
is exactly what the retry prompt feeds back so the agent can fix the failure.
With it empty the agent is told a gate failed and given nothing to act on, so
it burns every retry and the goal auto-pauses.

workspace_fingerprint has the same two calls; there a non-ASCII path in
`git status --porcelain` empties the fingerprint, silently disabling the
unchanged-gate skip that exists to stop a stalled agent re-running the same
red suite.

Decode as UTF-8 with errors="replace" — what git and modern toolchains emit,
and what 262 of the repo's 299 text-mode subprocess calls already do.
Widen of the PowerShell codepage cluster: the autostart registration
subprocess in tools_config.py was the last text=True capture in these
modules still decoding with the locale code page. Standardize on
encoding='utf-8', errors='replace' like the rest of the file (#53428).
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on 5d285ed

⚠️ Warnings

CI timings · View report · View job

Wall time 9m59s vs 6m56s (+44.0%). 20 job(s) slower, 11 faster, 3 unchanged.

  • JS & TS checks / apps/desktop / check:test:ui: +94.0s
  • Python tests / Run tests slice 7/12: +32.0s
  • Python tests / Run tests slice 6/12: -27.0s
  • Python tests / Run tests slice 3/12: +20.0s
  • Python tests / Run tests slice 12/12: +19.0s

OSV vulnerability scan · View job

64 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants