Skip to content

fix(tui): force UTF-8 output when reading clipboard via PowerShell - #54057

Closed
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/clipboard-read-utf8-cjk
Closed

fix(tui): force UTF-8 output when reading clipboard via PowerShell#54057
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/clipboard-read-utf8-cjk

Conversation

@Bartok9

@Bartok9 Bartok9 commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Closes #53963.

Root Cause

Symptom — Pasting Chinese/CJK (or emoji, accented, Cyrillic) text into the Web Dashboard / Desktop app on Windows/WSL produces literal ? per character. Example: 你好世界,测试中文????????????. Affects all non-ASCII text on non-UTF-8 Windows locales.

Root causeui-tui/src/lib/clipboard.ts reads the clipboard with powershell(.exe) -NoProfile -NonInteractive -Command Get-Clipboard -Raw. PowerShell writes that stdout using the system ANSI code page (e.g. CP936 on Chinese Windows), not UTF-8. Hermes then reads the child process stdout with encoding: 'utf8', so ANSI-coded bytes that aren't valid UTF-8 get replaced with ? / U+FFFD.

EvidencePOWERSHELL_ARGS at clipboard.ts:6. Notably the write path in the same file already solved the identical encoding problem (base64 + [System.Text.Encoding]::UTF8.GetString(...) for Set-Clipboard), but the read path was never given the matching treatment. New regression test feeds 你好世界,测试中文 through a mocked run and asserts the emitted command carries the UTF-8 directive; it fails on current main.

Fix + why this level — Prepend [Console]::OutputEncoding = [System.Text.Encoding]::UTF8; to the Get-Clipboard -Raw command so PowerShell emits UTF-8 stdout, which then matches the encoding: 'utf8' read. This is the correct layer: fixing the producer's output encoding removes the mismatch at the source, rather than guessing the ANSI code page on the Node side (which varies by locale and is unreliable). It mirrors the existing write-path solution for symmetry.

Scope / risk — Single constant change in clipboard.ts (shared by both the win32 and WSL powershell.exe read attempts). ASCII-only clipboards are unaffected (UTF-8 is a superset of ASCII). No change to the macOS/Wayland/X11 backends or the write path.

Verification

  • npx vitest run src/__tests__/clipboard.test.ts (in ui-tui/) — 20 passed
  • Updated the two existing PowerShell-args assertions and added forces UTF-8 output encoding in the PowerShell read command (CJK fix), which round-trips 你好世界,测试中文 and asserts the command contains [System.Text.Encoding]::UTF8. It FAILS on main (3 tests fail without the fix) and passes with it.

Real behavior proof

New + updated tests on this branch (after fix):

✓ src/__tests__/clipboard.test.ts (20 tests) 8ms
Test Files  1 passed (1)
     Tests  20 passed (20)

On origin/main (source reverted, tests kept):

Test Files  1 failed (1)
     Tests  3 failed | 17 passed (20)

Related

@alt-glitch alt-glitch added type/bug Something isn't working comp/tui Terminal UI (ui-tui/ + tui_gateway/) platform/windows Native Windows-specific behavior or breakage P3 Low — cosmetic, nice to have labels Jun 28, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Fixes CJK/emoji clipboard paste on Windows by forcing PowerShell to output UTF-8 via [Console]::OutputEncoding = [System.Text.Encoding]::UTF8. Without this, PowerShell emits the system ANSI code page (e.g. CP936 on Chinese Windows) and non-ASCII chars are decoded as ?. Clean one-line fix with a dedicated CJK regression test.


Reviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Forces UTF-8 output encoding when reading clipboard via PowerShell on Windows. Fixes CJK/emoji paste bug where non-ASCII chars were decoded as ? because PowerShell emitted ANSI code page.

Changes

  • ui-tui/src/lib/clipboard.ts: Prepends [Console]::OutputEncoding = [System.Text.Encoding]::UTF8; to Get-Clipboard command
  • ui-tui/src/__tests__/clipboard.test.ts: Updated existing tests + 1 new CJK-specific test

Looks Good

  • Mirrors the base64+UTF8.GetString approach already used on the write path
  • Existing tests updated to match new command format
  • Dedicated CJK test verifies the exact regression scenario

(Note: prior COMMENT review from tonydwb exists; this is a new review.)


Reviewed by Hermes Agent

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused Windows/WSL clipboard fix. Current main still passes raw Get-Clipboard -Raw through POWERSHELL_ARGS at ui-tui/src/lib/clipboard.ts:6; that array is shared by native Windows (:41-43) and WSL (:47-49), while readClipboardText() requests UTF-8 decoding at :77-84.

The updated assertions in ui-tui/src/__tests__/clipboard.test.ts verify the changed command for both PowerShell call paths. No substantive implementation issue found in this scoped diff.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 15, 2026
@teknium1

teknium1 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Closing — duplicate of the earlier #38090 (June 3 vs June 28), and both superseded by the base64 approach from #37212 merged in #81963. Thanks for the fix.

@teknium1 teknium1 closed this Aug 8, 2026
@Bartok9

Bartok9 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Understood — thanks for the pointer. Glad the base64 path from #37212 / #81963 landed as the lasting fix. Closing out on our side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have platform/windows Native Windows-specific behavior or breakage sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Pasting Chinese/CJK into Web Dashboard produces ??? due to PowerShell ANSI code page

4 participants