Skip to content

fix(tui): force UTF-8 output encoding in PowerShell clipboard read - #38090

Closed
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/windows-clipboard-emoji-utf8
Closed

fix(tui): force UTF-8 output encoding in PowerShell clipboard read#38090
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/windows-clipboard-emoji-utf8

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What does this PR do?

Forces UTF-8 output encoding in the PowerShell clipboard read command so that emoji and CJK characters are preserved instead of being mangled to ? on non-English Windows.

Related Issue

Fixes #38077

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • ui-tui/src/lib/clipboard.ts: Prefix the -Command script with [Console]::OutputEncoding = [System.Text.Encoding]::UTF8 before Get-Clipboard -Raw. Without this, PowerShell defaults to the system ANSI code page (e.g. CP936) and multi-byte characters are mangled to ? when piped through stdout.
  • ui-tui/src/__tests__/clipboard.test.ts: Updated existing Windows and WSL test assertions to match the new args. Added a dedicated regression test (sets UTF-8 output encoding on Windows to preserve emoji) that verifies the encoding prefix is present.

How to Test

  1. On Windows 11 with PowerShell 7 and Windows Terminal, copy an emoji string (e.g. 🎉 hello 🫶) to the clipboard.
  2. Launch hermes --tui and paste into the composer — the emoji should appear correctly, not as ??.
  3. Run cd ui-tui && npx vitest run src/__tests__/clipboard.test.ts — all 20 tests should pass (including the new sets UTF-8 output encoding on Windows to preserve emoji test).

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run npx vitest run src/__tests__/clipboard.test.ts and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS (TypeScript test suite, Windows-specific behavior verified via test mocks)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Code Intelligence

  • Analyzed: ui-tui/src/lib/clipboard.tsPOWERSHELL_ARGS constant, used by readClipboardCommands() (3 call sites: win32, WSL via WSL_INTEROP, WSL via WSL_DISTRO_NAME)
  • Blast radius: LOW — single constant change, PowerShell-only code path, read-only (no side effects)
  • Related patterns: The write path already handles UTF-8 via base64 encoding (_powershellWriteScript). This fix addresses the read path asymmetry. isUsableClipboardText() already handles null bytes and replacement characters as safety net.

PowerShell defaults to the system ANSI code page for console output.
On non-English Windows (e.g. CP936), this mangles emoji and CJK
characters to '?' when piping Get-Clipboard output through stdout.

Prefix the -Command script with [Console]::OutputEncoding assignment
to UTF-8, matching the approach already used on the write path (which
uses base64 encoding to bypass the same issue on stdin).

Fixes NousResearch#38077
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/tui Terminal UI (ui-tui/ + tui_gateway/) labels Jun 3, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused fix. Current main still passes raw Get-Clipboard -Raw through POWERSHELL_ARGS in ui-tui/src/lib/clipboard.ts:6, while readClipboardText() requests UTF-8 decoding at ui-tui/src/lib/clipboard.ts:75-84. The same argument array serves native Windows and WSL at ui-tui/src/lib/clipboard.ts:41-49, so the proposed command prefix covers both affected read paths.

The regression assertions in ui-tui/src/__tests__/clipboard.test.ts directly verify the changed PowerShell command. No substantive problems found.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 14, 2026
@teknium1

teknium1 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Closing — superseded by #37212's base64 strategy (merged in #81963). OutputEncoding forcing works in the common case but is weaker under -NoProfile and WSL interop; base64 removes the codepage from the equation entirely. Your PR correctly identified the bug and beat most to it — thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Windows] Clipboard paste mangles emoji to ?? due to PowerShell -NoProfile bypassing UTF-8 encoding

3 participants