Skip to content

# Fix Windows VT input encoding for IME and non-ASCII characters #5453

Description

@BDisp

Summary

When running Terminal.Gui with the ANSI/VT driver on Windows, non-ASCII characters entered through an IME (for example Chinese characters) may be received as ? instead of the expected Unicode character.

The issue occurs because the ANSI driver configures console output as UTF-8 but VT input decoding uses the current Windows console input code page (GetConsoleCP()).

If the input code page is not UTF-8 (65001), Windows can replace unsupported characters before Terminal.Gui receives them.

Reproduction

  1. Configure a Windows console with a non-UTF8 input code page (e.g. 437 or 850).
  2. Start a Terminal.Gui application using the ANSI driver.
  3. Switch to a Chinese IME.
  4. Enter a character such as:

Expected

The application receives:

Actual

The application receives:

?

Root Cause

The ANSI driver sets:

Console.OutputEncoding = Encoding.UTF8;

However, VT input decoding uses the console input code page:

Encoding.GetEncoding((int)GetConsoleCP())

This creates an encoding mismatch:

  • Output: UTF-8
  • Input: Current console code page (CP437, CP850, etc.)

Characters that cannot be represented by the active input code page are replaced before Terminal.Gui processes the VT input stream.

Solution

When the ANSI driver is active:

  1. Save the current console input code page.
  2. Set the console input code page to UTF-8 (65001).
  3. Decode VT input using UTF-8.
  4. Restore the original console input code page when disposing the driver.

Before

Console.OutputEncoding = Encoding.UTF8;

ConsoleInputEncoding =
    Encoding.GetEncoding((int)GetConsoleCP());

After

SetConsoleCP(65001);

ConsoleInputEncoding = Encoding.UTF8;

and restore the original code page during disposal.

Impact

This fixes:

  • Chinese IME input
  • Japanese IME input
  • Korean IME input
  • Other Unicode characters outside the active Windows code page

while preserving the user's original console configuration after the application exits.

Notes

The issue is only visible on Windows when the ANSI/VT driver is used and the console input code page is not already UTF-8.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Status
    ✅ Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions