-
Notifications
You must be signed in to change notification settings - Fork 782
Add wide character rendering repro app for terminal bug reports #5397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+344
−0
Merged
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # Bug Report: Wide/Fullwidth Character Rendering Causes Display Tearing | ||
|
|
||
| ## Summary | ||
|
|
||
| The terminal incorrectly renders wide (fullwidth) Unicode characters, advancing the cursor by only 1 column instead of the required 2. This causes all subsequent content on the line to shift left, breaking grid alignment and causing display "tearing" in TUI applications. | ||
|
|
||
| ## Reproduction | ||
|
|
||
| ```bash | ||
| git clone https://github.com/gui-cs/Terminal.Gui.git | ||
| cd Terminal.Gui/Examples/WideCharRepro | ||
| dotnet run | ||
| ``` | ||
|
|
||
| Or paste this minimal C# program (requires .NET 8+): | ||
|
|
||
| <details> | ||
| <summary>Minimal repro (click to expand)</summary> | ||
|
|
||
| ```csharp | ||
| using System.Text; | ||
|
|
||
| Console.OutputEncoding = Encoding.UTF8; | ||
|
|
||
| Console.WriteLine ("TEST: Emoji grid (each emoji = 2 columns)"); | ||
| Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┐"); | ||
| Console.WriteLine ("│😀│😁│😂│🤣│😄│😅│😆│😇│"); | ||
| Console.WriteLine ("│──│──│──│──│──│──│──│──│"); | ||
| Console.WriteLine ("│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│"); | ||
| Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┘"); | ||
| Console.WriteLine (); | ||
| Console.WriteLine ("TEST: Column alignment (X should be at column 20)"); | ||
| Console.WriteLine ("01234567890123456789X <- 20 narrow (20×1=20)"); | ||
| Console.WriteLine ("😀😁😂😃😄😅😆😇😈😉X <- 10 emoji (10×2=20)"); | ||
| Console.WriteLine ("你好世界测试宽字符验X <- 10 CJK (10×2=20)"); | ||
| ``` | ||
|
|
||
| </details> | ||
|
|
||
| ## Expected Behavior | ||
|
|
||
| Each wide character (emoji, CJK ideograph, fullwidth form) should advance the cursor by **2 columns**. Grid separators (`│`) should be vertically aligned: | ||
|
|
||
| ``` | ||
| ┌──┬──┬──┬──┬──┬──┬──┬──┐ | ||
| │😀│😁│😂│🤣│😄│😅│😆│😇│ | ||
| │──│──│──│──│──│──│──│──│ | ||
| │🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│ | ||
| └──┴──┴──┴──┴──┴──┴──┴──┘ | ||
| ``` | ||
|
|
||
| The `X` markers should vertically align at column 20: | ||
| ``` | ||
| 01234567890123456789X | ||
| 😀😁😂😃😄😅😆😇😈😉X | ||
| 你好世界测试宽字符验X | ||
| ``` | ||
|
|
||
| ## Actual Behavior | ||
|
|
||
| Wide characters advance the cursor by only 1 column. Grids misalign and text overlaps: | ||
|
|
||
| ``` | ||
| ┌──┬──┬──┬──┬──┬──┬──┬──┐ | ||
| │😀│😁│😂│🤣│😄│😅│😆│😇│ <- shifted left | ||
| │──│──│──│──│──│──│──│──│ | ||
| │🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│ <- shifted left | ||
| └──┴──┴──┴──┴──┴──┴──┴──┘ | ||
| ``` | ||
|
|
||
| ## Terminal Compatibility Matrix | ||
|
|
||
| | Terminal | Version | OS | Test 1 (Emoji) | Test 2 (CJK) | Test 3 (Mixed) | Test 4 (Grid) | Test 5 (Align) | Status | | ||
| |---------|---------|-----|:-:|:-:|:-:|:-:|:-:|--------| | ||
| | Windows Terminal | 1.22+ | Windows | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** | | ||
| | VS Code Insiders (terminal) | 1.x | Windows | ✅ | ✅ | ❌ | ✅ | ✅ | **FAIL** | | ||
| | GitHub Copilot (terminal) | — | Windows | ❌ | ❌ | ❌ | ❌ | ❌ | **FAIL** | | ||
| | iTerm2 | 3.5+ | macOS | | | | | | _untested_ | | ||
| | Terminal.app | — | macOS | | | | | | _untested_ | | ||
| | GNOME Terminal | 3.x | Linux | | | | | | _untested_ | | ||
| | Alacritty | 0.13+ | cross | | | | | | _untested_ | | ||
| | Kitty | 0.35+ | cross | | | | | | _untested_ | | ||
| | WezTerm | — | cross | | | | | | _untested_ | | ||
|
|
||
| > **To contributors**: please fill in your terminal's results and submit a PR or comment. | ||
|
|
||
| ## Affected Unicode Ranges | ||
|
|
||
| - **Emoji** (U+1F600–U+1F64F, U+1F900–U+1F9FF, etc.) | ||
| - **CJK Unified Ideographs** (U+4E00–U+9FFF) | ||
| - **CJK Compatibility Ideographs** (U+F900–U+FAFF) | ||
| - **Fullwidth Forms** (U+FF01–U+FF60) | ||
| - Any codepoint with Unicode `East_Asian_Width` = `W` (Wide) or `F` (Fullwidth) | ||
|
|
||
| ## Impact | ||
|
|
||
| This bug breaks any TUI application that renders wide characters in a grid or alongside narrow characters. Affected applications include: | ||
| - [Terminal.Gui](https://github.com/gui-cs/Terminal.Gui) (cross-platform .NET TUI toolkit) | ||
| - Any ncurses/curses-based app using CJK or emoji | ||
| - tmux, vim, etc. when displaying wide characters | ||
|
|
||
| ## Relevant Standards | ||
|
|
||
| - [Unicode TR#11 – East Asian Width](https://www.unicode.org/reports/tr11/) | ||
| - [POSIX `wcwidth(3)`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html) | ||
| - [Unicode TR#51 – Emoji Presentation](https://www.unicode.org/reports/tr51/) | ||
|
|
||
| ## Environment | ||
|
|
||
| - OS: [e.g., Windows 11 24H2] | ||
| - Terminal: [e.g., VS Code Insiders 1.x integrated terminal] | ||
| - Shell: [e.g., PowerShell 7.5] | ||
| - Font: [e.g., Cascadia Code NF] | ||
| - Locale: [e.g., en-US, UTF-8] | ||
|
|
||
| ## Screenshots | ||
|
|
||
| <!-- Attach screenshots of correct (Windows Terminal) vs. broken (your terminal) output --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| // WideCharRepro - Minimal reproduction of wide/fullwidth character rendering issues. | ||
| // | ||
| // PURPOSE: | ||
| // Many terminal emulators incorrectly handle "wide" (fullwidth) Unicode codepoints | ||
| // (those with East Asian Width = Wide/Fullwidth, or emoji presentation). These | ||
| // characters occupy 2 terminal columns, but some terminals only advance the cursor | ||
| // by 1 column, causing subsequent text to overlap and the display to "tear." | ||
| // | ||
| // EXPECTED BEHAVIOR (correct terminals like Windows Terminal): | ||
| // - Each wide character occupies exactly 2 columns. | ||
| // - The grid lines up perfectly with no overlapping or shifted text. | ||
| // - The separator '|' characters in each row are vertically aligned. | ||
| // | ||
| // BROKEN BEHAVIOR (terminals with the bug): | ||
| // - Wide characters only advance the cursor 1 column instead of 2. | ||
| // - Grid columns misalign; text overlaps or shifts left. | ||
| // - Vertical '|' separators are NOT aligned across rows. | ||
| // | ||
| // HOW TO USE: | ||
| // dotnet run | ||
| // Compare output against a known-good terminal (e.g., Windows Terminal). | ||
| // | ||
| // DIAGNOSIS: | ||
| // If the '|' separators are not vertically aligned, the terminal is not | ||
| // correctly handling wide character cursor advancement. | ||
|
|
||
| using System.Globalization; | ||
| using System.Text; | ||
|
|
||
| Console.OutputEncoding = Encoding.UTF8; | ||
|
|
||
| // Ensure we're in a mode that supports Unicode output | ||
| if (Environment.OSVersion.Platform == PlatformID.Win32NT) | ||
| { | ||
| // Enable virtual terminal processing on Windows | ||
| Console.Write ("\x1b[?25l"); // Hide cursor for cleaner output | ||
| } | ||
|
|
||
| Console.WriteLine ("═══════════════════════════════════════════════════════════════════"); | ||
| Console.WriteLine (" Wide Character Rendering Test"); | ||
| Console.WriteLine (" If '|' separators are NOT vertically aligned, the terminal has"); | ||
| Console.WriteLine (" a wide-character cursor advancement bug."); | ||
| Console.WriteLine ("═══════════════════════════════════════════════════════════════════"); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Test 1: Emoji (U+1F600 - U+1F64F) --- | ||
| Console.WriteLine ("TEST 1: Emoji (each should occupy 2 columns)"); | ||
| Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐"); | ||
|
|
||
| int startCodepoint = 0x1F600; | ||
| for (int row = 0; row < 4; row++) | ||
| { | ||
| Console.Write ("│"); | ||
| for (int col = 0; col < 16; col++) | ||
| { | ||
| int cp = startCodepoint + (row * 16) + col; | ||
| string ch = char.ConvertFromUtf32 (cp); | ||
| Console.Write (ch); | ||
| Console.Write ("│"); | ||
| } | ||
|
|
||
| Console.WriteLine (); | ||
| } | ||
|
|
||
| Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘"); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Test 2: CJK Ideographs (U+4E00+) --- | ||
| Console.WriteLine ("TEST 2: CJK Ideographs (each should occupy 2 columns)"); | ||
| Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐"); | ||
|
|
||
| startCodepoint = 0x4E00; | ||
| for (int row = 0; row < 4; row++) | ||
| { | ||
| Console.Write ("│"); | ||
| for (int col = 0; col < 16; col++) | ||
| { | ||
| int cp = startCodepoint + (row * 16) + col; | ||
| string ch = char.ConvertFromUtf32 (cp); | ||
| Console.Write (ch); | ||
| Console.Write ("│"); | ||
| } | ||
|
|
||
| Console.WriteLine (); | ||
| } | ||
|
|
||
| Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘"); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Test 3: Mixed narrow + wide on same line --- | ||
| // Each line below is EXACTLY 20 display columns between the │ delimiters. | ||
| Console.WriteLine ("TEST 3: Mixed content alignment"); | ||
| Console.WriteLine ("All lines are exactly 20 display columns. The '│' must align:"); | ||
| Console.WriteLine ("┌────────────────────┐"); | ||
| Console.WriteLine ("│ABCDEFGHIJKLMNOPQRST│ <- 20 narrow (20×1=20)"); | ||
| Console.WriteLine ("│😀😁😂😃😄😅😆😇😈😉│ <- 10 emoji (10×2=20)"); | ||
| Console.WriteLine ("│你好世界测试宽字符验│ <- 10 CJK (10×2=20)"); | ||
| Console.WriteLine ("│AB😀CD😁EF😂GH😃IJ😄│ <- mixed (10×1 + 5×2=20)"); | ||
| Console.WriteLine ("└────────────────────┘"); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Test 4: ANSI cursor positioning with wide chars --- | ||
| Console.WriteLine ("TEST 4: Programmatic cursor-positioned grid"); | ||
| Console.WriteLine (" Writing wide chars at absolute positions via ANSI escapes."); | ||
| Console.WriteLine (" If the terminal handles wcwidth correctly, all rows align."); | ||
| Console.WriteLine (); | ||
|
|
||
| // Get current cursor row (approximate - just write sequentially with known widths) | ||
| string [] testRows = | ||
| [ | ||
| "│😀│😁│😂│🤣│😄│😅│😆│😇│", | ||
| "│──│──│──│──│──│──│──│──│", | ||
| "│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│", | ||
| "│──│──│──│──│──│──│──│──│", | ||
| "│你│好│世│界│测│试│宽│字│", | ||
| "│──│──│──│──│──│──│──│──│", | ||
| ]; | ||
|
|
||
| Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┐"); | ||
|
|
||
| foreach (string line in testRows) | ||
| { | ||
| Console.WriteLine (line); | ||
| } | ||
|
|
||
| Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┘"); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Test 5: Explicit column-counting verification --- | ||
| Console.WriteLine ("TEST 5: Column-width verification"); | ||
| Console.WriteLine (" The 'X' markers below should align with column 20:"); | ||
| Console.WriteLine (); | ||
| Console.WriteLine ("01234567890123456789X <- 20 narrow (20×1=20)"); | ||
| Console.WriteLine ("😀😁😂😃😄😅😆😇😈😉X <- 10 emoji (10×2=20)"); | ||
| Console.WriteLine ("你好世界测试宽字符验X <- 10 CJK (10×2=20)"); | ||
| Console.WriteLine ("aあbいcうdえeおfかきX <- mixed (6×1 + 7×2=20)"); | ||
| Console.WriteLine (); | ||
| Console.WriteLine ("If the 'X' markers don't vertically align at column 20,"); | ||
| Console.WriteLine ("the terminal is miscounting wide character widths."); | ||
| Console.WriteLine (); | ||
|
|
||
| // --- Summary --- | ||
| Console.WriteLine ("═══════════════════════════════════════════════════════════════════"); | ||
| Console.WriteLine (" DIAGNOSIS:"); | ||
| Console.WriteLine (" • If all grids have aligned '│' separators → terminal is CORRECT"); | ||
| Console.WriteLine (" • If grids are torn/misaligned → terminal has wcwidth bug"); | ||
| Console.WriteLine (" • Common cause: terminal treats wide chars as 1 column, not 2"); | ||
| Console.WriteLine ("═══════════════════════════════════════════════════════════════════"); | ||
|
|
||
| // Show cursor again | ||
| if (Environment.OSVersion.Platform == PlatformID.Win32NT) | ||
| { | ||
| Console.Write ("\x1b[?25h"); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # Wide Character Rendering Reproduction | ||
|
|
||
| A minimal reproduction app demonstrating incorrect wide/fullwidth Unicode character rendering in terminal emulators. | ||
|
|
||
| ## The Problem | ||
|
|
||
| Terminal emulators must advance the cursor by **2 columns** for "wide" Unicode characters (East Asian Width = Wide/Fullwidth, emoji with presentation selectors). Some terminals incorrectly advance by only 1 column, causing: | ||
|
|
||
| - Grid misalignment / display "tearing" | ||
| - Overlapping text | ||
| - Broken TUI (text user interface) applications | ||
|
|
||
| ## Screenshots | ||
|
|
||
| ### Correct rendering (Windows Terminal) | ||
|
|
||
| All grid separators (`│`) are vertically aligned. Each wide character occupies exactly 2 terminal columns. | ||
|
|
||
|  | ||
|
|
||
| ### Broken rendering (affected terminals) | ||
|
|
||
| Grid separators are misaligned. Wide characters only advance the cursor by 1 column, causing all subsequent content on the line to shift left. | ||
|
|
||
|  | ||
|
|
||
| ## Running the Reproduction | ||
|
|
||
| ```bash | ||
| dotnet run | ||
| ``` | ||
|
|
||
| **Requirements:** .NET 10 SDK (or change `TargetFramework` in `.csproj` to your installed version, e.g., `net8.0` or `net9.0`). | ||
|
|
||
| ## What to Look For | ||
|
|
||
| 1. **Test 1–2 (Emoji / CJK grids):** The `│` separators should form perfectly vertical columns. | ||
| 2. **Test 3 (Mixed content):** Lines with mixed narrow + wide characters should fit within the box. | ||
| 3. **Test 4 (Programmatic grid):** Another grid alignment test with diverse wide characters. | ||
| 4. **Test 5 (Column verification):** The `X` markers should all appear at column 20. | ||
|
|
||
| If any of these are misaligned, the terminal has a wide-character width calculation bug. | ||
|
|
||
| ## Technical Details | ||
|
|
||
| Wide characters affected: | ||
| - **Emoji** (U+1F600–U+1F64F, U+1F900–U+1F9FF, etc.) | ||
| - **CJK Unified Ideographs** (U+4E00–U+9FFF) | ||
| - **CJK Compatibility Ideographs** (U+F900–U+FAFF) | ||
| - **Fullwidth Forms** (U+FF01–U+FF60) | ||
| - Any character with `East_Asian_Width` property = `W` (Wide) or `F` (Fullwidth) | ||
|
|
||
| The terminal must use the Unicode `East_Asian_Width` property (or an equivalent wcwidth implementation) to determine cursor advancement after printing each character. | ||
|
|
||
| ## Related Standards | ||
|
|
||
| - [Unicode TR#11 – East Asian Width](https://www.unicode.org/reports/tr11/) | ||
| - [POSIX `wcwidth(3)`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html) | ||
| - [Unicode Emoji Presentation](https://www.unicode.org/reports/tr51/) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>net10.0</TargetFramework> | ||
| <Nullable>enable</Nullable> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| </PropertyGroup> | ||
|
|
||
| </Project> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.