Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<PackageVersion Include="System.IO.Abstractions" Version="22.1.1" />
<PackageVersion Include="System.CommandLine" Version="2.0.7" />
<PackageVersion Include="Wcwidth" Version="4.0.1" />
<PackageVersion Include="Terminal.Gui.Editor" Version="2.2.6-local" />
<PackageVersion Include="Terminal.Gui.Editor" Version="2.4.1" />
<PackageVersion Include="JetBrains.Annotations" Version="2025.2.4" />
<PackageVersion Include="AnsiConsoleToHtml" Version="0.2.0" />
<PackageVersion Include="ColorHelper" Version="1.8.1" />
Expand Down
121 changes: 121 additions & 0 deletions Examples/WideCharRepro/BUG_REPORT_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Bug Report: Wide/Fullwidth Character Rendering Causes Display Tearing

## Summary

The terminal incorrectly renders wide (fullwidth) Unicode characters, advancing the cursor by only 1 column instead of the required 2. This causes all subsequent content on the line to shift left, breaking grid alignment and causing display "tearing" in TUI applications.

## Reproduction

```bash
git clone https://github.com/gui-cs/Terminal.Gui.git
cd Terminal.Gui/Examples/WideCharRepro
dotnet run
```

Or paste this minimal C# program (requires .NET 8+):

<details>
<summary>Minimal repro (click to expand)</summary>

```csharp
using System.Text;

Console.OutputEncoding = Encoding.UTF8;

Console.WriteLine ("TEST: Emoji grid (each emoji = 2 columns)");
Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┐");
Console.WriteLine ("│😀│😁│😂│🤣│😄│😅│😆│😇│");
Console.WriteLine ("│──│──│──│──│──│──│──│──│");
Console.WriteLine ("│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│");
Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┘");
Console.WriteLine ();
Console.WriteLine ("TEST: Column alignment (X should be at column 20)");
Console.WriteLine ("01234567890123456789X <- 20 narrow (20×1=20)");
Console.WriteLine ("😀😁😂😃😄😅😆😇😈😉X <- 10 emoji (10×2=20)");
Console.WriteLine ("你好世界测试宽字符验X <- 10 CJK (10×2=20)");
```

</details>

## Expected Behavior

Each wide character (emoji, CJK ideograph, fullwidth form) should advance the cursor by **2 columns**. Grid separators (`│`) should be vertically aligned:

```
┌──┬──┬──┬──┬──┬──┬──┬──┐
│😀│😁│😂│🤣│😄│😅│😆│😇│
│──│──│──│──│──│──│──│──│
│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│
└──┴──┴──┴──┴──┴──┴──┴──┘
```

The `X` markers should vertically align at column 20:
```
01234567890123456789X
😀😁😂😃😄😅😆😇😈😉X
你好世界测试宽字符验X
```

## Actual Behavior

Wide characters advance the cursor by only 1 column. Grids misalign and text overlaps:

```
┌──┬──┬──┬──┬──┬──┬──┬──┐
│😀│😁│😂│🤣│😄│😅│😆│😇│ <- shifted left
│──│──│──│──│──│──│──│──│
│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│ <- shifted left
└──┴──┴──┴──┴──┴──┴──┴──┘
```

## Terminal Compatibility Matrix

| Terminal | Version | OS | Test 1 (Emoji) | Test 2 (CJK) | Test 3 (Mixed) | Test 4 (Grid) | Test 5 (Align) | Status |
|---------|---------|-----|:-:|:-:|:-:|:-:|:-:|--------|
| Alacritty | 0.13+ | cross | | | | | | _untested_ |
| Ghostty | — | macOS | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| GitHub Copilot (terminal) | — | macOS | ❌ | ❌ | ❌ | ❌ | ❌ | **FAIL** |
| GitHub Copilot (terminal) | — | Windows | ❌ | ❌ | ❌ | ❌ | ❌ | **FAIL** |
| GNOME Terminal | 3.x | Linux | | | | | | _untested_ |
| iTerm2 | 3.5+ | macOS | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| Kitty | 0.35+ | macOS | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| Terminal.app | — | macOS | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| Visual Studio 2026 (terminal) | — | Windows | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| VS Code Insiders (terminal) | 1.x | Windows | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |
| WezTerm | — | cross | | | | | | _untested_ |
| Windows Terminal | 1.22+ | Windows | ✅ | ✅ | ✅ | ✅ | ✅ | **PASS** |

> **To contributors**: please fill in your terminal's results and submit a PR or comment.

## Affected Unicode Ranges

- **Emoji** (U+1F600–U+1F64F, U+1F900–U+1F9FF, etc.)
- **CJK Unified Ideographs** (U+4E00–U+9FFF)
- **CJK Compatibility Ideographs** (U+F900–U+FAFF)
- **Fullwidth Forms** (U+FF01–U+FF60)
- Any codepoint with Unicode `East_Asian_Width` = `W` (Wide) or `F` (Fullwidth)

## Impact

This bug breaks any TUI application that renders wide characters in a grid or alongside narrow characters. Affected applications include:
- [Terminal.Gui](https://github.com/gui-cs/Terminal.Gui) (cross-platform .NET TUI toolkit)
- Any ncurses/curses-based app using CJK or emoji
- tmux, vim, etc. when displaying wide characters

## Relevant Standards

- [Unicode TR#11 – East Asian Width](https://www.unicode.org/reports/tr11/)
- [POSIX `wcwidth(3)`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html)
- [Unicode TR#51 – Emoji Presentation](https://www.unicode.org/reports/tr51/)

## Environment

- OS: [e.g., Windows 11 24H2]
- Terminal: [e.g., VS Code Insiders 1.x integrated terminal]
- Shell: [e.g., PowerShell 7.5]
- Font: [e.g., Cascadia Code NF]
- Locale: [e.g., en-US, UTF-8]

## Screenshots

<!-- Attach screenshots of correct (Windows Terminal) vs. broken (your terminal) output -->
154 changes: 154 additions & 0 deletions Examples/WideCharRepro/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
// WideCharRepro - Minimal reproduction of wide/fullwidth character rendering issues.
//
// PURPOSE:
// Many terminal emulators incorrectly handle "wide" (fullwidth) Unicode codepoints
// (those with East Asian Width = Wide/Fullwidth, or emoji presentation). These
// characters occupy 2 terminal columns, but some terminals only advance the cursor
// by 1 column, causing subsequent text to overlap and the display to "tear."
//
// EXPECTED BEHAVIOR (correct terminals like Windows Terminal):
// - Each wide character occupies exactly 2 columns.
// - The grid lines up perfectly with no overlapping or shifted text.
// - The separator '|' characters in each row are vertically aligned.
//
// BROKEN BEHAVIOR (terminals with the bug):
// - Wide characters only advance the cursor 1 column instead of 2.
// - Grid columns misalign; text overlaps or shifts left.
// - Vertical '|' separators are NOT aligned across rows.
//
// HOW TO USE:
// dotnet run
// Compare output against a known-good terminal (e.g., Windows Terminal).
//
// DIAGNOSIS:
// If the '|' separators are not vertically aligned, the terminal is not
// correctly handling wide character cursor advancement.

using System.Globalization;
using System.Text;

Console.OutputEncoding = Encoding.UTF8;

// Ensure we're in a mode that supports Unicode output
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
{
// Enable virtual terminal processing on Windows
Console.Write ("\x1b[?25l"); // Hide cursor for cleaner output
}

Console.WriteLine ("═══════════════════════════════════════════════════════════════════");
Console.WriteLine (" Wide Character Rendering Test");
Console.WriteLine (" If '|' separators are NOT vertically aligned, the terminal has");
Console.WriteLine (" a wide-character cursor advancement bug.");
Console.WriteLine ("═══════════════════════════════════════════════════════════════════");
Console.WriteLine ();

// --- Test 1: Emoji (U+1F600 - U+1F64F) ---
Console.WriteLine ("TEST 1: Emoji (each should occupy 2 columns)");
Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐");

int startCodepoint = 0x1F600;
for (int row = 0; row < 4; row++)
{
Console.Write ("│");
for (int col = 0; col < 16; col++)
{
int cp = startCodepoint + (row * 16) + col;
string ch = char.ConvertFromUtf32 (cp);
Console.Write (ch);
Console.Write ("│");
}

Console.WriteLine ();
}

Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘");
Console.WriteLine ();

// --- Test 2: CJK Ideographs (U+4E00+) ---
Console.WriteLine ("TEST 2: CJK Ideographs (each should occupy 2 columns)");
Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐");

startCodepoint = 0x4E00;
for (int row = 0; row < 4; row++)
{
Console.Write ("│");
for (int col = 0; col < 16; col++)
{
int cp = startCodepoint + (row * 16) + col;
string ch = char.ConvertFromUtf32 (cp);
Console.Write (ch);
Console.Write ("│");
}

Console.WriteLine ();
}

Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘");
Console.WriteLine ();

// --- Test 3: Mixed narrow + wide on same line ---
// Each line below is EXACTLY 20 display columns between the │ delimiters.
Console.WriteLine ("TEST 3: Mixed content alignment");
Console.WriteLine ("All lines are exactly 20 display columns. The '│' must align:");
Console.WriteLine ("┌────────────────────┐");
Console.WriteLine ("│ABCDEFGHIJKLMNOPQRST│ <- 20 narrow (20×1=20)");
Console.WriteLine ("│😀😁😂😃😄😅😆😇😈😉│ <- 10 emoji (10×2=20)");
Console.WriteLine ("│你好世界测试宽字符验│ <- 10 CJK (10×2=20)");
Console.WriteLine ("│AB😀CD😁EF😂GH😃IJ😄│ <- mixed (10×1 + 5×2=20)");
Console.WriteLine ("└────────────────────┘");
Console.WriteLine ();

// --- Test 4: ANSI cursor positioning with wide chars ---
Console.WriteLine ("TEST 4: Programmatic cursor-positioned grid");
Console.WriteLine (" Writing wide chars at absolute positions via ANSI escapes.");
Console.WriteLine (" If the terminal handles wcwidth correctly, all rows align.");
Console.WriteLine ();

// Get current cursor row (approximate - just write sequentially with known widths)
string [] testRows =
[
"│😀│😁│😂│🤣│😄│😅│😆│😇│",
"│──│──│──│──│──│──│──│──│",
"│🐶│🐱│🐭│🐹│🐰│🦊│🐻│🐼│",
"│──│──│──│──│──│──│──│──│",
"│你│好│世│界│测│试│宽│字│",
"│──│──│──│──│──│──│──│──│",
];

Console.WriteLine ("┌──┬──┬──┬──┬──┬──┬──┬──┐");

foreach (string line in testRows)
{
Console.WriteLine (line);
}

Console.WriteLine ("└──┴──┴──┴──┴──┴──┴──┴──┘");
Console.WriteLine ();

// --- Test 5: Explicit column-counting verification ---
Console.WriteLine ("TEST 5: Column-width verification");
Console.WriteLine (" The 'X' markers below should align with column 20:");
Console.WriteLine ();
Console.WriteLine ("01234567890123456789X <- 20 narrow (20×1=20)");
Console.WriteLine ("😀😁😂😃😄😅😆😇😈😉X <- 10 emoji (10×2=20)");
Console.WriteLine ("你好世界测试宽字符验X <- 10 CJK (10×2=20)");
Console.WriteLine ("aあbいcうdえeおfかきX <- mixed (6×1 + 7×2=20)");
Console.WriteLine ();
Console.WriteLine ("If the 'X' markers don't vertically align at column 20,");
Console.WriteLine ("the terminal is miscounting wide character widths.");
Console.WriteLine ();

// --- Summary ---
Console.WriteLine ("═══════════════════════════════════════════════════════════════════");
Console.WriteLine (" DIAGNOSIS:");
Console.WriteLine (" • If all grids have aligned '│' separators → terminal is CORRECT");
Console.WriteLine (" • If grids are torn/misaligned → terminal has wcwidth bug");
Console.WriteLine (" • Common cause: terminal treats wide chars as 1 column, not 2");
Console.WriteLine ("═══════════════════════════════════════════════════════════════════");

// Show cursor again
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
{
Console.Write ("\x1b[?25h");
}
59 changes: 59 additions & 0 deletions Examples/WideCharRepro/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Wide Character Rendering Reproduction

A minimal reproduction app demonstrating incorrect wide/fullwidth Unicode character rendering in terminal emulators.

## The Problem

Terminal emulators must advance the cursor by **2 columns** for "wide" Unicode characters (East Asian Width = Wide/Fullwidth, emoji with presentation selectors). Some terminals incorrectly advance by only 1 column, causing:

- Grid misalignment / display "tearing"
- Overlapping text
- Broken TUI (text user interface) applications

## Screenshots

### Correct rendering (Windows Terminal)

All grid separators (`│`) are vertically aligned. Each wide character occupies exactly 2 terminal columns.

![Correct rendering](correct.png)

### Broken rendering (affected terminals)

Grid separators are misaligned. Wide characters only advance the cursor by 1 column, causing all subsequent content on the line to shift left.

![Broken rendering](broken.png)

## Running the Reproduction

```bash
dotnet run
```

**Requirements:** .NET 10 SDK (or change `TargetFramework` in `.csproj` to your installed version, e.g., `net8.0` or `net9.0`).

## What to Look For

1. **Test 1–2 (Emoji / CJK grids):** The `│` separators should form perfectly vertical columns.
2. **Test 3 (Mixed content):** Lines with mixed narrow + wide characters should fit within the box.
3. **Test 4 (Programmatic grid):** Another grid alignment test with diverse wide characters.
4. **Test 5 (Column verification):** The `X` markers should all appear at column 20.

If any of these are misaligned, the terminal has a wide-character width calculation bug.

## Technical Details

Wide characters affected:
- **Emoji** (U+1F600–U+1F64F, U+1F900–U+1F9FF, etc.)
- **CJK Unified Ideographs** (U+4E00–U+9FFF)
- **CJK Compatibility Ideographs** (U+F900–U+FAFF)
- **Fullwidth Forms** (U+FF01–U+FF60)
- Any character with `East_Asian_Width` property = `W` (Wide) or `F` (Fullwidth)

The terminal must use the Unicode `East_Asian_Width` property (or an equivalent wcwidth implementation) to determine cursor advancement after printing each character.

## Related Standards

- [Unicode TR#11 – East Asian Width](https://www.unicode.org/reports/tr11/)
- [POSIX `wcwidth(3)`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html)
- [Unicode Emoji Presentation](https://www.unicode.org/reports/tr51/)
10 changes: 10 additions & 0 deletions Examples/WideCharRepro/WideCharRepro.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>

</Project>
Loading
Loading