Console rendering bug when displaying VT on HDPI

[PhMajerus]

In RS2 / Creators Update (15063.138)
The console sometimes displays extra spaces in strings containing ANSI/VT colors when displayed at another DPI scaling than 100%. It happens a lot at 125% and 175% and a bit less but still common at 150%.

Testing the following in a console with the default Consolas font at 20 pixels high (default settings in latin script regions), on a system set to 150% (default on a Surface Pro device). ClearType is enabled, as that can change the rendered text width.

echo -e "\u2592##"
Displays a medium shade block followed by two # characters, 3 columns total.

echo -e "\E[41m\u2592#\E[49m#"
Should display the same characters, but showing the shade block and the first # over a red background.
Instead, it inserts an extra space of exactly 1 column width between the two # characters.

This is a bug in conhost and affects any console app, not just WSL. Confirmed in Win32 as well.
The console seems to output text in chunks of characters using identical attributes. If a color is changed, it computes the x location for the next chunk and starts displaying from that column. It seems to work fine when the display DPI is set to a 100% scaling, but it is regularly off by one when another system scaling ratio is used. This inserts spaces in colored text, breaking alignment.

It depends on previously displayed characters, but the shading blocks aren't the only ones affected. It seems the console computes the starting column of the next chunk based on the pixels width of the previous chunks instead of simply by counting the displayable characters, and probably rounds up some width that overflows a bit into the next column in some fonts and smoothing modes when rendered at some DPI scaling zoom.

This breaks many ANSI-art effects and complex text-based UI layouts.

Related: Parsing bug in 24bit VT colors (#1681)
These should probably be handled together, as they both exhibit a problem with how the console stores and render colored text.

[zadjii-msft]

Just adding some of my own investigation. This appears to vary based on a combination of font AND size, not just the font alone. The first console in this image has (Consolas, 16), the second is (Consolas, 20), and the third shows what happens when you output [(Courier New 20), (Consolas, 20), (Lucidia Console, 20)].

Consolas 16 doesn't have this problem, but Consolas 20 does.

@miniksa for more investigation. I opened MSFT:11641556 to track this.

[miniksa]

The problem here is that \u2592 is an ambiguous width character as defined in Unicode: http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt

That means I have no idea if it's going to be a single column or two columns wide. So the heuristic we use is how many pixels is it going to consume versus a '0' in the currently selected font. Apparently in Consolas 16, that shaded box character consumes less than a '0' and in Consolas 20, it consumes more pixels than a '0' which is what is causing this rendering snafu. Adding High DPI to the mix makes this worse because now there's a MulDiv calculation with rounding potential to make the shaded box have more or less pixels than a '0'.

If you have any ideas on how to more reliably detect whether it will take 1 or 2 columns in GDI, I'm all ears. This is just a very sensitive code path because drawing to the screen takes about 80% of our CPU time (because we still use GDI and it's done on the CPU) so I'm hesitant to make some wildly complicated mechanism here that might significantly impact the CPU usage/time. I think the correct solution is to just capture this problem in the scope of the work required to move to a modern renderer like DirectWrite on DirectX. Unfortunately that means no quick fix.

[PhMajerus]

Some tests showing how chunks of colored text aligns using a proportional font. This is an unsupported scenario as the console is only for monospace fonts, but it shows how it renders ANSI blocks of identical attributes.

Language is JavaScript with the console available through the object model, should be self-explanatory.
ActiveScript Shell (AXSH) is available on Win10 Store if you want to use it for testing. Repro commands:
with (console.font) { name="Arial"; size=20 }
echo("\33[41m\u2592#\33[49m#")
echo("\33[41mi#\33[49m#")
echo("\33[41mm#\33[49m#")
echo(fun.RainbowCat)
show(console.font)

[PhMajerus]

@miniksa I had good results with the following code to determine the columns used by a string:

ansi.getDisplayWidth uses GetStringTypeW(CT_CTYPE3, bstrText, cchLine, rgType), skips escape sequences, then simply check each character width as follow:
if (rgType[i] & (C3_HIRAGANA | C3_KATAKANA | C3_FULLWIDTH | C3_IDEOGRAPH)) // FullWidth catches fw-katakana and fw-romaji
{
// Full-width katakana with " or ° gives 0x8012 (C3_ALPHA|C3_KATAKANA|C3_DIACRITIC)
// so we check if it's a wide character family, and then if it isn't explicitly half-width variant
if (rgType[i] & C3_HALFWIDTH)
displaylength += 1;
else
displaylength += 2;
}
else
displaylength++;

You can completely ignore the non-spacing diacritics (C3_NONSPACING) as the console shows them as individual spacing characters anyway, so this test will correctly count them as 1 column.

[PhMajerus]

@miniksa If you really want to check the actual GDI width of the character in the specific font, asian glyphs are very consistent with double-width always taking a full square.
Since what you want is guess if it's single column or double column, placing the trigger at the middle of the second column seems more reliable.
And I wouldn't get the width of the '0' char, I would use the exact same code you have in GetCurrentConsoleFontEx to get the columns width in pixels returned as cfix.dwFontSize.X.

So I would change the test from (chrPixelWidth > pixelWidth('0')) to (charPixelWidth > cfix.dwFontSize.X*1.5), which should still catch all full-width without false-positives on half-width that bleeds a bit outside a single column.

[miniksa]

Good ideas indeed. I still have to think it through a bit further though because things like the selection highlight are still going to be messed up if something takes 9 pixels when a '0' (or whatever metric we use like perhaps your suggestion above instead) takes 8 pixels. The characters will jump on redraw or the selection rectangle won't be drawn on top of the right spot.

The only way I can think to fix that is to draw each character individually and chop it down to force it to fit inside its rectangle (or to leave enough space before drawing the next one) instead of drawing each row as a run like we do right now. The problem is, we draw runs because the performance is already terrible to call draw 25 times to draw 25 lines on the screen. I can't imagine how bad it would be to call to draw 80x25 individual characters on the screen (or more). Maybe some trade-off where it only draws individually when detecting an ambiguous character? Or maybe there's some GDI trickery I haven't figured out yet...

[simonbuchan]

@zadjii-msft Moving here from #1936

Hmm. If I have a choice between prioritizing support for 3rd-party consoles (which should also cover things like embedded terminals, e.g. VS code, right?) and making the default console awesome, I'll pick the first, since a third-party console doesn't have to care about compatibility and it takes the pressure off you guys to support everything perfectly now without breaking anything 😉

That said, this looks like a curly one. It sounds like you're currently using DrawText(), could switching to ExtTextOut() for the lpDx parameter be a decent workaround for now?

[ryanerwin]

@miniksa You wrote:

I think the correct solution is to just capture this problem in the scope of the work required to move to a modern renderer like DirectWrite on DirectX. Unfortunately that means no quick fix.

So do you folks actually have plans to make a new Terminal? There are so many excellent terminal's included with Linux, seems like porting one of those to Windows would be the easiest path... And how great to have forks of the terminal code to add things like image support, higher range of colors...

ConEmu seems to be the least painful option for now, but still a long ways from the native Linux terminals...

[simonbuchan]

@ryanerwin #111 seems to be, despite the obsolete title, the closest thing to an issue tracking "make the console api public", but basically all the label:console issues are "make a new terminal", in the sense of "fix everything in the current terminal: conhost.exe"

[PhMajerus]

@ryanerwin Considering the console performance is directly linked to its text rendering calls, making a cross-platform console with some abstraction in the text rendering isn't the right way to go. Abstraction always comes at a cost.

A modern console using DirectWrite is the best way to go for a console on Windows and since the performance would impact all text-based apps, the effort to write a good console taking advantage of the DirectX platform is justified.

[PhMajerus]

@miniksa Taking another approach here.

The problem is basically linked to how some fonts render at different sizes and mostly to how block elements and line (box) drawing characters render imperfectly at the pixel level, ending up a bit too narrow or too wide (and exhibit the same problem with their height, btw).
This issue is a font-hinting issue first, and even if your conhost code could work around it, it would still display glitches at the pixel level for ANSI-art and ASCII-based boxes, such as shading blocks not joined to a uniform shade (non-uniform checkerboard), or background colored lines showing a bit around full-block cells (U+2588).

So how about you push this problem to the Typography team and get them to fix the fonts that are now used for the console, mainly Consolas, to be better hinted and generate pixel-perfect blocks and lines glyphs at all DPI and font sizes supported by conhost?

This would mitigate the current issue by making the characters that currently exhibit these issues fit their cells perfectly. And it wouldn't be wasted work on a temporary fix as it would make the rendering with the standard fonts more precise and better looking even on a future DirectWrite-based console.
Even better, updating TTF files is probably easier to push in a monthly update than a new build of conhost, so this could fix many scenarios on a patch Tuesday.

[PhMajerus]

@miniksa
Most of this seems fixed in 1709, but I do have some bad news...

I didn't notice this bug before the release of 1709 because it only happens in some rare cases, and I must admit I'm still unable to build a simple project that reproduces the issue.
But there is a problem with how "\x1B[7m" and "\x1B[27m" are handled. The first sets reverse mode, the second returns to normal mode. Most of the time it works fine, but sometimes, the second one gets ignored, and reverse mode continues until some other in-band control changes the console internal state again.

This is very erratic, it seems to be linked to font size again, but even at the same font size, a call to SetConsoleMode, or showing and OK'ing the console properties dialog (which I assume calls the same API) usually clears the error forever in that console window.
Very very weird thing.
It doesn't seem to affect current LXSS shells, so I assume it is highly dependent on calls to SetConsoleMode for that conhost instance.

Any idea ?

[miniksa]

@zadjii-msft have you seen anything with the 7m and 27m states recently that might explain this?

[zadjii-msft]

Nope, this looks new to me. Might be related to the reflowing thing, but I don't think that code path would be hit here.

I've filed a bug to look into this later this cycle.

[PhMajerus]

@zadjii-msft
As usual, writing to explain about it to someone else makes my brain think about it from another perspective and other cases.
And... good news, I just had a guess and found how to replicate the issue !

I couldn't replicate it because I was focusing my sample code on the \x1B[7m and \x1B[27m codes, but it only happens if you've set 24-bit VT colors earlier.
I suppose calls to SetConsoleMode or OK'ing the console properties dialog resets some conhost internal state about the current color.
So it's very easy to replicate :

#include <windows.h>

#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif

int main()
{
	HANDLE hOutput = GetStdHandle(STD_OUTPUT_HANDLE);
	
	DWORD dwMode = 0;
	GetConsoleMode(hOutput, &dwMode);
	dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
	SetConsoleMode(hOutput, dwMode);

	LPTSTR pszTestString = TEXT("\x1B[7m Reversed \x1B[27m Normal \x1B[38;2;128;5;255m Set 24-bit foreColor \x1B[7m Reversed \x1B[27m Normal \r\n");
	WriteConsole(hOutput, pszTestString, _tcslen(pszTestString), NULL, NULL);

	getchar();
	return 0;
}

[PhMajerus]

And now with that understood, I can replicate in Ubuntu (Store build) on ver. 1709 (RTW, build 16299.19):
echo -e "\e[7m Reversed \e[27m Normal \e[38;2;250;20;40m 24-bit VT color \e[7m Reversed \e[27m Normal "

[PhMajerus]

@zadjii-msft
Since it doesn't seem, after all, to be linked to font sizes, should I leave this in this thread or close this one and post this info clean in a new issue thread?

[zadjii-msft]

@PhMajerus Awesome! thanks for the super simple repro steps. I'd say that it's unlikely I fixed the reverse sequence to work with RGB color. That shouldn't be too hard to fix though. Thanks for investigating!

I'd say it's best to keep the threads separate, so if you could file a new issue that'd be great.

[PhMajerus]

@zadjii-msft
Be careful it's not only when the current fore or back color is 24-bit, you have a state that lingers in conhost even if the color has been reset:
echo -e "\e[7m Reversed \e[27m Normal \e[38;2;250;20;40m 24-bit VT color \e[39m reset color \e[ 7m Reversed \e[27m Normal "

So it's not just in your \x1B[27m handler code path that handles if current fore or back is 24-bit, as in this sample it isn't anymore. There is something else that makes it stop working even if the color is back to a base console color.

[PhMajerus]

Closing this issue as characters width dependent on font-size issue is fixed.
VT "\x1B[7m" and "\x1B[27m" issue moved to its own thread: #2583
Thanks again for your hard work @zadjii-msft and @miniksa !