WebGL Renderer

[Tyriar]

This PR introduces the 'webgl' renderer which is basically canvas but much lower level and much faster. I see this eventually replacing the canvas renderer but it will need some time before that happens.

But there's already a canvas renderer?

To illustrate the difference when drawing, the canvas renderer basically does this:

1. Tell the GPU to draw character 0,0 from texture a
2. Tell the GPU to draw character 1,0 from texture a
3. Tell the GPU to draw character 2,0 from texture a
4. and so on.

These individual instructions are costly, the webgl renderer instead builds a Float32Array containing all the data needed to draw and a "webgl program" (a vertex shader and a fragment shader) that knows how to draw things from the Float32Array is uploaded to the GPU which does the actual drawing. Of course the details are much more involved than this, but that's the basics of why the webgl renderer will be superior to the canvas renderer.

Features

• It's super fast, it also scales much better with really large viewports.
• All characters are cached in the atlas, including unicode, combined chars (emoji), etc.
• The atlas trims glyphs to their minimal rectangles and stores that in the texture, this means better space utilization of the texture. Here's a smaller 256x256 example illustrating this:
  
• Selection is now drawn underneath the text (Selection should not change text color #720), using a uniform selection fg for accessibility reasons:
  
• Since the buffer is redrawn every frame, characters are no longer trimmed at the top and the bottom (Tall characters have their top and/or bottom cut off #1138).
  
  
• Renderer pausing when the canvas is offscreen is supported

Notes

• WebGL2 is used, this reduces browser compatibility a little but allows using some niceties like vertex array objects and drawing using drawElementsInstanced.
• Render layers still work, the webgl renderer actually still utilizes the CursorRenderLayer and LinkRenderLayer in order to simplify the rendering logic (and because they're pretty fast anyway).
• When the texture atlas has reached its capacity, it will currently just clear itself and restart. This could be improved but this route was chosen due to the relatively inexpensive cost of starting over and so we don't need to juggle multiple textures.
• I experimented with disabling double buffering and drawing the minimal set of changes, it didn't appear to be worth the effort (at least given the time I had to work on this).

Issues

🛑= merge blocker

• Currently you cannot customize the selection color. This is because cached glyphs do not currently support true color, this should be trivial to fix once true color is added.
• 🛑 Character joiners are not supported yet.
• 🛑 allowTransparency is not supported yet.
• Rendering seems to become offset on my macbook when rows are increased after 300x91, I'm guessing this is related to maximum canvas dimensions?

Comparing to canvas renderer

Summary

This table shows the average time to draw the frame before and after, see below for more details.

Benchmark	Canvas/dynamic (avg ms/frame)	WebGL (avg ms/frame)	Change (c/w-1)
Macbook 87x26	4.80	0.69	596% faster
Macbook 300x80	15.28	3.69	314% faster
Windows 87x26	7.31	0.73	901% faster
Windows 300x80	19.34	2.06	839% faster
Macbook 87x26 CJK	14.63	5.93	147% faster
Macbook 87x26 Emoji	27.47	19.28	42% faster

Methodology

The tests are done by running various commands in the build xterm.js repo directory and the following measuring code is injected to get the report:

(<any>window).frames = [];
(<any>window).report = () => {
  const frames = (<any>window).frames as number[];
  const average = Math.round(frames.reduce((p, c) => p + c, 0) / frames.length * 100) / 100;
  console.log(`...: frames ${frames.length}, average ${average}ms`);
};

Wrapping _renderRows of Renderer.ts (canvas) and WebglRenderer.ts:

const startTime = performance.now();
...
(<any>window).frames.push(performance.now() - startTime);

Parts of the chrome timelines are also presented. Note that the injected code is necessary because it's difficult to measure otherwise as the number of frames and average frame time isn't captured by the timeline.

The Macbook Pro is mid 2014 with an Intel Iris Pro 1536 MB, the Windows box has a GTX 760.

Macbook Viewport 87x26 (demo default), ls -lR .

Canvas Renderer (dynamic atlas)

Canvas: frames 69, average 4.8ms

WebGL Renderer

WebGL: frames 80, average 0.69ms

Macbook Viewport 300x80, ls -lR .

Canvas Renderer (dynamic atlas)

Canvas: frames 30, average 15.28ms

WebGL Renderer

WebGL: frames 43, average 3.69ms

Windows Viewport 87x26 (demo default), tree

Canvas Renderer (dynamic atlas)

Canvas: frames 54, average 7.31ms

WebGL Renderer

WebGL: frames 60, average 0.73ms

Windows Viewport 300x80, tree

Canvas Renderer (dynamic atlas)

Canvas: frames 59, average 19.34ms

WebGL Renderer

WebGL: frames 68, average 2.06ms

Macbook Viewport 87x26, cat zh_wcag_100.txt

zh_wcag_100.txt contains this contents of https://www.w3.org/Translations/WCAG20-zh/ 100 times to help test throughput for wide characters.

Canvas Renderer (dynamic atlas)

Canvas: frames 40, average 14.63ms

WebGL Renderer

WebGL: frames 45, average 5.93ms

Macbook Viewport 87x26, cat emoji_100.txt

emoji_100.txt contains this contents of https://getemoji.com/ times to help test throughput for combined characters.

Canvas Renderer (dynamic atlas)

Canvas: frames 13, average 27.47ms

WebGL Renderer

WebGL: frames 16, average 19.28ms

---

Fixes #720
Fixes #1138 (microsoft/vscode#35901)
Fixes #1170 (I think)
Obsoletes a bunch of issues related to the canvas renderer: #941, #955, #1389, #1614

[jerch]

@Tyriar Have not yet looked at the code but could not resist to do a few tests on my own. Therefore first a few notes about the perf results.

I see about the same perf boost (rendering being 3-5x faster) on Linux with either intel graphics or nvidia GT520M (very old) and GTX 960. With my typical ls -lR /usr/lib benchmark I also hit the system throughput, means the terminal has to wait in between for the pty to deliver more data, and is idle the rest of the time (both systems with SSDs lol). That are really great results!
Your timelines show weak performance for windows systems, really slow incoming data, hmm. This might be worth to be addressed upstream in win-pty/conpty.

Your results also show a big perf decrease on the MacBook for the bigger terminal view. I wonder how fullscreen would perform. Note that I cant resemble this decrease with Linux (even tried 500x100), I see only slightly worse runtime for intel or nvidia gc. Maybe thats a resolution problem (retina?) and could be tweaked further?

What I see now for my benchmark in the demo:

• total runtime: ~1800 ms (seems this cannot go faster due to system limits)
• ~ 120 ms render time (was 500 ms before) 😍
• ~ 900 ms for the input chain (parser, buffer inserts, state handling) 😒
• ~ 200 ms JS overhead (function calls, GC etc.)
• ~ 600 ms idle 😍

Last but not least: since many peeps nowadays are on laptops - does anyone know if the gl renderer would drain more power than the canvas renderer? Note that the canvas renderer also relies on hardware acc, still unsure whether this might trigger a more costly power profile.

[Tyriar]

On support, according to https://webglstats.com/ 68% of desktops support WebGL2. I looked into downgrading to WebGL1 (98% support) but there doesn't appear to be an extension for layout. I'm thinking we should push forward with replacing the canvas renderer with the WebGL2 implementation once stable and improve the DOM renderer a little so it has more reasonable performance, that way the majority of consumers will be on the best implementation, we don't muck the code up with WebGL1 messiness (+slowness?) and there is a viable alternative for the rest.

https://caniuse.com/#search=webgl2 says that Chrome, Firefox, Opera support WebGL2.

[jerch]

@Tyriar Yeah seems only Safari cant do Webgl2, I am not sure if Safari will ever see this as Apple is pushing WebGPU based on Metal/Vulkan. Webgl1/2 is based on OpenGL which got deprecated on macOS.

[Tyriar]

Currently experimenting with minimizing the amount of data uploaded to the GPU via bufferData, results looks favorable so far, especially when scrolling.

[Tyriar]

This is ready for review/merge, it's still a work in progress as these won't work:

• Underline attribute
• Selection color
• Character joiners
• allowTransparency
• Handle context loss when there are too many contexts (see WebGL context loss handling Tyriar/xterm.js#5)
• True color
• Move CharData usage over to CellData and remove CharDataCompat.ts
• WebglAddon.dispose is not implemented

However it's a standalone addon now, the only thing that affects the core is the very small diff outside of the addons/ directory. Once merged in we'll get the build and tests in CI to prevent future breakages and we can create issues to finish off the above issues.

[jerch]

I cannot not really review this monster :P - still I have a few questions/remarks.

[Eugeny]

Could you please republish the addon with the fixed typings file?

[Tyriar]

@Eugeny done

[Eugeny]

Looks like emojis are not considered to be double width chars anymore?

[Tyriar]

@Eugeny I didn't think that ever worked because we don't detect unicode version we have to guess?

[Tyriar]

Let's do this! 🚀

I've created a bunch of issues tracking the remaining issues under the addon/webgl label. Community welcome to help finish it off 😃. If I missed something please create another issue.