Proposal: implement rendering through HTML elements

[romgrk]

Hi there :)

I was wondering if you would consider switching the rendering implementation to HTML elements rather than a canvas?
I think this would be better because if there is one thing that browsers are optimized for, it's text rendering. Using a canvas does give good results but is still not as performant as what the browser can do (IMO), e.g. font aliasing (turning on optmizeLegibility in CSS)
Besides that, it also gives a whole lot more possibilities for future widgets, where one could use the full HTML specs (mouse hover events, etc.) to build on top of this.

If it can convince you, I did a prototype (it was for testing the external popup for neovim) which gives pretty good results: https://www.youtube.com/watch?v=TI5azVeDUDo

[rhysd]

I implemented NyaoVim with React.js and DOM at first. However, it causes big performance problem (e.g. scrolling screen causes entire re-rendering). I know that Atom and VS Code (monaco editor) use DOM for editor surface. Atom did so many rendering optimizations to improve performance but I don't have such a skill.

Additionally, rendering events from Neovim don't get along with DOM. Rendering events assumes to render an editor on 2D screen. For example, va is on buffer with javascript filetype and when I enter r, the buffer will get var with highlight. In the case, Neovim sends 'move cursor just before va', 'write var with highlight'. If we use DOM, we need to modify DOM tree with this 2D rendering events. Some events may require to split one non-highlighted text into highlighted part and non-highlighted part. It's very complicated.

So I introduced <canvas>. It renders screen with GPU so high performance. Now RPC is a bottle neck.

Anyway, you can try using DOM. I guess adding new screen renderer in addition to src/neovim/screen.ts is good way (e.g. dom-screen.ts).

[romgrk]

I indeed ran with the same exact problem with React, scrolling was a real issue. I found that replacing the whole React thing with a custom DOM manipulation interface much more efficient.
I also ran in the other problem that Neovim renders thing like if it was talking to a terminal, and has no other screen model whatsoever.

For both problems, I found implementing a LineModel helped a lot (see https://gist.github.com/romgrk/7b155dcc273d46f125c22fe5de2b2c4f).
This is a representation of a line. It is basically a list of tokens, where each token is { text: string, attr: string }. It exposes methods like .insert(position, token) where position is the character where you need to insert the token, and it handles the various problematic cases: insert a token in the middle of another token, insert a token at a position where there is no token yet (filling the voids with tokens containing spaces and no style attributes). The only missing optimization is merging tokens that are adjacent and have the same attr property. All the uggly details like finding where tokens end/start and splitting said tokens into parts is hidden from an external POV.
Having this model allows for fast rendering, because it essentially becomes a loop over the tokens, eg:

const renderToken = token => `<span style=${token.attr}>${token.text}</span>`;
const lineHTML = tokens.reduce((accumulator, current) => accumulator += renderToken(current), "");

Also, it means that we could also have an actual model of what's on the screen, which isn't possible with a canvas. This allows to not re-render the whole line but just the elements that have changed. (Much like React's VirtualDOM). It also means that we do not need to display the elements immediately when the neovim event comes in, and throttle them. (I found 5ms to be a correct latency)

I did implement a POC replacement for src/neovim/screen.ts here: https://gist.github.com/romgrk/01a74a06eb6f6ee26baf97b8c07c47bf. I'll try to make a branch so you can check by yourself.

On the scrolling issue, again the models handles the uggly cases by providing an .extract(position: number, size:number, items?: [Tokens]): [Tokens] method, extracting size characters starting from position, optionally inserting items. With it we can loop over a region, extracting tokens and inserting them above/below.

[ghost]

I was thinking about this, too, and I came to the realisation that a <canvas> element fits the problem much more nicely, because the Neovim RCP UI API does not know the concept of HTML elements. Above that, the canvas is just a screen buffer underneath, without concepts like padding, margin and so on. If I'm correct, and as @rhysd says, this still should be faster than rendering each line as a HTML element.