Rewrite app-endpoint-server in a native language

[Zubnix]

Currently the app-endpoint-server implementation uses nodejs. This is and was fine for prototyping and just getting things working quickly. It is however a bit of a PITA to setup and has some not to be underestimated performance penalties.

Ideally we want to a native language that can output a single static binary easily, and can inter-operate with C libraries seamingly. It needs to provide a C function pointer from a function implemented in the source language (this is required for use with libwayland). There is also the requirement to easily operate with gstreamer.

Candidates like Rust or Go come to mind, however good old C is fine as well.

[WSLUser]

Why not use C++ and make use of https://github.com/microsoft/STL / libc++ and https://github.com/microsoft/GSL and https://github.com/duneroadrunner/SaferCPlusPlus/blob/master/README.md

[Zubnix]

@WSLUser For now the use-case is to have the binary run on *nix so an MS based library would probably be a bad choice. A future use-case however, is to have a binary that bridges between the Win32 Display API and Wayland API (much like already exists between X11 & Wayland). In that case, using MS libraries would be an obvious choice.

[WSLUser]

I mentioned libc ++ which is Linux....STL is compatible license wise and supporting both should be relatively easy. I can't say it's a "write once and done" but it's near enough that there shouldn't be too many differences. Also Visual Studio Community can help as it supports both libc ++ as well as their own STL if you have Windows 10 (Windows 7 is now EOL and 8 should be skipped).

[trusktr]

What about if instead of serving anything, forking the browser code (f.e. Electron or NW.js as examples), and then instead of using network APIs send window pixel buffer memory pointers directly into the browser engine, perhaps into canvas-like element instances? These elements would be coded in native source and would grab the pixel data with the pointers, and the elements would be available for regular CSS styling on the DOM side (desktop "ricing" as unixporn calls it). Maybe these elements would be called <x-window>, or something.

Or is the goal here specifically to have a network-based solution that works in regular browser sandboxes?

Even if the server is ported away from Node.js, I'm sure that won't matter compared to network latency.

I saw in one of the youtube presentation that it is using libxkbd compiled to WebAssembly. Isn't this a bit heavy too?

The way I imagine it, the browser engine would do only what it does best: composite DOM elements on screen using HTML/CSS. Then keyboard and mouse events would be packaged up and sent to the native side for handling instead, passed to Wayland ot Xwayland.

The browser would do nothing more than it already does, merely exposing the pixels of X/Wayland windows in a special <x-window> element, while everything else like window borders, desktop features, etc, would be coded in plain HTML/CSS/JS/WebGL.

EDIT: Another idea is, instead of an <x-window> element or similar, to have an option that passes the pixels into a JS ArrayBuffer, then have a WebGL program handle the compositing.

[Zubnix]

Hi @trusktr, the original idea was to move away from nodejs, so people could simple use a single static binary with a config file to make the server side work, instead of having to mess around with nodejs. The performance improvements would be, like you said, negligible indeed.

Current greenfield implementation as it lives on branch: https://github.com/udevbe/greenfield/tree/modularize uses a single canvas with webgl to composite the entire scene (so no more canvas per surface). The main reason is performance caused by the fact that HTML canvas operations are not atomic, so there was a need for all kind of double buffering and funky workaround to ensure atomic scene updates.

If you want to port greenfield + greenfield-webshell (like it exists here: https://greenfield-preview.web.app/ ) to a browserless environment, then NW.js is the only viable candidate. the only challenge would be to import native opengl(es) texture handles as a webgl texture. Alternatively (but slower), you could settle on simply using shared-memory and upload it to a webgl texture. The native xkb-common library can also indeed by used as you said.

If you want to be minimalistic you could port greenfield without a user shell to node.js as greenfield itself has very little DOM dependencies (just input events + webgl). You would need to use a webgl polyfill, or write your own scene renderer from scratch, and find a native alternative for input events.

[trusktr]

HTML canvas operations are not atomic, so there was a need for all kind of double buffering and funky workaround to ensure atomic scene updates.

Just curious, what does it mean that the canvas operations are "not atomic"?

[Zubnix]

Well they are individually, but you can't group a set of operations and commit them atomically. A canvas resize for example, first clears the whole content to white and then resizes. So first you get a white flash, then a few ms later you see the new contents, This gives a bad user experience.

[markusbkk]

Personally, Go would be my suggestion.
It makes writing client-server communication a bliss, the setup is straightforward and there's already a well maintained WebTransport library