Add X11 input with XTEST for headless input commands

[ehfd]

CC @ayunami2000 @galister

There are situations where uinput/evdev for keyboard and mouse devices are not available.

The most representative situations are when the deployment is done within an environment where kernel modules are not accessible, which require inputs to be redirected directly to the X11 server (can be virtual X11 servers or a full-fledged Xorg server).

One example is when using containers, and since multiple container instances may be deployed in a single node, kernel modules must not be provisioned to provide isolation and the /dev/input directory must be isolated within the container.

Communications:

Me: Yes, more in focus, we are trying to run in Kubernetes clusters (academic/educational or commercial)
Kubernetes is vastly cheaper and easier to produce reproducible environments for people than VMs
So the process is to remove the pressure vessel layer and use LD_LIBRARY_PATH instead to access Sniper and Soldier runtimes. We have succeeded in doing that themselves, but the supported Keyboard and Mouse input is with X11 Xlib (libxinput and libxtest).

i'm not sure what you can do about uinput. the whole point of wlx is to emulate the actual mouse and keyboard so that it works with all apps.
you can probably write your own input provider using something like XTEST if you really want
the interface is pretty simple

the very old predecessor of wlx which was called x11overlay was using an x11 library called xtest
it allows arbitrary mouse and keyboard faking even with no privs
but obviously only works if you're on x11

let me find this for you. if you want to implement this into wlx and hide it behind a --toggle then i'll happily merge it

keyboard part:
https://github.com/galister/WlxOverlay/blob/bcecf1542b2bc5de4fe7caa22704b107dcd2586e/Desktop/X11/X11Keyboard.cs#L21

mouse part (coords are local to screen):
https://github.com/galister/WlxOverlay/blob/a5e5016586c2d595954ee13eda893344d83a4593/Desktop/X11/XScreenCapture.cs#L72-L79

helper library (raw xcb/xtest):
https://github.com/galister/xshm-cap/blob/4dbf29b9ca74a64e5df7cfeff59a8ee82702cf44/xshm-cap.c#L234-L254

but in the end we ended up using uinput for X11 as well because xtest would not play well with all apps
still, probably better than nothing

so on wlx-s, you'd have to implement the HidProvider trait and then decide in hid::initialize() which implementation is to be instantiated
probably that USE_UINPUT atomicbool could be changed to something that allows multi-option

set_desktop_origin and set_desktop_extent get called after discovering screens.
origin is the top-left coordinate of the desktop rect (always 0;0 on x11 afaik)
extent is the width and height of the desktop rect that encapsulates all monitors
also the wlx-s HidProvider uses global coords for mouse_move

But in the end we ended up using uinput for X11 as well because xtest would not play well with all apps
i honestly can't remember what the issue was

Me: I think this would be with relative mouse movement. Please assess https://github.com/selkies-project/selkies-gstreamer/pull/97/files.

[mitsuami-megane]

For Wayland this can be done via extensions, only problem is that they're all over the place.

• KDE publishes org_kde_kwin_fake_input for mouse + keyboard, but there's no info about the fact that they're actually implementing it.
• just like we'd expect, GNOME doesn't implement anything
• everything other than KDE & GNOME implements zwp_virtual_keyboard_v1, but this is keyboard-only
• wlroots & hypr implements wlr-virtual-pointer-unstable, but this is mouse-only

[ehfd]

@mitsuami-megane Maybe kind of input could be available through xdg-desktop-portal, perhaps (for KDE KWin and GNOME Mutter)?

https://github.com/nestrilabs/nestri
Here, Wayland is used in a containerization approach.