wayland support - #1910
Conversation
Wraps the Spike-0 focus-bypassing wl_pointer injection in the libei/EIS stack (the Zoom/RDP protocol family): an EIS server embedded in a cua-controlled compositor (patched tinywl, wlroots 0.19) accepts a standard libei client over a direct socket (no xdg-desktop-portal, headless), exposes a virtual absolute pointer, and routes the client's emulated clicks to UNFOCUSED windows via direct pointer-resource injection. Verified: a libei client drove "button CLICKED" into an unfocused GTK4 window x8 with no portal, no real device, no cursor move, no focus change, no raise. Includes the EIS-server patch + generator, a minimal libei client, a GTK4 test target, and an idempotent provision.sh (the VM containerDisk is ephemeral). Routing is a skeleton (clicks all unfocused windows at content-center, no device->window addressing / coord mapping / enter-leave bookkeeping yet) — those are Phase B. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdds a new spike directory ChangesPhase A EIS Spike
Sequence DiagramsequenceDiagram
participant Client as ei_client (libei sender)
participant Socket as CUA_EIS_SOCKET
participant Compositor as tinywl + EIS callback
participant GTK as gtk4_test window
Client->>Socket: connect sender
Socket->>Compositor: EIS_EVENT_CLIENT_CONNECT
Compositor->>Compositor: create cua-seat + abs-pointer device (1920×1080)
Client->>Socket: EI_EVENT_BUTTON press (left)
Socket->>Compositor: dispatch button event
Compositor->>Compositor: find unfocused toplevels (skip keyboard-focused)
Compositor->>GTK: wl_pointer.enter + motion + button press/release + frame
GTK->>GTK: on_clicked / on_pressed log to stdout
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/phase_a_patch.py`:
- Around line 138-150: The repl function currently only checks that the old
string appears exactly once, but this does not prevent duplicate injections on
subsequent script runs. To make the patch generator idempotent, modify the repl
function or its usage to detect when an injection has already been applied by
checking if the post-injection state (the combined old and new content) already
exists in the source string before performing the replacement. If the combined
state is already present, skip the injection and return the source unchanged;
otherwise, proceed with the replacement as normal. This approach ensures that
each injection is applied at most once, regardless of how many times the script
is executed.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/README.md`:
- Around line 16-17: Replace all references to `LIBEI_SOCKET` with
`CUA_EIS_SOCKET` in the README documentation to maintain consistency with the
actual client/server code implementation. Update the socket environment variable
name in the diagram/examples at lines 16-17 (where `LIBEI_SOCKET` appears in the
ei_setup_backend_socket call) and at lines 26-27 to ensure all documentation
uses the single, correct variable name `CUA_EIS_SOCKET` that the code actually
reads.
- Around line 12-24: The fenced code blocks in the README.md file are missing
language identifiers after the opening triple backticks, which violates
markdownlint rule MD040. Add a language specifier (such as `text` or `bash`)
immediately after the opening triple backticks for both code blocks: the ASCII
diagram block starting at line 12 and the second fenced block at lines 32-41.
For example, change ``` to ```text at the start of each code block to properly
declare the content language.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/tinywl.eis.patch`:
- Around line 146-149: The `wl_event_loop_add_fd()` call on line 147 ignores its
return value, which can be NULL if memory allocation fails during fd
registration. Capture the return value from `wl_event_loop_add_fd()` into a
variable, check if it is NULL before logging the success message, and add
appropriate error handling (such as logging an error and exiting) if the
registration fails. This prevents the code from incorrectly reporting that the
EIS server is listening when fd registration actually failed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b727c665-1743-4883-bc60-5c48abb64a7e
📒 Files selected for processing (6)
libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/README.mdlibs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/ei_client.clibs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/gtk4_test.clibs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/phase_a_patch.pylibs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/provision.shlibs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/tinywl.eis.patch
| def repl(s, old, new, label): | ||
| if s.count(old) != 1: | ||
| print("ANCHOR FAIL [%s]: count=%d" % (label, s.count(old))); sys.exit(1) | ||
| print("ok [%s]" % label); return s.replace(old, new) | ||
|
|
||
| src = repl(src, "struct tinywl_server {", INCLUDES + "struct tinywl_server {", "includes") | ||
| src = repl(src, | ||
| 'server.seat = wlr_seat_create(server.wl_display, "seat0");', | ||
| 'server.seat = wlr_seat_create(server.wl_display, "seat0");\n\twlr_seat_set_capabilities(server.seat, WL_SEAT_CAPABILITY_POINTER);', | ||
| "force-pointer-cap") | ||
| src = repl(src, "int main(int argc, char *argv[]) {", FUNCS + "int main(int argc, char *argv[]) {", "funcs") | ||
| src = repl(src, "wl_display_run(server.wl_display);", EIS_SETUP + "wl_display_run(server.wl_display);", "eis-setup") | ||
|
|
There was a problem hiding this comment.
Patch generator is not idempotent and re-applies injections on rerun.
Current anchor checks do not prevent duplicate injection; running this script again can duplicate FUNCS/EIS_SETUP and break compilation.
Suggested fix
-def repl(s, old, new, label):
- if s.count(old) != 1:
- print("ANCHOR FAIL [%s]: count=%d" % (label, s.count(old))); sys.exit(1)
- print("ok [%s]" % label); return s.replace(old, new)
+def repl(s, old, new, label):
+ if new in s:
+ print("skip [%s]: already patched" % label)
+ return s
+ if s.count(old) != 1:
+ print("ANCHOR FAIL [%s]: count=%d" % (label, s.count(old)))
+ sys.exit(1)
+ print("ok [%s]" % label)
+ return s.replace(old, new, 1)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/phase_a_patch.py`
around lines 138 - 150, The repl function currently only checks that the old
string appears exactly once, but this does not prevent duplicate injections on
subsequent script runs. To make the patch generator idempotent, modify the repl
function or its usage to detect when an injection has already been applied by
checking if the post-injection state (the combined old and new content) already
exists in the source string before performing the replacement. If the combined
state is already present, skip the injection and return the source unchanged;
otherwise, proceed with the replacement as normal. This approach ensures that
each injection is applied at most once, regardless of how many times the script
is executed.
| ``` | ||
| cua-driver (libei client) cua's compositor (tinywl + EIS server) | ||
| ───────────────────────── ────────────────────────────────────── | ||
| ei_new_sender() eis_new() + eis_setup_backend_socket() | ||
| ei_setup_backend_socket(LIBEI_SOCKET) ───▶ EIS_EVENT_CLIENT_CONNECT → accept | ||
| ei_seat_bind_capabilities(ABS, BUTTON) ──▶ SEAT_BIND → create virtual abs-pointer | ||
| ei_device_pointer_motion_absolute() ─────▶ EIS_EVENT_POINTER_MOTION_ABSOLUTE | ||
| ei_device_button_button(BTN_LEFT) ───────▶ EIS_EVENT_BUTTON_BUTTON | ||
| └─▶ route to UNFOCUSED window: | ||
| wl_pointer_send_enter/motion/button | ||
| directly to its pointer resource | ||
| (no wlr_seat focus, no cursor move) | ||
| ``` |
There was a problem hiding this comment.
Add languages to fenced code blocks to satisfy markdownlint MD040.
Both fenced blocks should declare a language (for example text or bash) to clear the linter warning.
Also applies to: 32-41
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 12-12: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/README.md`
around lines 12 - 24, The fenced code blocks in the README.md file are missing
language identifiers after the opening triple backticks, which violates
markdownlint rule MD040. Add a language specifier (such as `text` or `bash`)
immediately after the opening triple backticks for both code blocks: the ASCII
diagram block starting at line 12 and the second fenced block at lines 32-41.
For example, change ``` to ```text at the start of each code block to properly
declare the content language.
Source: Linters/SAST tools
| ei_setup_backend_socket(LIBEI_SOCKET) ───▶ EIS_EVENT_CLIENT_CONNECT → accept | ||
| ei_seat_bind_capabilities(ABS, BUTTON) ──▶ SEAT_BIND → create virtual abs-pointer |
There was a problem hiding this comment.
Use one socket env var name consistently (CUA_EIS_SOCKET).
Docs currently mix LIBEI_SOCKET and CUA_EIS_SOCKET, but the client/server code in this spike reads CUA_EIS_SOCKET.
Also applies to: 26-27
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/README.md`
around lines 16 - 17, Replace all references to `LIBEI_SOCKET` with
`CUA_EIS_SOCKET` in the README documentation to maintain consistency with the
actual client/server code implementation. Update the socket environment variable
name in the diagram/examples at lines 16-17 (where `LIBEI_SOCKET` appears in the
ei_setup_backend_socket call) and at lines 26-27 to ensure all documentation
uses the single, correct variable name `CUA_EIS_SOCKET` that the code actually
reads.
| + struct wl_event_loop *eloop = wl_display_get_event_loop(server.wl_display); | ||
| + wl_event_loop_add_fd(eloop, eis_get_fd(g_eis), WL_EVENT_READABLE, eis_loop_cb, &server); | ||
| + wlr_log(WLR_INFO, "[PHASE-A] EIS server listening on '%s' (under XDG_RUNTIME_DIR)", eis_sock); | ||
| + } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "tinywl.eis.patch" -type fRepository: trycua/cua
Length of output: 148
🏁 Script executed:
# Also let's explore the general structure of the repository
fd -t f "tinywl" | head -20Repository: trycua/cua
Length of output: 146
🏁 Script executed:
# Look for the specific patch file
find . -path "*wayland-bg-input-phase-a*" -name "*.patch"Repository: trycua/cua
Length of output: 148
🏁 Script executed:
cat -n "./libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/tinywl.eis.patch" | sed -n '140,160p'Repository: trycua/cua
Length of output: 770
🌐 Web query:
wl_event_loop_add_fd Wayland API return value error handling
💡 Result:
The wl_event_loop_add_fd function in the Wayland server API returns a pointer to a new wl_event_source object upon success [1][2][3]. If the function fails to allocate memory for the new event source, it returns NULL [4][5]. Error handling for this function is straightforward: check the returned pointer against NULL. If it is NULL, the allocation failed, and the caller should handle the error accordingly (typically by cleaning up resources or aborting, depending on the application's needs) [4][5]. It is important to note that wl_event_loop_add_fd does not use standard errno-based error reporting in the way some other system calls do; the primary indication of failure is the returned NULL pointer [4][5]. Once a valid wl_event_source is returned, the event source will persist until it is explicitly removed using wl_event_source_remove or until the event loop is destroyed [6][3].
Citations:
- 1: https://wayland.freedesktop.org/docs/html/apc.html
- 2: https://man.archlinux.org/man/wl_event_source.3.en
- 3: https://man.archlinux.org/man/wl_event_source.3.en.txt
- 4: https://android.googlesource.com/platform/external/wayland/+/818c5d8cc721053be659c92b5772179149058830/src/event-loop.c
- 5: https://codebrowser.dev/gtk/gtk/subprojects/wayland/src/event-loop.c.html
- 6: https://context7.com/git_gitlab_freedesktop_org/wayland_wayland/llms.txt
Check the return value of wl_event_loop_add_fd() before logging success.
The wl_event_loop_add_fd() function returns a wl_event_source* pointer on success and NULL on allocation failure. The current code at line 147 ignores this return value, allowing the code to log "[PHASE-A] EIS server listening" success even when the fd registration fails. This silently breaks EIS input dispatch.
Suggested fix
- struct wl_event_loop *eloop = wl_display_get_event_loop(server.wl_display);
- wl_event_loop_add_fd(eloop, eis_get_fd(g_eis), WL_EVENT_READABLE, eis_loop_cb, &server);
- wlr_log(WLR_INFO, "[PHASE-A] EIS server listening on '%s' (under XDG_RUNTIME_DIR)", eis_sock);
+ struct wl_event_loop *eloop = wl_display_get_event_loop(server.wl_display);
+ struct wl_event_source *src =
+ wl_event_loop_add_fd(eloop, eis_get_fd(g_eis), WL_EVENT_READABLE, eis_loop_cb, &server);
+ if (!src) {
+ wlr_log(WLR_ERROR, "[PHASE-A] failed to register EIS fd in Wayland event loop");
+ } else {
+ wlr_log(WLR_INFO, "[PHASE-A] EIS server listening on '%s' (under XDG_RUNTIME_DIR)", eis_sock);
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| + struct wl_event_loop *eloop = wl_display_get_event_loop(server.wl_display); | |
| + wl_event_loop_add_fd(eloop, eis_get_fd(g_eis), WL_EVENT_READABLE, eis_loop_cb, &server); | |
| + wlr_log(WLR_INFO, "[PHASE-A] EIS server listening on '%s' (under XDG_RUNTIME_DIR)", eis_sock); | |
| + } | |
| struct wl_event_loop *eloop = wl_display_get_event_loop(server.wl_display); | |
| struct wl_event_source *src = | |
| wl_event_loop_add_fd(eloop, eis_get_fd(g_eis), WL_EVENT_READABLE, eis_loop_cb, &server); | |
| if (!src) { | |
| wlr_log(WLR_ERROR, "[PHASE-A] failed to register EIS fd in Wayland event loop"); | |
| } else { | |
| wlr_log(WLR_INFO, "[PHASE-A] EIS server listening on '%s' (under XDG_RUNTIME_DIR)", eis_sock); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/tinywl.eis.patch`
around lines 146 - 149, The `wl_event_loop_add_fd()` call on line 147 ignores
its return value, which can be NULL if memory allocation fails during fd
registration. Capture the return value from `wl_event_loop_add_fd()` into a
variable, check if it is NULL before logging the success message, and add
appropriate error handling (such as logging an error and exiting) if the
registration fails. This prevents the code from incorrectly reporting that the
EIS server is listening when fd registration actually failed.
Linux visual regression artifactsMatrix jobs now run independently. Download visual artifacts from this workflow run.
|
Evolves the EIS-server compositor to coordinate-accurate, multi-cursor, per-window routing: N virtual abs-pointer devices (CUA_NDEV=16), device i -> toplevel i, absolute coords mapped to each window's surface-local space, with per-device enter/leave bookkeeping (fixes the back-to-back-enter violation, so drags render as one continuous stroke). A cursive libei client drives 16 concurrent strokes; all 16/16 background windows receive a clean per-window path (enter=1, motion=121) with no focus, no raise, no real pointer. Includes the rendered montage and a one-shot run.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🖋️ Phase B — 16-window cursive multi-cursor demo, working end-to-endThe libei/EIS path now does coordinate-accurate, multi-cursor, per-window routing, and it drives the headline demo: 16 background windows, each with its own independent libei cursor, all tracing a cursive stroke concurrently — no window focused or raised, no real pointer moved.
End-to-end pathVerified
What's new vs the Phase A skeleton in this PR
Artifacts: Still a spike (next steps toward product)Targets are 🤖 Generated with Claude Code |
Adds real screen-capture webms of the libei/EIS background-injection demo across three use cases (16-window concurrent multicursor, 4-window independent strokes, single-window precise freehand). Enables capture by creating the wlr-screencopy + xdg-output globals in tinywl and a custom headless output mode; windows are GTK4 ink canvases (render the received stroke); recorded with wf-recorder. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🎥 Demo recordings — it works e2e, captured on videoReal screen captures ( To make this capturable I enabled
1 · 16 background windows, 16 independent cursors, concurrent cursive — scale
2 · 4 windows, each a distinct stroke — independence
3 · One large window, slow precise freehand drag — fidelity
Each window's title bar stays un-activated and the window never raises while the colored stroke is drawn — the cursor driving it is a virtual libei device, not the seat pointer. Artifacts: Honesty notes
🤖 Generated with Claude Code |
Adds a recording with a non-driven "FOREGROUND" window raised on top of the 16-window grid; the windows behind/around it keep drawing their cursive strokes, demonstrating they are driven while backgrounded/occluded. Routing now uses a stable map index (cua_win[]) instead of stack order so the raised foreground is excluded from injection; the foreground is custom-placed (CUA_FRONT_*). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🪟 Proof of "backgrounded": foreground occluder over the 16 windowsPer the ask — here's the demo with a window on top of the grid so you can see the 16 are driven while backgrounded.
The centered "FOREGROUND – on top, NOT driven" window is raised above the grid (note its title bar overlapping How it's wired: the EIS server routes by a stable per-window map index ( 🤖 Generated with Claude Code |
Adds EIS keyboard injection (xkb keymap via memfd, wl_keyboard keymap/enter/key to the target window, bypassing wlr_seat focus) on top of the pointer routing. Demo (webm): a libei-driven session types `make` into a foot terminal (compiles a real GTK4 calculator), types `./mycalc` (launches it), then clicks 7 * 6 = -> 42 — keyboard to the terminal window, pointer to the calculator window, all background injection with no real input devices and no focus stealing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🧑💻 Bonus: code an app, compile it in a terminal, run it, and use it — over libeiPushing the EIS path further: added keyboard injection, so the driver can do a full agent-style session, all through background injection (no real input devices, no focus stealing). The clip, every action injected via libei:
Keyboard events route to the terminal window, pointer events to the calculator window (two windows, routed by device index in the compositor) — neither focused or raised by a real seat. How the keyboard half worksThe compositor builds an xkb keymap (memfd), advertises a virtual keyboard device over EIS, and forwards the client's evdev keycodes as Artifacts: 🤖 Generated with Claude Code |
Since injection bypasses the seat cursor (so the user's pointer never moves), there was no on-screen cursor showing where clicks land. Adds a per-device colored cursor drawn by the compositor (wlr_scene_rect) at each injection point, raised above windows, flashing white on button press (phase_f_patch.py). The driver now glides the pointer between targets so the cursor animates. Re-recorded the app demo (cursor glides to each calc button) and the 16-window cursive (16 colored cursors) to show it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🖱️ Added a visible agent cursorGood catch — you couldn't see a cursor because the injection bypasses the seat cursor on purpose (so the user's real pointer never moves), which means there was nothing on screen showing where clicks land. Fixed by adding a per-session agent cursor: the compositor now draws a colored cursor (a Re-recorded with the cursor
Implementation: 🤖 Generated with Claude Code |
Replace the rectangle cursor with a proper arrow pointer matching cua-driver's look: a cairo-drawn left-pointer (white outline + per-session color fill, tip = hotspot) rendered as a wlr_scene_buffer via a small cairo-backed wlr_buffer impl (phase_g_patch.py). Tip aligns with the injection point; scales up briefly on click. Re-recorded the app demo and the 16-window cursive (16 colored arrows). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
➤ Agent cursor is now a real arrow pointerSwapped the placeholder rectangle for a proper arrow pointer matching cua-driver's look: a cairo-drawn left-pointer with a white outline and per-session color fill, with the tip as the hotspot (it sits exactly on the click point) and a quick scale-up on click. It's drawn into a
Shape is cua's arrow geometry ( 🤖 Generated with Claude Code |
… bloom) Previous overlay was a generic X11 left_ptr. Reworked it to reproduce cua's cursor-overlay look: the 4-point dart geometry, a 3-stop gradient fill from the real 10-entry palette (default_blue/soft_purple/...), a white outline, and a radial bloom glow behind it (expands on click) — drawn with cairo into the wlr_buffer overlay, one palette per session. phase_h_patch.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
➤ Cursor now matches cua's real agent cursorThe previous overlay was a generic X11
Still drawn into the compositor's 🤖 Generated with Claude Code |
Compositor now rotates each agent cursor to point in its direction of motion (heading from successive injection points; pre-rendered per-angle dart buffers cached per device and swapped via wlr_scene_buffer_set_buffer, 24 buckets). The driver glides along an ease-in-out cubic bezier between targets instead of a straight line. Together the dart curves and turns to follow its path, matching cua's agent-cursor motion. phase_i_patch.py + type_use.c. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
➤ Added motion-direction rotation + eased glideThe cursor now moves like cua's agent cursor:
That's the dart + gradient + bloom + per-session palette and motion rotation + eased glide — the agent cursor now matches cua's look and movement. (cua's exact Dubins arc-planner is approximated here with a bezier; visually equivalent for these paths.) 🤖 Generated with Claude Code |
The Rust workspace was bumped to 0.5.3 but nix/cua-driver/package.nix still pinned 0.5.1, so the vendored-deps Cargo.lock no longer matched cargoHash and every NixOS test failed building cua-driver with "cargoHash is out of date". Bump version to 0.5.3 and set the cargoHash recomputed against the flake's pinned nixpkgs (fetchCargoVendor of the current Cargo.lock). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…1896) Squashed import of PR #1896: native-Wayland test suite for cua-driver across labwc/sway/kwin/mutter — session.nix launcher, driver-client.nix (window discovery via list_windows, no X11 fallback), integration/screenshot/ cursor-click-gif/background-terminal-gif/parallel-drag scenarios, background-gui matrix, record-wayland-gif helper, flake checks, and the nix-wayland.yml CI workflow. (package.nix 0.5.3+cargoHash already present on this branch.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per feasibility: cua-driver as a standard Wayland client can do enumeration (foreign-toplevel), capture (screencopy) and focused input (virtual-pointer/ keyboard) on wlroots (labwc/sway) — those + background-gui stay blocking. The focus-free (background-terminal-gif) and multi-pointer (parallel-drag) scenarios, and the KDE/GNOME desktops, are infeasible on stock compositors, so they run for visibility but are continue-on-error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Don't mask the infeasible cells — keep every desktop×scenario blocking so the failures stay visible while the native-Wayland backend is implemented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First slice of native Wayland support: list_windows now enumerates toplevels via
zwlr_foreign_toplevel_manager_v1 (wlroots: labwc/sway) when running under a
Wayland session with no X11 (WAYLAND_DISPLAY set, DISPLAY unset), returning
{window_id, title:"<title> [<app_id>]"}. New src/wayland/mod.rs (pure-rust
wayland-client); list_windows dispatches Wayland-vs-X11; doctor reports the
Wayland session. Validated on headless labwc + foot: list_windows returns the
"cua-wayland-foot" toplevel. Makes the integration scenario pass on labwc/sway;
capture + input slices follow. Bumps cargoHash for the added wayland-client/
wayland-protocols-wlr deps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7bfdd60 to
95c7cd7
Compare
…--title foot's --title is immediately overwritten by the shell prompt's OSC title (e.g. "root@machine: /tmp"), so find_window could never match the launch title against a correctly-enumerating driver. app_id is stable and is one of the fields find_window already matches on, so launch foot with --app-id=<id> too. Not masking a driver gap — list_windows correctly enumerates the toplevel; this fixes the test's window-identity. (Unimplemented scenarios still fail at their capture/input step, as intended.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion) get_window_state (capture_mode=vision/som) now captures on native Wayland by shelling out to `grim` (wlroots screenshot via wlr-screencopy; works on labwc/sway) — the Wayland analog of the X11 `import`/`xwd` path. Dispatched in crate::wayland::screenshot_dispatch (Wayland when WAYLAND_DISPLAY set & no DISPLAY, else X11). foreign-toplevel exposes no per-window geometry so this captures the output, which satisfies the vision payload. Validated on headless labwc: grim -t png - returns a valid PNG. No new deps (grim is in the test session packages; wlr-screencopy needs no Rust client here). Makes the screenshot + background-gui(foot) scenarios pass on labwc/sway. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When CUA_WAYLAND_NEST is set, cua-driver spawns its own private headless wlroots compositor (labwc by default) at startup and points WAYLAND_DISPLAY at it. Every app launched via launch_app runs inside this session, so enumeration (foreign-toplevel), capture (grim), and input work without ever touching the host compositor or its focus. This is the path to supporting KDE (kwin) and GNOME (mutter), which expose no client protocols for cross-window control: cua-driver brings its own compositor instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reframe the kde/gnome cells around the nested model: cua-driver brings its own headless labwc (CUA_WAYLAND_NEST=1, injected by a per-desktop driver wrapper used as CUA_DRIVER_BIN) and every app is launched THROUGH the driver (launch_app), so it lands in the session the driver owns — the host compositor on wlroots desktops, the nested labwc on kde/gnome. This unifies the harness: the python scripts are now desktop-agnostic. - session.nix: add nested flag + driverWrapper; best-effort host bring-up for nested cells (never blocks the test on kwin/mutter); bake native Wayland toolkit backends into the wrapper env. - driver-client.nix: add launch_app helper. - record-wayland-gif.nix: self-resolve the driver's published nested socket ($XDG_RUNTIME_DIR/.cua-nested-display) when WAYLAND_DISPLAY is unset. - integration/screenshot/background-gui: launch via launch_app; run the driver through the wrapper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cursor-click slice. Wayland forbids a client from learning another window's geometry or injecting at a pixel, so input is focus-based: - click(window_id): focus+raise the target toplevel via foreign-toplevel `activate` (the protocol object we already track for list_windows) — the deterministic equivalent of clicking a window, since window-local x/y can't be mapped to a global pointer position. - type_text / press_key: shell out to `wtype` (zwp_virtual_keyboard_v1; it builds the xkb keymap + resolves shift levels for us) to type into the focused surface. Mirrors the capture slice's shell-out to grim — no new Rust deps, no cargoHash change. Keys are pid/window-free (Wayland delivers to the focused surface); pair with click/activate to focus the intended window. Wired in impl_.rs ClickTool/TypeTextTool/PressKeyTool behind is_wayland(). cursor-click-gif.nix reframed to the unified model (launch via launch_app + driver wrapper); wtype added to the session packages. Validated e2e on the VM in nested mode: launch control+target foot, click(target)→activate, type 'echo click-focus > …'→Return, file written with 'click-focus'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sway's headless seat has no keyboard until a pointer interaction occurs, so
activate-alone (which works on labwc) left virtual-keyboard input unrouted on
sway. Two fixes make all four desktops green:
- click() now lands a real virtual-pointer button press at the output centre
(zwlr_virtual_pointer_v1 + wl_output for extents) after foreign-toplevel
activate. The pointer interaction wakes sway's keyboard routing; centre
targeting is reliable because the activated window is centred (labwc) or
fills the output (sway, via the new workspace_layout stacking). Harmless on
labwc, where activate alone already routed.
- type_text() leads with a no-op Shift_L tap: the first virtual-keyboard event
on a freshly-focused headless seat is dropped while routing wires up, so the
modifier tap absorbs the drop and the real text lands intact.
Validated e2e: host sway + nested labwc both type the full command
('click-focus') into the target terminal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… input The Phase-C own-compositor path for the two inputs stock Wayland forbids a client from doing: focus-FREE per-surface keyboard injection (type into an unfocused window) and MULTI-cursor pointer injection (N independent cursors on one window). - nix/cua-driver/compositor/cua_compositor_patch.py turns wlroots' upstream tinywl (0.19) into cua-compositor: a headless wlroots compositor exposing xdg-shell + foreign-toplevel (list_windows) + screencopy (grim) + a unix control socket ($CUA_INJECT_SOCKET) speaking a tiny line protocol. It routes injection to a target window by xdg app_id, writing wl_keyboard/wl_pointer straight to the client's resources — no seat focus, no real cursor. Keyboard uses an xkb keymap + a char->keycode table built from it (handles shift). - wayland/mod.rs: an inject client (is_inject_mode/app_id_for_window/ inject_type_text/inject_press_key/inject_parallel_drags) that speaks the control protocol; window_id -> app_id via foreign-toplevel enumeration. - impl_.rs: type_text / press_key / parallel_mouse_drag route through the inject socket when CUA_INJECT_SOCKET is set (skipping the X11 MPX blocker), else the existing wtype/virtual-pointer paths. Validated e2e on the VM: cua-driver drives cua-compositor to type 'echo hello | tee …' into an UNFOCUSED foot (file written) and run two concurrent drags. Nix packaging + test wiring follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…itor Nix-package cua-compositor (built from wlroots_0_19's tinywl + the patcher) and add an `eis` mode to session.nix: the driver wrapper nests cua-compositor (CUA_WAYLAND_NEST_COMPOSITOR + CUA_INJECT_SOCKET) on any host, and apps launch through cua-driver into it. Reframe the two cells to the unified launch_app model: - background-terminal-gif: launch target then control (target left unfocused), type focus-free into the target, assert the command ran (file side-effect). - parallel-drag: launch one window, run two concurrent drags through parallel_mouse_drag (multi-cursor via the control socket). The compositor builds against the same wlroots_0_19 the patch targets, so it tracks the nixpkgs wlroots API. GIF recorder self-resolves the nested socket. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a native-Wayland NixOS test matrix for cua-driver (covering multiple compositors and scenarios) and wires initial Wayland dispatch paths into the Linux tool implementations (Wayland window enumeration, screenshot capture via grim, focus/activate click semantics, and EIS/nested-compositor injection helpers).
Changes:
- Introduces a Wayland session bring-up layer plus multiple Wayland-first NixOS test scenarios (integration, screenshot, cursor-click GIF, background-terminal GIF, parallel drag, background GUI).
- Adds a minimal headless wlroots “cua-compositor” package and supporting patch generator to enable focus-free typing and multi-cursor injection for EIS-mode tests.
- Updates
platform-linuxtools to dispatch to Wayland paths where appropriate and adds Wayland-related Rust dependencies.
Reviewed changes
Copilot reviewed 42 out of 55 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| nix/cua-driver/tests/wayland/session.nix | Shared Wayland session bootstrap (native vs nested vs EIS) and driver wrapper env setup |
| nix/cua-driver/tests/wayland/screenshot.nix | Wayland screenshot TDD test scenario |
| nix/cua-driver/tests/wayland/record-wayland-gif.nix | Best-effort grim frame capture + GIF stitch helper |
| nix/cua-driver/tests/wayland/README.md | Suite documentation and matrix overview |
| nix/cua-driver/tests/wayland/parallel-drag.nix | Multi-cursor drag scenario (EIS-mode) |
| nix/cua-driver/tests/wayland/integration.nix | Wayland integration test asserting enumeration works |
| nix/cua-driver/tests/wayland/driver-client.nix | Shared Python MCP client harness for the Wayland tests |
| nix/cua-driver/tests/wayland/cursor-click-gif.nix | Click + type + GIF capture scenario |
| nix/cua-driver/tests/wayland/background-terminal-gif.nix | Focus-free typing into unfocused window (EIS-mode) |
| nix/cua-driver/tests/wayland/background-gui.nix | Background GUI “read-only” capture skeleton per toolkit |
| nix/cua-driver/package.nix | Bumps cua-driver version + cargoHash for dependency changes |
| nix/cua-driver/compositor/default.nix | Adds cua-compositor (patched tinywl build) derivation |
| nix/cua-driver/compositor/cua_compositor_patch.py | Generator that transforms wlroots tinywl into cua-compositor with control socket + protocols |
| libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs | Wayland dispatch for list_windows/screenshot/click/type/press_key + inject-mode parallel drag + doctor output |
| libs/cua-driver/rust/crates/platform-linux/src/lib.rs | Ensures nested Wayland session setup before registering tools |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/videos/README.md | Spike artifacts documentation |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/tinywl.eis-multicursor.patch | Phase B multicursor EIS patch (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/run.sh | Phase B one-shot build/run script (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/render_demo.py | Phase B montage renderer (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/README.md | Phase B spike documentation |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/phase_b_patch.py | Phase B patch generator (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-b/ei_client_b.c | Phase B libei client (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/tinywl.eis.patch | Phase A EIS patch (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/README.md | Phase A spike documentation |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/provision.sh | Phase A provisioning script (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/phase_a_patch.py | Phase A patch generator (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/gtk4_test.c | Phase A GTK4 target app (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-phase-a/ei_client.c | Phase A libei client (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/type_use.c | App demo driver (libei client) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/run_termdemo.sh | App demo orchestration script |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/README.md | App demo documentation |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/phase_i_patch.py | App demo patch generator variant (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/phase_h_patch.py | App demo patch generator variant (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/phase_g_patch.py | App demo patch generator variant (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/phase_f_patch.py | App demo patch generator variant (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/phase_e_patch.py | App demo patch generator variant (spike) |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/Makefile | App demo build rule for calculator |
| libs/cua-driver/rust/crates/platform-linux/spikes/wayland-bg-input-app-demo/calc.c | GTK4 calculator demo app exporting button coordinates |
| libs/cua-driver/rust/crates/platform-linux/Cargo.toml | Adds Wayland client + wlr protocols dependencies |
| libs/cua-driver/rust/Cargo.lock | Locks new Wayland dependencies |
| flake.nix | Adds Wayland check matrix to flake checks |
| .github/workflows/nix-wayland.yml | Adds CI workflow to run the Wayland matrix and upload artifacts |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| th = threading.Thread(target=reader) | ||
| th.start() | ||
| th.join(timeout) | ||
| if th.is_alive(): | ||
| raise TimeoutError("no response from driver within timeout") |
| # Native model: block until the HOST compositor's socket appears, publish it. | ||
| nativeStart = '' | ||
| dbus-run-session -- sh -c ' | ||
| wlsock() { ls "$XDG_RUNTIME_DIR" 2>/dev/null | grep -E "^wayland-[0-9]+$" | head -1; } |
| # record the host socket (if any) for logs; the driver wrapper ignores it. | ||
| nestedStart = '' | ||
| dbus-run-session -- sh -c ' | ||
| wlsock() { ls "$XDG_RUNTIME_DIR" 2>/dev/null | grep -E "^wayland-[0-9]+$" | head -1; } |
| ) | ||
| # Native-Wayland TDD matrix — reproduce the cua-driver scenarios on | ||
| # real, NATIVE Wayland sessions (XFCE on labwc/wayfire/sway, plus KDE | ||
| # and GNOME). Apps run as Wayland clients and the tests never set | ||
| # DISPLAY, so the X11-only driver cannot see them: this is a RED suite | ||
| # specifying native Wayland support. One check per (desktop × scenario) | ||
| # and per (desktop × background-GUI app). See | ||
| # nix/cua-driver/tests/wayland/README.md. | ||
| // pkgs.lib.optionalAttrs (system == "x86_64-linux") ( | ||
| let |
| - name: Run cua-driver-wayland-${{ matrix.desktop }}-${{ matrix.scenario }} | ||
| run: | | ||
| nix build ".#checks.x86_64-linux.cua-driver-wayland-${{ matrix.desktop }}-${{ matrix.scenario }}" \ | ||
| --print-build-logs --show-trace \ | ||
| -o "result-${{ matrix.desktop }}-${{ matrix.scenario }}" | ||
|
|
| - name: Run cua-driver-wayland-${{ matrix.desktop }}-background-gui-${{ matrix.app }} | ||
| run: | | ||
| nix build ".#checks.x86_64-linux.cua-driver-wayland-${{ matrix.desktop }}-background-gui-${{ matrix.app }}" \ | ||
| --print-build-logs --show-trace \ | ||
| -o "result-${{ matrix.desktop }}-bg-gui-${{ matrix.app }}" | ||
|
|
| async fn invoke(&self, args: Value) -> ToolResult { | ||
| use cua_driver_core::tool_args::ArgsExt; | ||
| let filter_pid = args.opt_u64("pid").map(|v| v as u32); | ||
| let windows = tokio::task::spawn_blocking(move || crate::x11::list_windows(filter_pid)).await.unwrap_or_default(); | ||
| let windows = tokio::task::spawn_blocking(move || crate::wayland::list_windows_dispatch(filter_pid)).await.unwrap_or_default(); | ||
| let mut lines = vec![format!("Found {} windows:", windows.len())]; |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
A review autofix set LAUNCH = "${cfg.env} ${cfg.launch}" for the background-gui
cells, but launch_app execs the command's FIRST whitespace token as the program
— so "GDK_BACKEND=wayland gedit" tries to exec a binary literally named
"GDK_BACKEND=wayland" and fails (gtk3-gedit + qt6-kcalc cells went red; foot,
with no env, stayed green). The native Wayland backends are already exported by
the session driver wrapper (appBackendEnv), so the per-app env was redundant:
drop the dead `env` field and launch the bare command.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
r33drichards
changed the title
Remove libs/cua-driver/rust/crates/platform-linux/spikes/ — 3.3 MB of non-functional research artifacts (proof-of-concept webm recordings, montage PNGs, throwaway tinywl patches + C test programs from Phases A/B/demo). The exploration they captured is now realized in the shipped product (nix/cua-driver/compositor/ + the platform-linux Wayland backend); nothing builds or runs from spikes/. Reworded the one comment that referenced the path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Reviewed this end-to-end — nice work, and the #1909 spike result (GTK4/Qt6/Chromium all accept backgrounded per-window injection) is the important de-risk. A few things already make this safe to land:
One ask before merge, then you're clear to land it and keep iterating on That way pure-Wayland sessions keep returning the unsupported message by default until Phase A is actually done, released binaries behave identically to 0.5.5 for every real-world session, devs/CI flip the flag to drive the TDD, and we avoid a long-lived branch. Keep LINUX.md / linux.mdx saying native Wayland is unsupported until the flag defaults on. With that gate in, merge away — this is the right shape. 🙏 |
|
Took this over per our chat — thanks @r33drichards, merged. ✅ On the X11 backwards-compat you flagged: verified it holds by construction. Every dispatch point falls through to the existing X11 path when not pure-Wayland:
The guards — Two small non-blocking follow-ups:
|
… flag (#1935) * feat(cua-driver-rs)(linux): gate the native-Wayland backend behind an opt-in flag The native-Wayland backend (#1910) was default-on: `is_wayland()` engaged it for any pure-Wayland session (WAYLAND_DISPLAY set, DISPLAY unset). But the backend is incomplete (toplevel enumeration + virtual-pointer/keyboard input; capture and full AT-SPI parity still landing), so silently driving it on every Wayland session is the wrong default. Make it opt-in: `is_wayland()` now also requires `CUA_DRIVER_RS_ENABLE_WAYLAND` (any value other than empty/0/false). Off by default, a pure-Wayland session is treated as unsupported instead of quietly using a half-complete path; set the flag to try the experimental backend. XWayland sessions are unaffected (they already have DISPLAY set, so is_wayland() was already false for them). `doctor` reflects the state (native Wayland session — backend ENABLED vs OFF with the flag to set), and the structured output gains `wayland_enabled`. Docs: linux.mdx gains an "Experimental native Wayland" section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(cua-driver-rs)(nix): opt into Wayland backend in the wayland session wrapper The native-Wayland backend is now opt-in (CUA_DRIVER_RS_ENABLE_WAYLAND). Every Wayland NixOS test (xfce-labwc/xfce-sway/kde/gnome x all scenarios) drives the shared session.nix driverWrapper, so set the flag once in appBackendEnv — these tests exist specifically to exercise the backend, which would otherwise be off. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…nifest typo - Add libwayland-client0 (deb) and libwayland-client (rpm) to pkg_install because the CD workflow links against libwayland-dev (native Wayland backend, added in #1910); without the runtime lib the dynamic linker fails on any distro that doesn't install it by default. - Fix typo: 'manifst' -> 'manifest' in doctor smoke-test comment. - Update comment block to mention Wayland alongside X11 deps.
* ci(cua-driver): add distro-compat smoke-test workflow with X11 runtime deps Add .github/workflows/ci-distro-compat-cua-driver.yml from PR #1959 (f/cua-599-nixos-cua-driver-test-suite-has-blind-spots-for-real-distro) with a fix for the X11 runtime dependency issue. The workflow smoke-tests the released cua-driver-rs binary across 5 distros (debian:12, ubuntu:22.04, ubuntu:24.04, rockylinux:9, fedora:41) to catch glibc ABI floor regressions. **Fix from original PR:** The pkg_install matrix entries now include the required X11 runtime libs (libx11-6 libxi6 libxtst6 libxext6 on Debian/Ubuntu; libX11 libXi libXtst libXext on Rocky/Fedora). Without these, the cua-driver binary fails with exit 127 (dynamic linker cannot resolve libXi.so.6 etc.) before main() even runs -- which the smoke-test correctly interprets as an ABI failure. The cua-driver-rs-v0.5.8 release (published 2026-06-21) provides the binary assets this workflow downloads. The binary URL pattern matches: https://github.com/trycua/cua/releases/download/cua-driver-rs-v{VERSION}/ cua-driver-rs-{VERSION}-linux-x86_64-binary.tar.gz Fixes: CUA-606 Related: CUA-599, PR #1959 * ci(cua-driver): fix rocky linux curl conflict in distro-compat workflow Rocky Linux 9 ships curl-minimal in the base image which conflicts with the full curl package. Use --allowerasing to let dnf replace curl-minimal with the full curl package transparently. * review: add libwayland-client0/libwayland-client runtime dep + fix manifest typo - Add libwayland-client0 (deb) and libwayland-client (rpm) to pkg_install because the CD workflow links against libwayland-dev (native Wayland backend, added in #1910); without the runtime lib the dynamic linker fails on any distro that doesn't install it by default. - Fix typo: 'manifst' -> 'manifest' in doctor smoke-test comment. - Update comment block to mention Wayland alongside X11 deps. --------- Co-authored-by: OpenClaw <openclaw@trycua.com>
…on pure Wayland (#1921) (#1994) On a pure-Wayland session (WAYLAND_DISPLAY set, no DISPLAY) with the native Wayland backend not opted in, every input tool routed to the X11 XSendEvent path, which cannot deliver events without an X server — yet the failure was a cryptic x11rb connect error at best and a silent false-success at worst. An agent then believes its clicks/keys landed when nothing happened. - Add wayland::wayland_input_unavailable_reason(): Some(actionable msg) only on a pure Wayland session with the backend off; None for X11/XWayland. - Route all 10 X11 input connect sites through connect_x11_for_input(), which bails with that reason instead of attempting X11, and otherwise surfaces DISPLAY on any connect failure so the cause is diagnosable. Addresses the issue's 'fail loudly rather than fall through to the X11 no-op and report success' direction. Wiring native zwlr_virtual_pointer as the default remains the larger #1910 follow-up. Claude-Session: https://claude.ai/code/session_01KMXCW4M5uK1HRGjjH4wueZ Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This pull request introduces a comprehensive native Wayland test suite for the
cua-driverproject, focused on enabling and validating support for Wayland compositors in addition to the existing X11-based tests. It also adds a new demo for background input injection via Wayland, including documentation and a sample GTK4 calculator application. The changes are grouped into three main areas: CI/test infrastructure, Nix flake integration, and demo spike additions.CI/Test Infrastructure:
.github/workflows/nix-wayland.yml) that runs a matrix of native Wayland TDD tests across multiple desktops (XFCE/labwc, XFCE/sway, KDE, GNOME) and scenarios, with artifact collection and Nix cache integration. This suite is expected to fail until native Wayland support is fully implemented, enforcing a "red" TDD workflow.Nix Flake Integration:
flake.nixchecks to generate one Nix check per (Wayland desktop × scenario) and per (Wayland desktop × background GUI app), mirroring the new CI matrix. This covers integration, screenshot, cursor click, and other scenarios, as well as background GUI app tests.Demo Spike Additions:
wayland-client,wayland-protocols-wlr) to the Rust platform crate for native Wayland backend support, laying the groundwork for pure-Rust Wayland client functionality.spikes/wayland-bg-input-app-demo:Makefilefor building a sample GTK4 calculator app.README.md, explaining end-to-end keyboard and pointer injection via libei/EIS, and how the demo orchestrates compiling and interacting with the calculator app using only injected input.calc.c, a real GTK4 calculator app that exports button coordinates for automated input and supports headless operation for CI/testing.