ci(cua-driver): automated lane for the interactive modality #[ignore] suite - #2060
Conversation
… suite The harness/modality integration tests are `#[ignore]` — they need a display, the AT-SPI bus, and the GTK3 harness, so the rest of CI only COMPILES them and their behavioral signal came from manual VM/host runs. This adds a Linux job that actually RUNS them, so behavior is guarded on every cua-driver change: - modality_capture_mode_test (ax→tree-only / vision→image-only / som→both) - modality_desktop_scope_linux_test (window-less screen-absolute click + gate) Session is provided headlessly: Xvfb for the display, a per-job D-Bus session bus so AT-SPI's `org.a11y.Bus` auto-activates, at-spi2-core for the bridge, and the GTK3/PyGObject runtime for the harness. The tests skip-with-note when the a11y bus is unavailable (rather than false-failing), so a vacuous run is visible without breaking the build — and this PR's own run is the experiment that tells us whether headless AT-SPI is viable here or whether this coverage must live in the Nix desktop sessions (nix-wayland.yml) instead. Runs on free GitHub runners, path-filtered to the Rust driver + harness, plus push-to-main and manual dispatch. Companion to — not a replacement for — nix-build.yml / nix-wayland.yml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdds a GitHub Actions job for the Linux interactive modality harness. It installs GUI/accessibility dependencies, builds the Rust driver and GTK3 harness, runs the interactive tests under Xvfb and D-Bus, and uploads MP4 and log artifacts. ChangesInteractive Linux CI workflow
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-cua-driver-interactive-linux.yml:
- Around line 58-59: The Rust toolchain step is using a floating action
reference, so update the dtolnay/rust-toolchain usage in the workflow to an
immutable commit SHA like the other third-party actions. Keep the step name and
behavior the same, but replace the `@stable` reference in the Rust toolchain
action with a pinned SHA so the workflow stays deterministic.
- Around line 46-47: The Checkout step in this workflow leaves repository
credentials available to later commands, which is unnecessary for this job.
Update the actions/checkout usage to disable persisted credentials by setting
persist-credentials to false, keeping the existing Checkout step and its
surrounding build/test steps unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 5be44d6f-4230-4e8f-98bb-41599476be35
📒 Files selected for processing (1)
.github/workflows/ci-cua-driver-interactive-linux.yml
| - name: Checkout | ||
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Disable persisted checkout credentials.
This job runs repository-controlled code after checkout (bash libs/cua-driver/test-harness/build/linux.sh, cargo build, cargo test), so leaving the workflow token in the local git config widens the blast radius if those steps are compromised. Set persist-credentials: false on actions/checkout.
Suggested change
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+ with:
+ persist-credentials: false📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - name: Checkout | |
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | |
| - name: Checkout | |
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | |
| with: | |
| persist-credentials: false |
🧰 Tools
🪛 zizmor (1.26.1)
[warning] 46-47: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-cua-driver-interactive-linux.yml around lines 46 - 47,
The Checkout step in this workflow leaves repository credentials available to
later commands, which is unnecessary for this job. Update the actions/checkout
usage to disable persisted credentials by setting persist-credentials to false,
keeping the existing Checkout step and its surrounding build/test steps
unchanged.
| - name: Rust toolchain | ||
| uses: dtolnay/rust-toolchain@stable |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Pin dtolnay/rust-toolchain to an immutable commit SHA.
@stable is a floating ref, so this workflow can change behavior outside review. Pin it the same way as the other third-party actions in this file.
🧰 Tools
🪛 zizmor (1.26.1)
[error] 59-59: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-cua-driver-interactive-linux.yml around lines 58 - 59,
The Rust toolchain step is using a floating action reference, so update the
dtolnay/rust-toolchain usage in the workflow to an immutable commit SHA like the
other third-party actions. Keep the step name and behavior the same, but replace
the `@stable` reference in the Rust toolchain action with a pinned SHA so the
workflow stays deterministic.
…se race The Linux video run surfaced a cold-start race: the test clicks within ~1.5s of launching the GTK3 harness, before some WMs (XFCE/Openbox) raise the freshly-mapped window under the pointer — so the screen-absolute XTest click misses the first attempt and the counter stays 0 (`pre=0 post=0`). The feature is correct (recorded landing the click + the counter climbing 0→1→2→3); only the test was too eager, and its retry loop re-fetched the button frame but not the click. Retry the window-less click until the counter advances (12×500ms). Re-issuing the same click is safe — extra landed clicks only increment further, and `post > pre` still holds. Still asserts the first attempt was dispatched as desktop scope, and that it lands within the budget. Also makes this test reliable under the new headless interactive CI lane (Xvfb window-raise timing). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Bundled a robustness fix surfaced by the Linux video run: |
Linux visual regression artifactsMatrix jobs now run independently. Download visual artifacts from this workflow run.
|
First run proved headless AT-SPI works — the capture_mode matrix passed with real assertions (ax→tree-only / vision→image-only / som→both). The desktop-scope test failed because bare Xvfb has no window manager, so the freshly-mapped GTK3 harness window is never raised/focused and the screen-absolute XTest click can't land (button at (62,56), counter stayed 0 through all retries). Launch openbox (the same WM the Nix tests use) under Xvfb before the suite so the window is managed; the capture_mode tests are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two CI runs proved the split: headless Xvfb+AT-SPI runs the capture_mode matrix (ax/vision/som) with real assertions, and the desktop-scope GATE (desktop_scope_disabled). The desktop-scope *landing* (counter advances on a window-less screen-absolute click) needs a real display + WM to deliver the XTest click — even with openbox, Xvfb doesn't faithfully deliver it (the click never lands, button at a sensible 812,343). That assertion is covered on real desktops (the Azure GNOME VM run + the recorded mp4) and is a follow-up for a real-session Nix lane. Scope this lane to the headless-reliable subset so it is green + meaningful rather than red on an environment limitation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 participant
Why
The harness/modality integration tests are
#[ignore]— they need a display, the AT-SPI bus, and the GTK3 harness. So the rest of CI only compiles them; their behavioral signal has come from manual VM/host runs at a point in time, not continuous protection. This was called out as the single biggest gap in the desktop-scope confidence review.What
A Linux job that actually runs them on every cua-driver change:
modality_capture_mode_test(ax→tree-only / vision→image-only / som→both)modality_desktop_scope_linux_test(window-less screen-absolute click via XTest + thedesktop_scope_disabledgate)The GUI session is provided headlessly: Xvfb (display) + a per-job D-Bus session bus (so AT-SPI
org.a11y.Busauto-activates) + at-spi2-core (bridge) + GTK3/PyGObject (harness runtime). An AT-SPI reachability probe prints up front for visibility.Honest framing
The tests skip-with-note when the a11y bus is unavailable rather than false-failing, so a vacuous run is visible (no green-for-nothing). This PR's own run is the experiment: it tells us whether headless AT-SPI is viable on GitHub runners, or whether this coverage must live in the Nix desktop sessions (
nix-wayland.yml) instead. If the AT-SPI probe + theax/somassertions come back real (not skipped), we have the lane; if not, the follow-up is a Nix-session check.Runs on free runners, path-filtered to the Rust driver + harness, plus push-to-main and manual dispatch. Companion to — not a replacement for —
nix-build.yml/nix-wayland.yml.🤖 Generated with Claude Code
Summary by CodeRabbit