Skip to content

fix(cua-driver/linux): glide the agent cursor to the desktop-scope click point - #2061

Merged
f-trycua merged 1 commit into
mainfrom
fix/linux-desktop-scope-cursor-overlay-glide
Jun 27, 2026
Merged

fix(cua-driver/linux): glide the agent cursor to the desktop-scope click point#2061
f-trycua merged 1 commit into
mainfrom
fix/linux-desktop-scope-cursor-overlay-glide

Conversation

@f-trycua

@f-trycua f-trycua commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

What

The Linux window-less desktop-scope click warped only the real pointer (XTest) and never moved the agent-cursor overlay — unlike the macOS/Windows desktop paths, which glide it. So the most visible cursor on screen sat idle elsewhere while the click landed on the target: a viewer (and a screen recording) sees the cursor "click somewhere else" even though the action is correct (the harness counter does advance).

Fix

Reuse the existing overlay_glide_to_for helper to glide the overlay to (sx, sy) before the XTest click, matching the other platforms.

How it was caught

Inspecting the Linux desktop-scope demo recording frame-by-frame — the overlay was parked in the lower-left while the counter advanced via the (warped) real pointer. Pure cosmetic/UX, but it makes the desktop-scope action legible in recordings and consistent across platforms.

Compile-checked via the Linux build in CI (couldn't build platform-linux on the dev Mac).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved desktop clicking behavior in full-screen/window-less mode by moving the visible cursor overlay to the target position before the click is sent.
    • This makes cursor movement and click actions appear more consistent during screen interactions.

…ick point

The Linux window-less desktop-scope click warped only the *real* pointer (XTest)
and never moved the agent-cursor *overlay* — unlike the macOS/Windows desktop
paths, which glide it. So the most visible cursor on screen sat idle wherever it
was last, while the click landed elsewhere: a viewer (and a screen recording)
sees the cursor "click somewhere else" even though the action is correct (the
harness counter does advance).

Reuse the existing `overlay_glide_to_for` helper to glide the overlay to (sx,sy)
before the XTest click, matching the other platforms. Cosmetic/UX + makes the
desktop-scope action legible in recordings.

Caught by inspecting the Linux desktop-scope demo recording frame-by-frame.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Jun 27, 2026 4:37am

Request Review

@coderabbitai

coderabbitai Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: de757aba-b2c8-45c8-bc66-b68e80c48508

📥 Commits

Reviewing files that changed from the base of the PR and between 86913e2 and ab9375c.

📒 Files selected for processing (1)
  • libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs

📝 Walkthrough

Walkthrough

The Linux desktop-scope click path now awaits the cursor overlay glide to the target coordinates before dispatching the desktop click event.

Changes

Desktop click overlay order

Layer / File(s) Summary
Overlay glide before desktop click
libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs
The window-less desktop click branch now awaits overlay_glide_to_for(&cursor_id, sx, sy) before send_click_xtest_desktop.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Possibly related PRs

  • trycua/cua#2056: Also changes the Linux click desktop-scope path around overlay handling and click dispatch ordering.
  • trycua/cua#1789: Modifies the same Linux click flow to move the overlay toward the target before the click is sent.
  • trycua/cua#1731: Touches the same impl_.rs overlay-click behavior in the Linux driver.

Poem

🐰 I hopped to the pixel, light and quick,
Then glided the overlay right to the click.
A tiny bounce before the desktop chime,
One neat little hop at just the right time.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the Linux cursor-overlay click fix and matches the main change in the PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/linux-desktop-scope-cursor-overlay-glide

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@f-trycua
f-trycua merged commit 7468487 into main Jun 27, 2026
70 checks passed
@f-trycua
f-trycua deleted the fix/linux-desktop-scope-cursor-overlay-glide branch June 27, 2026 04:45
@github-actions

Copy link
Copy Markdown
Contributor

Linux visual regression artifacts

Matrix jobs now run independently. Download visual artifacts from this workflow run.
Each background-GUI job uploads a .gif of the interaction plus two annotated PNGs (<app>.png raw, <app>-atspi.png with AT-SPI element boxes); the cua-driver-linux-som-overlays artifact adds <app>-som.png cua Set-of-Marks overlays:

  • cua-driver-linux-cursor-click-gif
  • cua-driver-linux-background-terminal-gif
  • cua-driver-linux-parallel-drag-xserver
  • cua-driver-linux-background-gui-chromium
  • cua-driver-linux-background-gui-tk
  • cua-driver-linux-background-gui-gtk3-gedit
  • cua-driver-linux-background-gui-gtk3-mousepad
  • cua-driver-linux-background-gui-gtk3-scite
  • cua-driver-linux-background-gui-gtk4-characters
  • cua-driver-linux-background-gui-qt5-manuskript
  • cua-driver-linux-background-gui-qt5-klog
  • cua-driver-linux-background-gui-qt5-openambit
  • cua-driver-linux-background-gui-qt6-kate
  • cua-driver-linux-background-gui-qt6-kcalc
  • cua-driver-linux-background-gui-qt6-okular
  • cua-driver-linux-background-gui-qt6-qownnotes
  • cua-driver-linux-background-gui-electron-zettlr
  • cua-driver-linux-background-gui-electron-joplin
  • cua-driver-linux-background-gui-electron-logseq
  • cua-driver-linux-som-overlays

Open workflow run and download artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant