Skip to content

fix(cua-driver-rs)(macos): sync agent-cursor registry on click - #1769

Merged
f-trycua merged 1 commit into
mainfrom
fix/cua-driver-rs-agent-cursor-registry-sync
May 30, 2026
Merged

fix(cua-driver-rs)(macos): sync agent-cursor registry on click#1769
f-trycua merged 1 commit into
mainfrom
fix/cua-driver-rs-agent-cursor-registry-sync

Conversation

@f-trycua

@f-trycua f-trycua commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Problem

The click tool has two addressing modes — element_index (AXPress) and pixel (x, y). Both animate the agent-cursor overlay to the target, but neither wrote the resulting position back to the CursorRegistry. Only move_cursor did.

As a result, get_agent_cursor_state always reported position: null after a click, even though the overlay had moved to the element. The state query and the rendered overlay were tracked in two separate places that were never synced — so a backgrounded automation clicking via element_index (the recommended path) could never report where its cursor was.

Fix

Mirror each animate_cursor_to(...) with a cursor_registry.update_position("default", x, y) call on both paths in click.rs (AX path + pixel path), so the reported state matches what is actually rendered regardless of which mode drove the click.

Pure state-reporting fix — clicking behaviour itself is unchanged.

Verification

  • cargo check -p platform-macos → exit 0.

Follow-up

A docs note for get_agent_cursor_state (that position now tracks click-driven movement) will land in a separate PR.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved cursor position tracking accuracy to ensure cursor state reporting remains synchronized with actual cursor movements during click operations.

Review Change Stack

The `click` tool's two addressing modes — element_index (AXPress) and
pixel (x, y) — both animate the agent-cursor overlay to the target, but
neither wrote the resulting position back to the CursorRegistry. Only
move_cursor did. As a result get_agent_cursor_state always reported
`position: null` after a click, even though the overlay had moved to the
element: the state query and the rendered overlay were tracked in two
places that were never synced.

Mirror each animate_cursor_to(...) with a cursor_registry.update_position()
call on both paths so the reported state matches what is rendered,
regardless of which addressing mode drove the click. Pure state-reporting
fix — clicking behaviour is unchanged.

Docs: note in mcp-tools.mdx that get_agent_cursor_state.position now
tracks click-driven movement, not just explicit move_cursor calls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 30, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Ignored Ignored May 30, 2026 4:59am

Request Review

@coderabbitai

coderabbitai Bot commented May 30, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 62dcfaf8-a2aa-4cd0-8ea4-671f4152315a

📥 Commits

Reviewing files that changed from the base of the PR and between be8ecec and 2cf8929.

📒 Files selected for processing (1)
  • libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs

📝 Walkthrough

Walkthrough

The ClickTool::invoke method now synchronizes cursor_registry state with overlay cursor position after animation in both click addressing modes: updating registry with animated coordinates in the AX element path and with screen coordinates in the pixel path.

Changes

Cursor State Synchronization

Layer / File(s) Summary
Sync cursor state across click paths
libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs
After animating the overlay cursor to the element center in the AX element path and to the computed click point in the pixel path, cursor_registry is updated with the respective coordinates to keep cursor state truthful and aligned with overlay position.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 A cursor hops and finds its way,
Through AX paths and pixels' play,
Registry synced, state stays true,
Two clicks, one heart, a rabbit's brew!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/cua-driver-rs-agent-cursor-registry-sync

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@f-trycua
f-trycua merged commit b5300f1 into main May 30, 2026
9 of 10 checks passed
@f-trycua
f-trycua deleted the fix/cua-driver-rs-agent-cursor-registry-sync branch May 30, 2026 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant