feat(cua-driver-rs): Phase 2 panel + structural fixes (opt-in) - #1566
Merged
Conversation
…y default
Adds Phase 2 of the native NSPanel onboarding gate (live status rows
with red/green icons, dynamic heading, auto-dismiss on all-green via a
1 Hz NSTimer in NSModalPanelRunLoopMode) and a handful of structural
fixes that surfaced during live testing.
Critical change in user-facing default: the panel is now **opt-in**.
The previous default of "panel when launched from /Applications/CuaDriver.app"
relied on a chain of macOS behaviours that break in practice:
* Cargo's default linker-signing identifier (`cua_driver-<random>`)
differs from the bundle id, so TCC keys registrations under the
random hash and the bundle never appears in System Settings.
* `tccutil reset` invalidates the database row but not the session-
level TCC trust cache, so an in-process AXIsProcessTrusted() call
still returns the cached "trusted" state — the gate short-
circuits before the panel can be presented.
* The cursor overlay's NSApp run-loop coexistence works in the
Serve arm today but is fragile to future changes; an opt-in
default reduces blast radius if the assumption regresses.
Until those are stabilised end-to-end, the terminal flow stays the
default (banner + 1 Hz poll + "still waiting on: X" status lines).
The panel still exists and can be exercised with
`CUA_DRIVER_RS_PERMISSIONS_PANEL=1`. Accepted on-sentinels (case-
insensitive): 1, true, yes, on.
Structural fixes bundled in this commit:
* Rename `libs/cua-driver-rs/scripts/CuaDriver.app/` to
`scripts/CuaDriverBundle/`. The previous `.app` suffix on the
repo-template caused macOS LaunchServices on developer machines
to index it as a second installed app with the `com.trycua.driver`
bundle id, surfacing a "ghost CuaDriver" entry in System
Settings → Privacy & Security pointing at an empty
Contents/MacOS/ directory. Dropping the `.app` suffix removes
the indexing. CD workflow + install-script comments updated to
point at the new path.
* Move `request_accessibility()` / `request_screen_recording()`
calls in `gate::run_if_needed` from after the panel to before
it. These calls have the side effect of registering the calling
process with TCC — without that, when a user clicks "Open
System Settings" mid-panel they land in a Privacy & Security
pane with cua-driver missing from the list. Hoisting the calls
ahead of presentation closes that UX regression.
* Phase 2 internals: row construction returns `RowHandles` (icon
view + title label) stashed in a thread-local so the poll
callback can update them in place. Heading/subheading text
rebinds on every grant flip via `setStringValue:`. Auto-dismiss
triggers via `[NSApp stopModal]` from the poll callback when
both grants are green. Timer is added to
`NSModalPanelRunLoopMode` explicitly because
`[NSApp runModalForWindow:]` does not run timers added to the
default mode or to NSRunLoopCommonModes.
Tests: 13/13 permissions tests passing
(`cargo test -p platform-macos --lib permissions::`). New parity
tests cover the heading-text matrix and the "subheading mentions
remaining permission" property.
Docs (`docs/.../getting-started/installation.mdx`) rewritten to
describe the new opt-in semantics; CD workflow updated to copy from
`scripts/CuaDriverBundle/`; install-script comments + PARITY.md
point at the new path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the previous commit's bundle-skeleton rename. Lands the
Phase 2 panel implementation, the gate-flow restructuring, and the
matching CD/docs/PARITY updates.
* libs/cua-driver-rs/crates/platform-macos/src/permissions/panel.rs:
Phase 2 live status rows (RowHandles struct stashed in a thread-
local so the poll callback can update icon + label color in
place), dynamic heading/subheading via setStringValue: on every
tick, auto-dismiss via [NSApp stopModal] when both grants flip
green. Timer added to NSModalPanelRunLoopMode explicitly because
[NSApp runModalForWindow:] does NOT fire timers in the default
mode or NSRunLoopCommonModes. Panel flipped to opt-in:
panel_enabled() now requires CUA_DRIVER_RS_PERMISSIONS_PANEL set
to an on-sentinel (1/true/yes/on, case-insensitive); env unset =
terminal flow.
* libs/cua-driver-rs/crates/platform-macos/src/permissions/gate.rs:
Hoist request_accessibility() / request_screen_recording() calls
from after the panel to before it. These calls have the side
effect of registering the calling process with TCC; without
that, clicking "Open System Settings" mid-panel lands the user
in a Privacy & Security pane where cua-driver is missing from
the list. New PanelPresentation::ShownAllGranted variant lets
the gate skip the trailing wait_for_grants loop when the
panel's poll already saw both grants green.
* libs/cua-driver-rs/PARITY.md + libs/cua-driver/scripts/_install-rust.sh:
Update the references to the previous scripts/CuaDriver.app
skeleton to point at scripts/CuaDriverBundle. Adds a paragraph
explaining why the path no longer ends in `.app` (LaunchServices
ghost-bundle prevention).
* .github/workflows/cd-rust-cua-driver.yml: Copy from
scripts/CuaDriverBundle/Contents into release/CuaDriver.app/Contents
instead of from scripts/CuaDriver.app. Comment updated.
* docs/content/docs/cua-driver/guide/getting-started/installation.mdx:
Rewrite the permissions-gate Callout to describe the opt-in
semantics: terminal flow is the documented default; native panel
is "experimental" and opt-in via CUA_DRIVER_RS_PERMISSIONS_PANEL=1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughThe PR refactors the macOS permissions panel from a static modal to a live-updating NSPanel, reorganizes the app bundle skeleton storage to prevent LaunchServices indexing, and updates gate orchestration and documentation. The panel now auto-dismisses when all required grants are active, while the bundle structure moves from ChangesmacOS Permissions Panel Live-Update and Bundle Structure
Sequence Diagram(s)sequenceDiagram
participant run_if_needed
participant show_modal_unsafe
participant pollTick_callback
participant current_status
participant NSModalSession
run_if_needed->>show_modal_unsafe: call with initial PermissionsStatus
show_modal_unsafe->>NSModalSession: create and run modal loop
show_modal_unsafe->>pollTick_callback: register timer callback
loop Poll on timer tick
pollTick_callback->>current_status: get current grant status
pollTick_callback->>show_modal_unsafe: update row icons/tints
pollTick_callback->>show_modal_unsafe: show "All set" strip if all granted
alt All grants green
pollTick_callback->>show_modal_unsafe: set AllGranted outcome
pollTick_callback->>NSModalSession: stop modal
end
end
show_modal_unsafe->>run_if_needed: return PanelOutcome
sequenceDiagram
participant run_if_needed
participant TCC_API
participant present_panel_if_available
participant wait_for_grants
run_if_needed->>TCC_API: raise prompts early (if also_raise_prompts)
run_if_needed->>present_panel_if_available: call with initial PermissionsStatus
alt Panel shown and all grants became active
present_panel_if_available->>run_if_needed: return ShownAllGranted
run_if_needed->>run_if_needed: skip wait_for_grants loop
else Panel dismissed or opened settings
present_panel_if_available->>run_if_needed: return OpenSettings or Dismissed
run_if_needed->>wait_for_grants: poll for grants
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced May 18, 2026
f-trycua
added a commit
that referenced
this pull request
May 21, 2026
…vas-like surfaces `click(x, y)` previously routed through `try_invoke_in_window_at_point` for any UIA element advertising InvokePattern at the click coordinates. For container surfaces (Pane / Image / Custom / Document / Group) `Invoke()` fires the element's default action at its centre and ignores the requested (x, y) — silently breaking pixel precision on canvases, paint surfaces, image maps, and 3D viewports. ## Repro (from #1621, verified 2026-05-21 on the Windows VM against `libs/cua-driver-fixtures/test_page.html` loaded in Edge) ``` cua-driver call click '{"pid":<edge-pid>,"x":110,"y":677}' # → "✅ Performed UIA Invoke at (110,677)" # But #canvas-status shows: "canvas: clicked at (152,77)" # ─ canvas center, not the requested (110, 677) ``` The canvas's `mousedown` handler fired at synthesised centre coords because UIA `Invoke()` has no notion of "where inside the element". ## Fix Add `is_coord_independent_action()` — a control-type whitelist for elements whose primary action is coord-independent (Button, MenuItem, Hyperlink, TabItem, ListItem, CheckBox, RadioButton, SplitButton, TreeItem). For these, UIA Invoke is the semantically correct path — the element identity *is* the action target, and the click coords don't matter past hit-testing. For everything else (Pane / Image / Custom / Document / Group / etc.), even if the element advertises InvokePattern, fall through to PostMessage with the literal coords. This preserves UWP / WebView2 coverage for buttons + menu items (the original motivation for the UIA-first path) without silently rerouting canvas clicks to "click at centre". ## What this does NOT change - Element-indexed click (`click(element_index=N)`) is unchanged — it takes the UIA Invoke path explicitly via a different code site (`impl_.rs:1168`). Callers asking for "invoke this specific element" by index keep the previous behaviour. - Right-click and multi-click already skipped the UIA path (`use_uia = (btn == "left" || btn == "middle") && count == 1`); they remain on PostMessage. - The ExpandCollapsePattern preference for Qt menu-bar items (added in #1566) is unchanged — that path runs after the new control-type filter and only fires when a whitelisted-type element happens to also have ExpandCollapse. ## Test plan - [x] `cargo check -p platform-windows` clean on the VM (41.82s, 0 new warnings — all 28 warnings are pre-existing) - [x] `cargo build --release -p cua-driver` clean (24.43s release build) - [ ] **Runtime verification deferred to next interactive RDP session**: load `libs/cua-driver-fixtures/test_page.html` in Edge with the four anti-occlusion + a11y flags (see #1620), call `click(pid, x, y)` at a non-centre point inside the canvas, expect tool response to say `"✅ Posted click to pid <pid>"` (not `"Performed UIA Invoke"`) and `#canvas-status` to report the requested coords ±2px. The SSH-only session in tonight's autonomous run can't reach an interactive desktop (daemon ends up in Session 0; `list_windows` returns empty) so the click test couldn't run end-to-end — user needs to RDP in to verify. ## Related - #1620 — Chromium anti-throttling flags (separate fix; needed for any Edge/Chrome DOM verification on hidden launches, including this test) Closes #1621. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
f-trycua
added a commit
that referenced
this pull request
May 21, 2026
…vas-like surfaces (#1622) `click(x, y)` previously routed through `try_invoke_in_window_at_point` for any UIA element advertising InvokePattern at the click coordinates. For container surfaces (Pane / Image / Custom / Document / Group) `Invoke()` fires the element's default action at its centre and ignores the requested (x, y) — silently breaking pixel precision on canvases, paint surfaces, image maps, and 3D viewports. ## Repro (from #1621, verified 2026-05-21 on the Windows VM against `libs/cua-driver-fixtures/test_page.html` loaded in Edge) ``` cua-driver call click '{"pid":<edge-pid>,"x":110,"y":677}' # → "✅ Performed UIA Invoke at (110,677)" # But #canvas-status shows: "canvas: clicked at (152,77)" # ─ canvas center, not the requested (110, 677) ``` The canvas's `mousedown` handler fired at synthesised centre coords because UIA `Invoke()` has no notion of "where inside the element". ## Fix Add `is_coord_independent_action()` — a control-type whitelist for elements whose primary action is coord-independent (Button, MenuItem, Hyperlink, TabItem, ListItem, CheckBox, RadioButton, SplitButton, TreeItem). For these, UIA Invoke is the semantically correct path — the element identity *is* the action target, and the click coords don't matter past hit-testing. For everything else (Pane / Image / Custom / Document / Group / etc.), even if the element advertises InvokePattern, fall through to PostMessage with the literal coords. This preserves UWP / WebView2 coverage for buttons + menu items (the original motivation for the UIA-first path) without silently rerouting canvas clicks to "click at centre". ## What this does NOT change - Element-indexed click (`click(element_index=N)`) is unchanged — it takes the UIA Invoke path explicitly via a different code site (`impl_.rs:1168`). Callers asking for "invoke this specific element" by index keep the previous behaviour. - Right-click and multi-click already skipped the UIA path (`use_uia = (btn == "left" || btn == "middle") && count == 1`); they remain on PostMessage. - The ExpandCollapsePattern preference for Qt menu-bar items (added in #1566) is unchanged — that path runs after the new control-type filter and only fires when a whitelisted-type element happens to also have ExpandCollapse. ## Test plan - [x] `cargo check -p platform-windows` clean on the VM (41.82s, 0 new warnings — all 28 warnings are pre-existing) - [x] `cargo build --release -p cua-driver` clean (24.43s release build) - [ ] **Runtime verification deferred to next interactive RDP session**: load `libs/cua-driver-fixtures/test_page.html` in Edge with the four anti-occlusion + a11y flags (see #1620), call `click(pid, x, y)` at a non-centre point inside the canvas, expect tool response to say `"✅ Posted click to pid <pid>"` (not `"Performed UIA Invoke"`) and `#canvas-status` to report the requested coords ±2px. The SSH-only session in tonight's autonomous run can't reach an interactive desktop (daemon ends up in Session 0; `list_windows` returns empty) so the click test couldn't run end-to-end — user needs to RDP in to verify. ## Related - #1620 — Chromium anti-throttling flags (separate fix; needed for any Edge/Chrome DOM verification on hidden launches, including this test) Closes #1621. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on #1565 (Phase 1 panel). Lands Phase 2 of the native NSPanel onboarding gate AND a handful of structural macOS fixes that surfaced during live testing on this Mac. Key user-facing change: the panel is now opt-in via
CUA_DRIVER_RS_PERMISSIONS_PANEL=1rather than auto-shown when the daemon is launched from the bundled.app. The terminal banner is the documented default again.Why opt-in
During live testing the panel works once you grant TCC manually — but a chain of macOS behaviours makes the "auto-shown" default fragile in dev environments:
cua_driver-9402b19cac15ebc5, not thecom.trycua.driverbundle id. TCC keys registrations under the codesign identifier, so dev builds never appear in System Settings → Privacy & Security. Manualcodesign --force --sign - -i com.trycua.driverafter build fixes it; CD's Developer-ID signing step already does. (Follow-up: bake this intoinstall-local.shand/or a Makefile target.)tccutil resetdoesn't invalidate the session-level TCC trust cache: an in-processAXIsProcessTrusted()call still returns the cached "trusted" state after a reset, so the gate short-circuits before the panel can be presented. Only a logout/reboot fully clears the cache.[NSApp runModalForWindow:]+ a 1HzNSTimerinNSModalPanelRunLoopMode(NOTNSRunLoopCommonModes, which silently drops timer fires in modal sessions); that needed a specific fix and is fragile to future changes.Until those are stabilised end-to-end, terminal flow remains the default. The panel can be exercised by power users via
CUA_DRIVER_RS_PERMISSIONS_PANEL=1.What's in this PR
Phase 2 panel code (`crates/platform-macos/src/permissions/panel.rs`)
Gate restructuring (`crates/platform-macos/src/permissions/gate.rs`)
Structural macOS fixes
Test plan
Related
tccutil reset Accessibility com.trycua.driver(macOS) #1561 (permissions onboarding parity gap).🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Improvements
Documentation