Skip to content

fix(cua-driver): correct macos click delivery and recording evidence - #2907

Merged
injaneity merged 11 commits into
mainfrom
codex/fix-2874-macos-single-route-click
Sep 8, 2026
Merged

fix(cua-driver): correct macos click delivery and recording evidence#2907
injaneity merged 11 commits into
mainfrom
codex/fix-2874-macos-single-route-click

Conversation

@injaneity

@injaneity injaneity commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Current verification status

Current head: 5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9. Rebased onto main 00678fa8ec8f0f371716993ae4a207df812ef667, preserving original commit authorship. The existing #2908 compositor-sibling fix is now included; no new keyboard workaround or input architecture was introduced.

Exact-current-head macOS gate — PASS

Authorized full installed-wrapper run 20260908T173403Z-250fd96b tested 5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9 with standalone-browser enabled and no retries.

  • 158/158 declared desktop rows passed. Canonical failures: failure_count=0, preflight_failed=false, report_failed=false; no failing lanes or videos.
  • 18/18 standalone Chrome rows passed, including trusted_click, with the strict browser summary and video/trajectory evidence validator passing.
  • The raw foreground single-click invariant and SwiftUI semantic recording evidence passed as part of this exact-source run. No assertion, authorization rule, or evidence gate was weakened.
  • Installed wrapper exit 0; evidence collected, standard daemon restoration verified, worker stopped.
  • Clone source: cua-driver-browser-baseline-20260908-chrome152, the explicitly prepared Chrome 152.0.7977.83 local baseline derived from prior workers. This passing run does not certify it as an immutable private seed or erase its recorded lineage.
  • Rechecked PR head unchanged after the run. All current GitHub checks pass, including Arch pacman, contributor attribution, and release metadata. Final diff remains 11 files; the releasing fix(cua-driver) title accurately reflects the production changes.

The earlier failures and focused diagnostics below remain part of the evidence history. The old sentinel failure did not recur in this complete run; its historical cause is not claimed as resolved. No Windows/Linux native-desktop certification or post-merge release verification is implied by this macOS result.

The outstanding GitHub changes-requested review is attached to old commit 0072ef325cab4a382cfacc30082bc84d90d26625. Final diff review confirms double_click, ordinary click, selection fallback, and browser setup use the shared posture-aware left-click entry point; count-2 background coverage is retained; the description uses Refs #2874 and explicitly excludes generic/right/middle/drag/wheel transport redesign. Maintainer re-review is needed before merge; no review was dismissed and nothing was merged.

Authorized rebase onto the pacman fix

Rebased cleanly onto 00678fa8ec8f0f371716993ae4a207df812ef667 (#3636) and pushed with an exact force-with-lease against ba15dcb2e9952a6b2ba9fb075ee4f26b270e12ea. All 11 replayed commits are patch-equivalent by range-diff; their authors, messages, and coauthor trailers were checked unchanged. The native pacman script now exists on the PR head and matches main, together with its associated product code and tests. No CI skip or partial script transplant.

Focused validation at 5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9: 604 core unit plus 6 integration tests passed; 93 testkit unit plus 2 additional tests passed; 365 macOS unit tests passed (2 ignored); 2 release-channel CLI tests passed. AppKit and standalone-browser targets compile. Rustfmt, diff checks, and pacman script shell syntax pass. New-head GitHub CI and exact-SHA native certification remain pending; previous native/focused evidence is retained below and is not relabeled as a test of this rebased SHA.

Current diagnostic-only delta

The saved sentinel error retained only text, discarding the driver's structured observation (including the observed frontmost window ID and whether the target was visible). Current head retains that payload in the failing result. No product activation behavior, postcondition, recording boundary, or retry behavior changed. 93 testkit unit tests and 2 additional tests passed; the standalone-browser target compiles, formatting/diff checks pass. The release-built diagnostic harness was exercised in the correctly authorized focused replay below. The last full native-gate product SHA remains 4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2.

Related active #3530 and #3573 were checked; neither establishes the cause of this exact foreground-setup failure. No competing focus implementation was started. The failure remains unresolved pending a focused native reproduction that preserves the observation, rather than a speculative focus change or another intermediate full-matrix rerun.

Correctly authorized focused browser replay

Run focused-authorized-20260908T162447Z invoked exactly standalone_browser_trusted_click once in the worker GUI session, using a release-built harness from ba15dcb2e9952a6b2ba9fb075ee4f26b270e12ea and the unchanged installed product from 4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2. It sourced the unchanged canonical runner's library helpers for keychain unlock, unrestricted daemon ownership/watchdog, and standard-mode restoration. Unrestricted mode was explicitly verified before the test; standard restoration was verified afterward. No test retry, TCC repair, or assertion relaxation.

Passed: sentinel setup, the expected browser_input_trust_unavailable refusal, and all five oracles (fixture state, focus, z-order, no leaked input, cursor). Canonical cua-e2e-report --require-video validation passed. Evidence collected and worker stopped. The prior foreground-setup failure did not reproduce; its cause is still unknown, and no browser-click production defect is established. This focused result does not overwrite the failed full run or claim full exact-current-candidate certification. Worker-derived baseline provenance remains unchanged.

Current-head check revalidation also found a separate Arch native pacman updates failure: its job invokes absent scripts/ci/linux/test-pacman-updates.sh and exits 127 before running its tests. Log: https://github.com/trycua/cua/actions/runs/34248245465/job/102135713599 . No fix, rebase, or rerun of that job has been performed.

Focused setup diagnostic

A single standalone_browser_trusted_click invocation was launched in the disposable worker's GUI session, with harness source ba15dcb2e9952a6b2ba9fb075ee4f26b270e12ea and the unchanged installed product at 4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2. Sentinel preparation succeeded and behavioral recording started: the original foreground-setup failure did not reproduce. The invocation then failed at browser binding with browser_consent_required, because this diagnostic used the restored standard-mode daemon rather than the canonical full gate's unrestricted test mode. No authorization was weakened to force a pass. This is partial diagnostic evidence, not a passing browser test, not an environment-matched gate replay, and not proof of flakiness. Evidence was collected and the worker stopped. Current-head certification and the original failure remain unresolved.

Browser-equipped rerun at 4c8b9ff

  • Authorized run 20260908T151005Z-8f4f9830 used the unchanged installed Lume wrapper with standalone-browser enabled.
  • Chrome 152.0.7977.83 was installed from Google's official universal installer, with Google signing identity and Apple notarization verified. Welcome setup was completed signed out, with default-browser and usage-reporting opt-ins disabled. The inspect page reopened without a consent dialog.
  • The previously retained worker was shut down and its stopped state verified before cloning separate browser baseline cua-driver-browser-baseline-20260908-chrome152. The original PR feat(cua-driver): add target-scoped post-action rediscovery #3373 baseline and previous failed evidence were preserved. This is still a local worker-derived baseline, not a certified immutable private seed.
  • Desktop: 158/158 rows passed, with no preflight, report, lane, or video failures. Both click/recording corrections remain green.
  • Standalone browser: 17/18 typed rows passed. macos-chrome-standalone-trusted_click failed during sentinel setup: exact window 3007 for PID 47608 could not be verified as frontmost (request_accepted=true, process_activated=true, focused=true, frontmost_ordinary=false). The trusted-click behavior assertion was not reached. This is not yet evidence of a browser-input regression or established flakiness.
  • The failed setup left that row's recording pending with no video/turn evidence; the strict browser report validator rejected it. Overall runner exit 1; no passing standalone summary is claimed.
  • Evidence collected; this run's worker stopped successfully. No automatic test retry or permission repair. The previous worker-cleanup blocker is also resolved by the verified shutdown used for browser preparation.
  • Exact PR head was rechecked unchanged; ordinary GitHub CI is green. Browser discovery is no longer the blocker.

Native run at 4c8b9ff

  • Authorized Lume run 20260908T071657Z-993e1868, with standalone-browser requested.
  • All 158 declared desktop rows passed. Canonical failures.json: failure_count=0, preflight_failed=false, report_failed=false, no lane or video failures.
  • Both previous evidence defects are resolved on the exact candidate: the raw single-click invariant now owns its finalized video/trajectory, and SwiftUI popover evidence retains captured before/after screenshots while correctly classifying the nonspatial AXPress marker as not_applicable/semantic_action_without_point.
  • Overall run FAILED (exit 1): 18 standalone-browser scenarios stopped with no standalone Chrome, Edge, or Chromium executable was found, producing zero typed browser rows. The standalone runner reported 19 failed steps including reporting. No browser pass or complete certification is claimed.
  • Host tool connection was lost, but the original guest runner continued and completed without restart. Its evidence was recovered using the installed wrapper's archive-validation helpers. Standard autostart daemon restoration was verified.
  • Cleanup remains unresolved: first worker-stop command was interrupted; a subsequent stop returned zero but fresh inventory still reported the owned worker running. No VM was deleted, test rerun, browser installed, or permission repaired.
  • Clone source remains the retained PR feat(cua-driver): add target-scoped post-action rediscovery #3373 worker, a clone-only local baseline, not a certified immutable private seed.
  • PR head was rechecked unchanged after execution; ordinary GitHub checks are green.

Native run at 7619b29

  • Authorized Lume run 20260908T060357Z-272d94fc, with standalone-browser requested.
  • All 157 declared behavior rows passed, with zero same_pid_keyboard_ambiguity failures. This resolves the 28 behavioral failures observed on the old branch.
  • The additional raw foreground count:1 invariant also passed: exactly one ordered native down/up pair at the requested target, clickCount=1. Its behavior video was created and finalized.
  • Overall gate still FAILED on evidence validation: the invariant video was orphaned because the test did not declare a matrix result; the SwiftUI popover row was also missing required turn-00001/click.png, classified capture_failed. Its before/after screenshots and behavior assertion passed. Follow-up testing established a recording classification defect: the strict semantic-without-point rule omitted macos_ax_action in both recorder and report validator. The native reason why the point was unavailable remains unproven.
  • The orphan integration now uses the existing execute_case / recording_evidence lifecycle and prints receiver events. Current head also applies the existing strict semantic-without-point rule to macos_ax_action in recorder and report validator. Pixel targets, fallbacks, inconsistent action truth, resolved points without markers, and missing before/after captures remain rejected. Both corrections still require native candidate verification.
  • No standalone-browser passing result is claimed after the primary gate failure. No automatic retry was performed.
  • Clone source: retained PR feat(cua-driver): add target-scoped post-action rediscovery #3373 worker cua-driver-macos-e2e-pr3373-20260904T173717Z, a clone-only local baseline, not a certified immutable private seed. Evidence collected and worker stopped.

Focused checks

  • Rebased product: 365 macOS unit tests passed, 2 ignored; 15 core background-input tests passed.
  • Current reporting correction: 93 testkit unit tests passed; AppKit invariant target compiles; formatting/diff checks passed.
  • Current head: public RecordingSession regression reproduced the incorrect capture_failed classification before the correction and passes afterward; the public validate_catalog regression separately reproduced the reporter exclusion before its correction and passes afterward. All 604 core unit tests, 6 core integration tests, and 93 testkit unit tests passed; formatting/diff checks passed. Existing strict action-truth rejection coverage now includes macOS.
  • Ordinary GitHub CI was green at native-tested head 7619b29; checks at current head are still running. Contributor attribution and release metadata passed.

Preserved earlier evidence

  • Installed 0.23.2 binary 67ccfc99e69ebb5881fdfc3787d85abcd8cc2beb7423f255549557623cab6907 reproduced down/down/up/up twice. Its source SHA is unavailable.
  • Lume run 20260908T043947Z-311000ea at 85fb599 passed the invariant but failed 28 background keyboard rows and an unstarted recording. The keyboard failures were traced to missing merged PR fix(cua-driver): ignore macOS compositor sibling surfaces #2908, not waived as flaky.

The requested exact-current-head macOS desktop and standalone-browser gates now pass, and current GitHub CI is green. The PR is ready for maintainer re-review of the outstanding old-head change request, with prior failures and worker-derived baseline limitations preserved. It has not been merged; this does not claim complete cross-platform/release certification. Do not treat passing behavior assertions as complete desktop certification.

Earlier implementation and historical validation

summary

  • route every window-scoped left-click consumer through one posture-aware api
  • use one public pid post for foreground delivery
  • use one skylight post for background delivery, with public fallback only when skylight is unavailable
  • cover click, double_click, selection fallback, and browser setup without changing right/middle click, drag, wheel, or interactive input
  • require the appkit background count-2 receiver to report exactly clicks=2
  • focus the macos foreground sentinel through desktop hid so fixture setup does not exercise the pid-routed behavior under test

refs #2874

verification

platform: macos 26.5.2 (25f84), arm64
head: 341d4c7

  • cargo fmt --manifest-path libs/cua-driver/rust/Cargo.toml --all -- --check: passed
  • cargo test --manifest-path libs/cua-driver/rust/Cargo.toml -p platform-macos --lib: 327 passed
  • native appkit count-2 background receiver: one move, two downs, two ups; click states 1, 1, 2, 2
  • native appkit count-1 background and foreground receivers: one down and one up in each mode
  • harness_appkit_double_click_px_background: passed twice consecutively; background receiver reported clicks=2, while the electron sentinel remained foreground before and after and recorded no leaked input
  • harness_appkit_counter_px_background: passed; the ordinary background click consumer still delivered once without displacing the foreground sentinel
  • cargo clippy --manifest-path libs/cua-driver/rust/Cargo.toml -p platform-macos --lib --tests -- -D warnings: blocked by existing clippy findings, first at crates/cursor-overlay/src/session_badge.rs:317

@injaneity injaneity changed the title fix(cua-driver): post macos window clicks through one transport fix(cua-driver): avoid duplicate macos window clicks Aug 5, 2026
@injaneity
injaneity marked this pull request as ready for review August 5, 2026 19:25
@injaneity
injaneity requested a review from f-trycua as a code owner August 5, 2026 19:25

@f-trycua f-trycua left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking cross-tool regression: this PR changes click_at_xy_with_window_local to the public-only posting path, but platform-macos/src/tools/double_click.rs still calls that generic primitive directly for window-scoped background delivery. The ordinary click tool was updated to choose click_at_xy_chromium for background targets; double_click was not. That can remove the SkyLight route required by background Chromium/Catalyst while fixing duplicate left-click delivery.

Please route double_click and every shared consumer through the same target-aware single-route policy, and commit the count:2/background receiver regression fixture. Also replace Fixes #2874 with Refs #2874 unless the remaining acceptance rows (modified input, AppKit/Catalyst/Chromium/both-route fixtures, and right/middle/drag/wheel audit) are completed in this PR.

@injaneity
injaneity requested a review from f-trycua August 6, 2026 15:51
@injaneity

Copy link
Copy Markdown
Collaborator Author

addressed in dac8999 and 341d4c7. double_click, click, the ax selection fallback, and the ax double-click fallback now use the shared WindowClickDelivery policy. the committed macos appkit background fixture requires count:2 and passed twice consecutively on macos 26.5.2 arm64: receiver clicks=2, foreground sentinel unchanged before/after, and zero leaked sentinel input. the sentinel setup now uses desktop hid, so setup no longer exercises the pid-routed behavior under test. the pr body uses refs #2874 and records the audit boundary.

@injaneity

Copy link
Copy Markdown
Collaborator Author

proposed vertical-slice tech spec for simplifying this existing workstream: #2874 (comment)

scope: explicit-window left-click consumers, one transport selected before the pointer sequence, exact native event-journal tests, and a measured production-code/complexity reduction. right/middle click, drag, wheel, and snapshot ownership remain excluded. this is a proposal for maintainer scope approval, not a new implementation selection or a claim of native certification. no competing implementation pr has been opened.

@injaneity
injaneity force-pushed the codex/fix-2874-macos-single-route-click branch from fea7def to 7619b29 Compare September 8, 2026 06:02
@injaneity injaneity changed the title fix(cua-driver): avoid duplicate macos window clicks fix(cua-driver): correct macos click delivery and recording evidence Sep 8, 2026
injaneity and others added 11 commits September 8, 2026 12:09
Refs #2874. One public MCP count:1 request must produce exactly down/up at the target with clickCount=1. The opt-in AppKit receiver records raw callbacks independently of button actions. No production code changes.

Installed macOS 0.23.2 binary 67ccfc99e69ebb5881fdfc3787d85abcd8cc2beb7423f255549557623cab6907 reproduced down/down/up/up twice. Fixture build and native test compilation passed. Candidate native comparison is blocked by missing Accessibility and Screen Recording permissions; this is not exact-source or desktop-matrix certification.
Include the new invariant in the explicit AppKit runner list. Restore four trusted attribution mappings verbatim from main so the older PR branch preserves current contributor identities. No product behavior changes.
The invariant passed at 85fb599 in Lume, but the video completeness gate correctly rejected its unstarted recording. Start the standard behavior clip after fixture readiness. Native validation of this harness correction remains pending; preserve the original run failure.
Use the standard execute_case and recording_evidence lifecycle so the passing native invariant owns its video in the canonical report. The Lume run at 7619b29 passed all 157 declared behavior rows plus this invariant, but correctly rejected its orphan recording and a missing SwiftUI popover click screenshot. Preserve that failed run; native verification of this reporting correction is pending.
The public RecordingSession regression first reproduced capture_failed despite successful before/after screenshots and a single MacosAxAction with no drawable point. Apply the existing strict semantic predicate to that transport, without exempting pixels, fallback attempts, mismatched delivery, or missing capture evidence.

A second red through validate_catalog exposed the corresponding reporter exclusion. Align that predicate and run its existing negative evidence checks for macOS too. Core and testkit suites pass. Native candidate verification remains required.
Preserve the structured bring_to_front result in the failing observation instead of discarding the observed foreground window and target visibility. This changes diagnostics only: activation, postcondition checks, recording boundaries, and retry behavior are unchanged.
@injaneity
injaneity force-pushed the codex/fix-2874-macos-single-route-click branch from ba15dcb to 5c9bc45 Compare September 8, 2026 17:10
@injaneity
injaneity merged commit 467c103 into main Sep 8, 2026
41 checks passed
@injaneity
injaneity deleted the codex/fix-2874-macos-single-route-click branch September 8, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants