fix(cua-driver): correct macos click delivery and recording evidence - #2907
Conversation
f-trycua
left a comment
There was a problem hiding this comment.
Blocking cross-tool regression: this PR changes click_at_xy_with_window_local to the public-only posting path, but platform-macos/src/tools/double_click.rs still calls that generic primitive directly for window-scoped background delivery. The ordinary click tool was updated to choose click_at_xy_chromium for background targets; double_click was not. That can remove the SkyLight route required by background Chromium/Catalyst while fixing duplicate left-click delivery.
Please route double_click and every shared consumer through the same target-aware single-route policy, and commit the count:2/background receiver regression fixture. Also replace Fixes #2874 with Refs #2874 unless the remaining acceptance rows (modified input, AppKit/Catalyst/Chromium/both-route fixtures, and right/middle/drag/wheel audit) are completed in this PR.
|
addressed in dac8999 and 341d4c7. double_click, click, the ax selection fallback, and the ax double-click fallback now use the shared WindowClickDelivery policy. the committed macos appkit background fixture requires count:2 and passed twice consecutively on macos 26.5.2 arm64: receiver clicks=2, foreground sentinel unchanged before/after, and zero leaked sentinel input. the sentinel setup now uses desktop hid, so setup no longer exercises the pid-routed behavior under test. the pr body uses refs #2874 and records the audit boundary. |
|
proposed vertical-slice tech spec for simplifying this existing workstream: #2874 (comment) scope: explicit-window left-click consumers, one transport selected before the pointer sequence, exact native event-journal tests, and a measured production-code/complexity reduction. right/middle click, drag, wheel, and snapshot ownership remain excluded. this is a proposal for maintainer scope approval, not a new implementation selection or a claim of native certification. no competing implementation pr has been opened. |
fea7def to
7619b29
Compare
This reverts commit 06d5707.
Refs #2874. One public MCP count:1 request must produce exactly down/up at the target with clickCount=1. The opt-in AppKit receiver records raw callbacks independently of button actions. No production code changes. Installed macOS 0.23.2 binary 67ccfc99e69ebb5881fdfc3787d85abcd8cc2beb7423f255549557623cab6907 reproduced down/down/up/up twice. Fixture build and native test compilation passed. Candidate native comparison is blocked by missing Accessibility and Screen Recording permissions; this is not exact-source or desktop-matrix certification.
Include the new invariant in the explicit AppKit runner list. Restore four trusted attribution mappings verbatim from main so the older PR branch preserves current contributor identities. No product behavior changes.
The invariant passed at 85fb599 in Lume, but the video completeness gate correctly rejected its unstarted recording. Start the standard behavior clip after fixture readiness. Native validation of this harness correction remains pending; preserve the original run failure.
Use the standard execute_case and recording_evidence lifecycle so the passing native invariant owns its video in the canonical report. The Lume run at 7619b29 passed all 157 declared behavior rows plus this invariant, but correctly rejected its orphan recording and a missing SwiftUI popover click screenshot. Preserve that failed run; native verification of this reporting correction is pending.
The public RecordingSession regression first reproduced capture_failed despite successful before/after screenshots and a single MacosAxAction with no drawable point. Apply the existing strict semantic predicate to that transport, without exempting pixels, fallback attempts, mismatched delivery, or missing capture evidence. A second red through validate_catalog exposed the corresponding reporter exclusion. Align that predicate and run its existing negative evidence checks for macOS too. Core and testkit suites pass. Native candidate verification remains required.
Preserve the structured bring_to_front result in the failing observation instead of discarding the observed foreground window and target visibility. This changes diagnostics only: activation, postcondition checks, recording boundaries, and retry behavior are unchanged.
ba15dcb to
5c9bc45
Compare
Current verification status
Current head:
5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9. Rebased onto main00678fa8ec8f0f371716993ae4a207df812ef667, preserving original commit authorship. The existing #2908 compositor-sibling fix is now included; no new keyboard workaround or input architecture was introduced.Exact-current-head macOS gate — PASS
Authorized full installed-wrapper run
20260908T173403Z-250fd96btested5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9with standalone-browser enabled and no retries.trusted_click, with the strict browser summary and video/trajectory evidence validator passing.cua-driver-browser-baseline-20260908-chrome152, the explicitly prepared Chrome 152.0.7977.83 local baseline derived from prior workers. This passing run does not certify it as an immutable private seed or erase its recorded lineage.fix(cua-driver)title accurately reflects the production changes.The earlier failures and focused diagnostics below remain part of the evidence history. The old sentinel failure did not recur in this complete run; its historical cause is not claimed as resolved. No Windows/Linux native-desktop certification or post-merge release verification is implied by this macOS result.
The outstanding GitHub changes-requested review is attached to old commit
0072ef325cab4a382cfacc30082bc84d90d26625. Final diff review confirmsdouble_click, ordinary click, selection fallback, and browser setup use the shared posture-aware left-click entry point; count-2 background coverage is retained; the description uses Refs #2874 and explicitly excludes generic/right/middle/drag/wheel transport redesign. Maintainer re-review is needed before merge; no review was dismissed and nothing was merged.Authorized rebase onto the pacman fix
Rebased cleanly onto
00678fa8ec8f0f371716993ae4a207df812ef667(#3636) and pushed with an exact force-with-lease againstba15dcb2e9952a6b2ba9fb075ee4f26b270e12ea. All 11 replayed commits are patch-equivalent by range-diff; their authors, messages, and coauthor trailers were checked unchanged. The native pacman script now exists on the PR head and matches main, together with its associated product code and tests. No CI skip or partial script transplant.Focused validation at
5c9bc45f6fe94994bc3a27b4f7395b829a6b2ea9: 604 core unit plus 6 integration tests passed; 93 testkit unit plus 2 additional tests passed; 365 macOS unit tests passed (2 ignored); 2 release-channel CLI tests passed. AppKit and standalone-browser targets compile. Rustfmt, diff checks, and pacman script shell syntax pass. New-head GitHub CI and exact-SHA native certification remain pending; previous native/focused evidence is retained below and is not relabeled as a test of this rebased SHA.Current diagnostic-only delta
The saved sentinel error retained only text, discarding the driver's structured observation (including the observed frontmost window ID and whether the target was visible). Current head retains that payload in the failing result. No product activation behavior, postcondition, recording boundary, or retry behavior changed. 93 testkit unit tests and 2 additional tests passed; the standalone-browser target compiles, formatting/diff checks pass. The release-built diagnostic harness was exercised in the correctly authorized focused replay below. The last full native-gate product SHA remains
4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2.Related active #3530 and #3573 were checked; neither establishes the cause of this exact foreground-setup failure. No competing focus implementation was started. The failure remains unresolved pending a focused native reproduction that preserves the observation, rather than a speculative focus change or another intermediate full-matrix rerun.
Correctly authorized focused browser replay
Run
focused-authorized-20260908T162447Zinvoked exactlystandalone_browser_trusted_clickonce in the worker GUI session, using a release-built harness fromba15dcb2e9952a6b2ba9fb075ee4f26b270e12eaand the unchanged installed product from4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2. It sourced the unchanged canonical runner's library helpers for keychain unlock, unrestricted daemon ownership/watchdog, and standard-mode restoration. Unrestricted mode was explicitly verified before the test; standard restoration was verified afterward. No test retry, TCC repair, or assertion relaxation.Passed: sentinel setup, the expected
browser_input_trust_unavailablerefusal, and all five oracles (fixture state, focus, z-order, no leaked input, cursor). Canonicalcua-e2e-report --require-videovalidation passed. Evidence collected and worker stopped. The prior foreground-setup failure did not reproduce; its cause is still unknown, and no browser-click production defect is established. This focused result does not overwrite the failed full run or claim full exact-current-candidate certification. Worker-derived baseline provenance remains unchanged.Current-head check revalidation also found a separate
Arch native pacman updatesfailure: its job invokes absentscripts/ci/linux/test-pacman-updates.shand exits 127 before running its tests. Log: https://github.com/trycua/cua/actions/runs/34248245465/job/102135713599 . No fix, rebase, or rerun of that job has been performed.Focused setup diagnostic
A single
standalone_browser_trusted_clickinvocation was launched in the disposable worker's GUI session, with harness sourceba15dcb2e9952a6b2ba9fb075ee4f26b270e12eaand the unchanged installed product at4c8b9ff20867de6e0398ef6de7a3a2a3413cdbd2. Sentinel preparation succeeded and behavioral recording started: the original foreground-setup failure did not reproduce. The invocation then failed at browser binding withbrowser_consent_required, because this diagnostic used the restored standard-mode daemon rather than the canonical full gate's unrestricted test mode. No authorization was weakened to force a pass. This is partial diagnostic evidence, not a passing browser test, not an environment-matched gate replay, and not proof of flakiness. Evidence was collected and the worker stopped. Current-head certification and the original failure remain unresolved.Browser-equipped rerun at 4c8b9ff
20260908T151005Z-8f4f9830used the unchanged installed Lume wrapper with standalone-browser enabled.cua-driver-browser-baseline-20260908-chrome152. The original PR feat(cua-driver): add target-scoped post-action rediscovery #3373 baseline and previous failed evidence were preserved. This is still a local worker-derived baseline, not a certified immutable private seed.macos-chrome-standalone-trusted_clickfailed during sentinel setup: exact window 3007 for PID 47608 could not be verified as frontmost (request_accepted=true,process_activated=true,focused=true,frontmost_ordinary=false). The trusted-click behavior assertion was not reached. This is not yet evidence of a browser-input regression or established flakiness.Native run at 4c8b9ff
20260908T071657Z-993e1868, with standalone-browser requested.failures.json: failure_count=0, preflight_failed=false, report_failed=false, no lane or video failures.no standalone Chrome, Edge, or Chromium executable was found, producing zero typed browser rows. The standalone runner reported 19 failed steps including reporting. No browser pass or complete certification is claimed.Native run at 7619b29
20260908T060357Z-272d94fc, with standalone-browser requested.same_pid_keyboard_ambiguityfailures. This resolves the 28 behavioral failures observed on the old branch.turn-00001/click.png, classifiedcapture_failed. Its before/after screenshots and behavior assertion passed. Follow-up testing established a recording classification defect: the strict semantic-without-point rule omittedmacos_ax_actionin both recorder and report validator. The native reason why the point was unavailable remains unproven.execute_case/recording_evidencelifecycle and prints receiver events. Current head also applies the existing strict semantic-without-point rule tomacos_ax_actionin recorder and report validator. Pixel targets, fallbacks, inconsistent action truth, resolved points without markers, and missing before/after captures remain rejected. Both corrections still require native candidate verification.cua-driver-macos-e2e-pr3373-20260904T173717Z, a clone-only local baseline, not a certified immutable private seed. Evidence collected and worker stopped.Focused checks
RecordingSessionregression reproduced the incorrectcapture_failedclassification before the correction and passes afterward; the publicvalidate_catalogregression separately reproduced the reporter exclusion before its correction and passes afterward. All 604 core unit tests, 6 core integration tests, and 93 testkit unit tests passed; formatting/diff checks passed. Existing strict action-truth rejection coverage now includes macOS.Preserved earlier evidence
67ccfc99e69ebb5881fdfc3787d85abcd8cc2beb7423f255549557623cab6907reproduced down/down/up/up twice. Its source SHA is unavailable.20260908T043947Z-311000eaat 85fb599 passed the invariant but failed 28 background keyboard rows and an unstarted recording. The keyboard failures were traced to missing merged PR fix(cua-driver): ignore macOS compositor sibling surfaces #2908, not waived as flaky.The requested exact-current-head macOS desktop and standalone-browser gates now pass, and current GitHub CI is green. The PR is ready for maintainer re-review of the outstanding old-head change request, with prior failures and worker-derived baseline limitations preserved. It has not been merged; this does not claim complete cross-platform/release certification. Do not treat passing behavior assertions as complete desktop certification.
Earlier implementation and historical validation
summary
clicks=2refs #2874
verification
platform: macos 26.5.2 (25f84), arm64
head: 341d4c7
cargo fmt --manifest-path libs/cua-driver/rust/Cargo.toml --all -- --check: passedcargo test --manifest-path libs/cua-driver/rust/Cargo.toml -p platform-macos --lib: 327 passedharness_appkit_double_click_px_background: passed twice consecutively; background receiver reportedclicks=2, while the electron sentinel remained foreground before and after and recorded no leaked inputharness_appkit_counter_px_background: passed; the ordinary background click consumer still delivered once without displacing the foreground sentinelcargo clippy --manifest-path libs/cua-driver/rust/Cargo.toml -p platform-macos --lib --tests -- -D warnings: blocked by existing clippy findings, first atcrates/cursor-overlay/src/session_badge.rs:317