fix(cua-driver): unify desktop snapshot identity and payload ownership - #3616
Open
injaneity wants to merge 17 commits into
Open
fix(cua-driver): unify desktop snapshot identity and payload ownership#3616injaneity wants to merge 17 commits into
injaneity wants to merge 17 commits into
Conversation
Refs #3473. Tests-first characterization only; four desired shared invariants currently fail. No production ownership abstraction added.
This was referenced Sep 7, 2026
injaneity
changed the title
Refs #3473. Evolve the runtime-owned cache into the sole identity and native payload authority, preserve full-width window IDs, and retain admitted native targets through worker completion. Preserve the existing tests-first history. Add cross-platform invariants, the AppKit pending-publication regression, and reproducible production/latency measurements. Native desktop certification remains pending; the known excluded cancelled-worker drain diagnostic remains visible.
Refs #3473. Remove the investigation-only cancelled-worker drain test at maintainer request. Preserve in-scope lifecycle coverage and document the finding and recoverable source in the worklog. No production behavior changes.
Refs #3473. Preserve upstream lifecycle maintenance, recording coordinate projection and transport reporting, capture-only snapshots, and Linux target scoping while retaining the unified runtime-owned snapshot authority. Restore inherited browser-input changes to main so they do not ship in this slice. Existing contributor commits remain in ancestry. macOS core/platform/SDK and shared invariants pass; production comparison against main is -151 lines and -64 structural complexity points. Cross-platform and native desktop certification remain pending.
Refs #3473. Remove only the dying cache binding from weak discovery and free an empty directory allocation. Preserve newer bindings in the same scope and keep payload destruction outside the directory lock. Extend lifetime coverage without changing worker draining or Memcheck policy.
Refs #3473. Align the checkout with the current CI workflow. Snapshot implementation and the Memcheck-verified cleanup are unchanged.
The public RecordingSession regression first reproduced capture_failed despite successful before/after screenshots and a single MacosAxAction with no drawable point. Apply the existing strict semantic predicate to that transport, without exempting pixels, fallback attempts, mismatched delivery, or missing capture evidence. A second red through validate_catalog exposed the corresponding reporter exclusion. Align that predicate and run its existing negative evidence checks for macOS too. Core and testkit suites pass. Native candidate verification remains required. (cherry picked from commit 4c8b9ff)
This was referenced Sep 8, 2026
Merge main 6c0348b. Preserve the admitted native target guard while adopting the upstream typed UIA outcome and refusal handling. Includes the upstream Windows provider recovery and macOS click fixes without reimplementing their contributions.
Open
3 tasks
injaneity
marked this pull request as ready for review
September 9, 2026 15:45
This was referenced Sep 10, 2026
Draft
This was referenced Sep 10, 2026
Open
injaneity
added this pull request to stack #3721
September 10, 2026 19:37
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix a race where an old snapshot token could act on a replacement element. Snapshot identity and native payload now share one runtime-owned cache across macOS, Windows, and Linux; actions retain the admitted target instead of looking it up again later.
Removes
TokenRegistryand duplicate bookkeeping.Refs #3473 — implements the selected desktop slice, not the full RFC. Browser migration, general SDK cancellation draining, and Windows geometry freshness (#2075) remain out of scope.
Measured changes
Baseline
6c0348b05→ tested product2a8b17060, using identical changed-file production scope:Pinned Rust parser, zero parse errors; tests/comments excluded and macros not expanded. Production projection: 1,395 additions / 1,532 deletions. These are not total PR diff counts.
16 alternating paired blocks; 320 measured samples per version/task plus five warmups per block. Milliseconds are medians across block medians; changes use geometric means of paired ratios, not the quotient of those displayed medians. Intervals use 20,000 paired bootstrap resamples, seed 3473, with a predeclared +5% upper-bound gate.
Result: essentially flat native latency, not a demonstrated speedup. All four upper bounds are below +5%. Scope is macOS AppKit over persistent MCP with screenshots/state checks; background pixels use AX hit-testing, not raw physical delivery. Other-platform/browser latency and native memory/read counts remain unmeasured.
Validation
Current candidate:
117910224ba1675c3d15ad11d6d643093c0dc213, including #3680's macOS ScreenCaptureKit/linker correction.20260910T194125Z-4f2a36b9), including the original wrong-target regression. Retained Windows 138/138 and Linux X11 131/131 evidence at3d13b8037, including installer lanes.3d13b8037is the macOS ScreenCaptureKit dependency/lock entry plus its macOS-only linker check and workflow. Windows/Linux product dependencies and runners are unchanged. The affected macOS matrix was recertified once after explicit selection; this is not a claim that all three platforms ran at the same SHA.bring_to_front_exact_window_unverifiedbeforebrowser_semantic_stateandbrowser_ref_staleassertions. Strict reporting rejected missing video/turn evidence. [Bug]: macOS standalone Chrome matrix refuses exact-window setup before ambiguity assertion #3681 remains open; no causal attribution to this PR or fix(cua-driver): eliminate duplicate Swift bridge symbols #3680. No retries or relaxed assertions. Earlier attempts remain retained.Disposition: the explicitly selected #3680 macOS follow-up is complete. Maintainer approval and disposition of the optional-browser exception remain. Main subsequently advanced with the separate browser consent-label change #3706 (
4c96f2496); it was not folded into this bounded run. Do not relabel an eventual newer merge result as this tested SHA; account for any further executable diff and validate its affected evidence before merge.The measurement table above is historical evidence at
2a8b17060, not a fresh task benchmark of3d13b8037. One-off measurement tools/data and planning documents are excluded from the landing diff; the immutable links below preserve them. No batching optimization is included.Detailed evidence and remaining limits · Reproducible measurements
Credit: semantic-recording evidence correction Salvaged from #2907, cherry-picked with
-xand subsequently incorporated through main.