feat(cua-driver): add stable AX node identity tokens

[hqhq1025]

Status: tracking draft. This PR grew to 19 files / +1274 −766, which is well outside what is reviewable as a single change in this repo (recent merged PRs sit around ~111 lines). It is being split into focused PRs; this one stays open only to track that split. Please review the individual PRs below instead.

Split status

Part	PR	Status
Full 32-bit snapshot generation (token collision fix)	#2608	open, independent
set_value AXValue read-back with verified / changed	#2621	open, independent

Both are single-file, independent of each other, and independent of the token work below.

Re-scoped after review of upstream main

Two items I originally listed for extraction turned out not to be upstream defects at all, and I'm not proposing them as standalone fixes:

token_for returning Option instead of panicking. Upstream's token_for is a pure formatting helper — it does not consult the registry and cannot panic. The .expect("token requested for an unregistered snapshot element") I was pointing at was introduced by this PR's own registry rework, not by upstream. It is an internal correctness issue of the mechanism proposed here, not a bug to fix in main.

Element-action TOCTOU. The race you identified in review —

If another session re-snapshots the same window between validation and cache lookup, that index can refer to a different AX node

— applies to the token-resolution path this PR adds. Upstream ax/cache.rs has only update(pid, window_id, nodes); there is no generation tracking and no resolve_token, so there is nothing in main to harden independently. This belongs with whatever token design lands, not before it.

I'd rather say this explicitly than ship two PRs that claim to fix upstream bugs which aren't there.

Deliberately dropped from the split

Three parts of this PR should not land in their current form:

Opaque UUID tokens. Replacing the token body with a UUID v4 conflicts with the format goals already documented in element_token.rs ("8-16 char budget", "debug-grep-able without a side-table"), and at least one user in #2207 types tokens by hand on the command line. Replay resistance across daemon restarts is better served by a per-process nonce that keeps the readable shape, e.g. s{nonce}-{generation}:{index}.

allow_pixel_fallback(via_token) = !via_token. This couples addressing to dispatch strategy, which are orthogonal. cua-driver's architecture is AX-first with a pixel ladder; a token should make "which element" more precise, not change "how it is delivered". The requirement it encoded ("never fall back to pixels") belongs in the downstream client, which can already enforce it by rejecting structuredContent.path != "ax". If a driver-side switch is still wanted it should be an explicit opt-in parameter that leaves current behaviour unchanged by default, proposed separately.

node_identity. The value stored was element_ptr as u64 — the same pointer the cache already holds. A fresh AX walk yields a new AXUIElementRef, so it cannot survive a re-snapshot and does not deliver what the PR title claimed. A real stable identity needs CFEqual / AXIdentifier (both already used in ax/tree.rs), and that deserves its own change with its own justification.

What remains for the token mechanism itself

The remaining pieces (opaque-capability tokens, generation binding, typed rejection codes) change a public contract, so per rfcs/README.md they should go through an RFC issue rather than arriving as a PR. I'll open one rather than re-pushing this branch.

Original description

Expand

Hardens the existing cua-driver element_token protocol on macOS without introducing a second token abstraction: keeps the cross-platform TokenRegistry as the single token source, widens the generation, attaches macOS AX node identity to the snapshot entry, binds token use to PID / window / generation / index / node identity, returns typed fail-closed error codes, retains the validated AX node for click and set_value, and performs set_value write/read-back on that same node reporting idempotent writes as changed:false.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[hqhq1025]

@f-trycua Could you please review this stable AX node identity PR? It follows the same safety boundary as #2166, where explicit page identity must be preserved and wrong-target execution must fail closed.

This PR adds opaque element_token values from get_window_state, strictly bound to PID, window_id, generation/snapshot, element index, and AX node identity. click, set_value, type_text_chars, press_key, and double_click validate the token and never silently fall back to a stale index. set_value performs same-node readback and reports changed:false for idempotent writes.

Validation completed locally:

• cua-driver-core: 115 tests passed
• platform-macos: 119 tests passed
• modified Rust files pass rustfmt check
• no Windows/Linux/CLI files changed

This is also the blocking dependency for Maka PR #984, which consolidates the downstream target-identity fixes. The downstream Maka adapter is intentionally fail-closed until this opaque-token protocol is available. Thanks!

[hqhq1025]

Update after rebasing onto cua-driver-rs 0.8.1 (ab2b7041):

• cua-driver-core: 136 tests passed
• platform-macos: 125 tests passed
• rustfmt + diff checks passed
• downstream Maka now consumes the opaque token directly (no custom node resolver)
• real OpenAI gpt-5.6-sol AppKit runs passed: AX set_value, AX click_element, multi-step set_value→fresh-token click with intervention recovery, and process-restart stale-token recovery
• all successful mutations used AX dispatch; zero pixel fallback

The fork workflows are currently waiting for maintainer approval to run. Could a maintainer approve the workflows and review this PR? This is the remaining release dependency for Maka #984.

[hqhq1025]

Adversarial review found and fixed two protocol gaps in the previous revision. The latest head f15f8d87 now issues UUID v4 opaque capabilities backed by a server-side token→binding registry, so tokens no longer expose or permit derivation of generation/index and cannot replay across daemon restarts. Both strict AX resolution and the shared token resolver reject superseded snapshots. Token-targeted middle click also fails closed instead of falling through to CGEvent/pixels.\n\nValidation on the latest head:\n- cua-driver-core: 126 passed\n- platform-macos: 126 passed, 1 ignored\n- modified Rust files pass rustfmt and diff checks\n- real gpt-5.6-sol AppKit qualification with candidate SHA 3c80f2a8c758e5c55b6393300f7108a023ece32db1ef7d84e13f4d95ef29441b passed AX set_value, AX click, set_value→click multi-step, and process-restart recovery\n- restart run: stale old-PID action returned target_missing; model rediscovered the new PID/window and the fresh AX set_value succeeded\n- all successful mutations used AX dispatch; zero pixel dispatch\n\nThe fork workflow suites are again waiting for maintainer approval on the new head.

[f-trycua]

I reviewed the current head, f15f8d87, and found three issues that should be addressed before merge:

1. [P1] Empty Linux and Windows snapshots now panic the daemon. token_for now calls expect when the requested element token does not exist (source). Linux and Windows still call token_for(snapshot_id, 0) unconditionally to populate snapshot_id (Linux, Windows). When element_count == 0, the registry creates no element tokens, so the supported empty/degraded-tree path panics. The old formatter did not require a registered element.

2. [P1] Several destructive token consumers still discard the exact-node binding. The generic resolver returns only (window_id, element_index) (source). type_text, type_text_chars, press_key, double_click, right_click, and scroll then reacquire the macOS element by raw index. If another session re-snapshots the same window between validation and cache lookup, that index can refer to a different AX node. type_text_chars and press_key can also continue when the focus lookup misses. These tools should use the same generation/identity-aware retained-node resolver used by click and set_value.

3. [P2] snapshot_id changes type only on macOS. macOS now returns the numeric generation as snapshot_id (source), while Linux and Windows still return a string. This breaks the existing structured-output shape and creates cross-platform schema drift. Please preserve the old snapshot_id type or add generation as a separate field.

Local verification on this head passed:

• cargo test -p cua-driver-core: 123 unit and 3 integration tests
• cargo test -p platform-macos: 126 tests; 1 doc test ignored
• cargo test -p cua-driver
• Targeted rustfmt --check and git diff --check

The linked update's core test count is stale for the current head, and GitHub currently reports no checks for this branch.

[hqhq1025]

Addressed all three review blockers in eeb24aa3 and pushed the updated head.

1. Empty/degraded snapshots

• snapshot_id metadata no longer calls token_for(generation, 0), so zero-element Linux/Windows snapshots do not panic.
• Added a core regression test covering an empty registered snapshot with no element token.

2. Exact-node binding for destructive macOS consumers

• type_text, type_text_chars, press_key, double_click, right_click, and scroll now resolve token targets through the generation/identity-aware ElementCache::resolve_token path and retain the exact AX node through dispatch.
• Token-targeted keyboard actions use strict AX focus and fail closed on focus miss.
• Token-targeted type_text does not degrade to focused-element/CGEvent typing.
• Token-targeted double-click/right-click/scroll refuse pixel/CGEvent fallback when the exact AX action is unavailable. Legacy index and explicit pixel behavior is unchanged.
• Added regressions for superseded snapshot + same-index node replacement, strict focus miss, and no pixel fallback for exact-node capabilities.

3. Cross-platform snapshot schema

• Preserved the existing string snapshot_id shape (s########) on macOS/Linux/Windows.
• Added numeric generation as a separate field on all three platforms.

Validation on eeb24aa3:

• cargo test -p cua-driver-core: 124 unit + 3 integration passed
• cargo test -p platform-macos: 129 passed, 1 doc test ignored
• cargo test -p platform-linux: 16 passed
• cargo test -p platform-windows: 32 passed on the macOS-buildable cross-platform test surface
• cargo test -p cua-driver --test protocol_element_token_test --test protocol_schema_test: 4 passed
• cargo test -p cua-driver --test schema_consistency_test: 1 passed
• git diff --check: passed
• targeted rustfmt --check for the changed core/cache/AX-action files: passed

The full stable cargo fmt --all -- --check remains red on pre-existing formatting in the parent f15f8d87 tree (reproduced against a clean HEAD copy, e.g. platform-macos/src/tools/type_text.rs), so I did not introduce a large unrelated formatting-only diff.

Opaque-token/replay audit remains fail-closed: tokens are UUID v4 capabilities with server-side bindings, a new registry instance rejects pre-restart tokens as unknown, and a newer snapshot rejects the old same-window token as stale before any cache index can be reused.

[hqhq1025]

Rebased the branch onto the latest upstream main and force-pushed with lease.

New base/head:

• base: 6b6ad1e0 (docs: link computer-use overview from related guides (#2346))
• head: c40481bd (fix(cua-driver): preserve exact token targets)
• rewritten stack: 462719f7 → 1528e910 → a5f79df6 → c40481bd

Conflict resolution summary:

• Preserved upstream 0.9.0 behavior, including daemon-backed dispatch, the 20-second product AX deadline, per-element AX messaging timeouts, deterministic Rust 1.97.1 formatting, and the new browser consent/setup fixtures.
• Reapplied the opaque UUID token registry, generation/identity-aware macOS cache, superseded-token rejection, and restart replay rejection.
• Preserved the three review-blocker fixes: empty snapshots no longer request token index 0; destructive macOS token consumers retain and dispatch against the exact node with strict focus/fail-closed fallback policy; snapshot_id remains a string while numeric generation is separate on all platforms.
• Resolved non-text integration issues from upstream additions: removed the duplicate core uuid dependency and added node_identity to the new browser consent/setup AXNode fixtures.

Post-rebase validation:

• cargo test -p cua-driver-core: 290 unit + 3 integration passed; 1 doc test ignored
• cargo test -p platform-macos: 152 passed; 1 doc test ignored
• cargo test -p platform-linux: 16 passed
• cargo test -p platform-windows: 32 passed on the macOS-buildable cross-platform surface
• cargo test -p cua-driver --test protocol_element_token_test --test protocol_schema_test --test schema_consistency_test: 6 passed
• cargo fmt --all -- --check: passed with the upstream pinned Rust 1.97.1 toolchain
• git diff --check: passed
• git merge-base --is-ancestor upstream/main HEAD: passed
• local git merge-tree against upstream/main reports no conflict markers

The branch was pushed with an explicit lease against prior head eeb24aa3; no concurrent remote update was overwritten.

[hqhq1025]

Closing. Everything in here that stands on its own has been sent separately, and what remains is not worth another rebase.

Merged from the split:

• fix(cua-driver): carry full 32-bit snapshot generation in element tokens #2608 — full 32-bit snapshot generation in element tokens
• fix(cua-driver): verify set_value writes with an AXValue read-back #2621 — set_value AXValue read-back verification

Open from the split:

• feat(cua-driver): expose element AX actions in structured elements #2622 — expose element AX actions in structured elements
• fix(cua-driver): refuse stale element targets in press_key and type_text_chars #2672 — refuse stale element targets in press_key / type_text_chars, which is your review's [P1] point 2 reduced to the part that needs no new machinery

Point 1 of your review (empty snapshots panicking token_for) does not apply to main — token_for there is a pure formatter over format_token and cannot panic. That was only ever a defect of this branch's registry. Point 3 (snapshot_id type drift) I fixed here by keeping snapshot_id a string and adding generation alongside it, but that fix only exists because this branch changed the type in the first place; main never had the drift.

Deliberately not carried forward, with reasons in the PR body above: the UUID token format (conflicts with the documented 8–16 character greppable goal), allow_pixel_fallback(via_token) (couples addressing to dispatch strategy — that requirement belongs in the downstream client), and node_identity (it stored element_ptr as u64, which cannot survive a re-snapshot, so it did not deliver what the name claimed).

What is genuinely left undone: the residual TOCTOU between resolve() succeeding and get_element_retained() running. Closing that needs the snapshot generation stored in each platform's element cache plus a generation-checked accessor — three platform crates and eight call sites. That is precisely how this PR grew to 19 files, and it should be its own change with its own review, not a rider on anything else.

Thanks for the reviews on #2608 and #2621 — splitting this up was the right call.