Skip to content

feat(cua-driver): add checked state verification - #2705

Merged
f-trycua merged 9 commits into
mainfrom
codex/cua-driver-checked-state
Jul 31, 2026
Merged

f-trycua merged 9 commits into
mainfrom
codex/cua-driver-checked-state

Conversation

@f-trycua

@f-trycua f-trycua commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a contract-generated verify_state surface shared by MCP, Rust, Python, and TypeScript
  • evaluate bounded window and semantic accessibility predicates with explicit satisfied / unsatisfied / unknown results and consecutive stable samples
  • keep multimodal visual reading in the agent harness via optional uninterpreted screenshot evidence
  • preserve the existing action ladder and canonical runtime: verification directly consumes platform observation tools without registry re-entry or a second dispatcher
  • make verification observations non-mutating so they cannot replace the live action element-index/token caches
  • normalize role/control state across macOS AX, Windows UIA/MSAA, and Linux AT-SPI and fail closed for incomplete or untrusted observations
  • add representative SwiftUI, WinUI 3, and GTK 3 native harness cases

Local validation

  • cargo fmt --all --check
  • cargo test -p cua-driver
  • cargo test -p cua-driver-core --lib
  • cargo test -p cua-driver-sdk --lib
  • cargo test -p platform-macos --lib
  • cargo test -p cua-driver-core --test contract_parity
  • cargo run --locked -p cua-driver-contract --bin cua-contract-gen -- all --check
  • node scripts/generate-uniffi-bindings.mjs --check
  • cargo run --locked -p cua-driver-bindgen --bin cua-driver-abi-header -- --check
  • npx tsx scripts/docs-generators/runner.ts --library cua-driver --check
  • Python 3.12: 28 passed, 3 environment-dependent skips
  • TypeScript: typecheck and 6 tests passed

Independent design review

Claude Code Fable completed an evidence-based follow-up review at the final source state with a CLEAR verdict. All seven earlier trust-boundary findings were verified closed; no new merge blockers were found.

Exact-SHA cross-platform validation

Base: 771e0165e7ecd2566cbb17b793a863755a8d651f
Head: 4d59e2f3c392fcda21c37a8d8193e818f18c83bb

@f-trycua
f-trycua marked this pull request as ready for review July 31, 2026 11:26
@f-trycua

Copy link
Copy Markdown
Collaborator Author

Exact-SHA macOS certification completed on the disposable Lume VM at 8573e5a3816d42e79296b1e24fe81b617a87963f.

  • Canonical macos-lume/run-all.sh suite: 152/152 passing
  • Outcomes: 144 delivered, 8 expected refusals, 0 failed, 0 skipped
  • Environment preflight: passed
  • Coverage included AppKit, SwiftUI, Electron, Tauri, WKWebView, TextEdit, browser routes, AX/background, AX/foreground, pixel/background, pixel/foreground, focus/z-order/cursor/no-leaked-input oracles, and verify_state
  • Pulled evidence summary records the same source SHA and failure_count: 0

This PR is ready for review. It should still land after the pending 0.14.2 release PR so the release sequence remains ordered.

@f-trycua
f-trycua force-pushed the codex/cua-driver-checked-state branch from 8573e5a to e52ff35 Compare July 31, 2026 14:50
@f-trycua

Copy link
Copy Markdown
Collaborator Author

Exact-SHA macOS certification is complete for e52ff35fdc2756194659d442ce03841dbfd51162 (Cua Driver 0.14.2) on a fresh disposable Lume macOS VM.

  • canonical visible-Aqua runner: libs/cua-driver/tests/runners/macos-lume/run-all.sh
  • 152/152 passed
  • 144 verified deliveries
  • 8 expected refusals
  • 0 failures, 0 skips
  • no retry record / no bounded retry used
  • source marker and environment.jsonl both report the exact PR SHA
  • harness coverage: Electron 41, Tauri 41, WKWebView 41, AppKit 19, SwiftUI 5, plus desktop/Calculator/TextEdit coverage

All 28 GitHub checks on the rebased head are also complete with no failures. Independent review remains the only merge gate.

@f-trycua
f-trycua merged commit 84440dd into main Jul 31, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant