Skip to content

feat(cua-driver): complete the browser action surface - #2323

Merged
f-trycua merged 21 commits into
mainfrom
codex/browser-completeness
Jul 18, 2026
Merged

f-trycua merged 21 commits into
mainfrom
codex/browser-completeness

Conversation

@f-trycua

@f-trycua f-trycua commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • make active-tab reporting exact and refuse ambiguity instead of guessing
  • add exact-tab JavaScript dialog handling, ref-bound file uploads, pointer-action parity, and approval-gated downloads
  • preserve full-background behavior where the browser can prove it, with explicit trusted-versus-synthetic input routes and structured refusals
  • extend the canonical standalone browser matrix, recordings, telemetry contract, skill, and Diataxis documentation

Delivery contract

  • Windows and macOS resolve page-owned JavaScript dialogs in the background under strict focus, z-order, leaked-input, and cursor oracles
  • Linux Chromium cannot resolve its native JavaScript modal without changing foreground posture; background resolution now refuses before dispatch, while delivery_mode="foreground" is separately tested and supported
  • trusted Chromium pointer input is delivered in the background on Windows; macOS and Linux refuse that route before dispatch and retain the explicit ref-targeted dom_event route for full-background page interaction

Safety and privacy

  • dialog state retains only kind and an opaque generation; prompt text and page URLs are not journaled
  • upload paths and prompt contents are redacted from recordings and telemetry
  • downloads require non-forgeable MCP-host approval, an existing canonical directory, exact frame/event correlation, and path-free output
  • exact target, tab, snapshot, frame, and document capabilities fail closed when stale or ambiguous
  • the complete branch diff was scanned for local paths, credentials, and partner-specific material

Validation

  • immutable source: 6fe95ec1d2158d5f03cc965791effb77b01a900f
  • Windows and Linux standalone browser workflow:
    • Windows/Win32, Chrome + Edge: 28 delivered, 2 refused, 0 failed, 0 skipped
    • Linux/X11, Chrome + Edge: 26 delivered, 6 refused, 0 failed, 0 skipped
  • macOS/Quartz, Chrome, same source SHA: 13 delivered, 2 refused, 0 failed, 0 skipped, with all 15 videos present
  • cargo test -p cua-driver-core browser --lib (135 passed)
  • cargo test -p cua-driver-testkit browser_fixture --lib (3 passed)
  • standalone browser test target compiles
  • docs generator, docs hygiene, and link checks pass
  • release attribution and historical backfill suites: 29 passed

Attribution

The exact page-targeting foundation and material dialog design lineage come from Haoqing Wang in #2166. The landing history contains a parsed Co-authored-by trailer and this pull request carries Salvaged from #2166, so release attribution continues to credit the contributor even though this PR supersedes and reimplements parts of the earlier code shape rather than merging that PR verbatim.

Salvaged from #2166

Closes #818
Closes #2240
Closes #2307
Closes #2320
Closes #2321

trycua-release[bot] and others added 20 commits July 17, 2026 09:22
…eness

# Conflicts:
#	docs/content/docs/reference/cua-driver/telemetry.mdx
#	libs/cua-driver/rust/crates/cua-driver-core/src/server.rs
#	libs/cua-driver/rust/crates/cua-driver-core/src/session.rs
#	libs/cua-driver/rust/crates/cua-driver/src/telemetry.rs
Keep native-window correlation separate from selected-tab proof. Duplicate, empty, or colliding titles now serialize active as null instead of guessing, while unique title and embedded single-page proofs retain exact booleans.\n\nRefs #2307
Add exact-tab JavaScript dialog capabilities and ref-targeted CDP file input assignment. Dialog state retains only bounded kind and generation; uploads validate direct regular files and omit paths from results.\n\nSalvaged from #2166\nRefs #2240 #2320\n\nCo-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>
Salvaged from #2166

Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>
Represent Linux Chromium JavaScript-dialog resolution as an explicit foreground capability with a side-effect-free background refusal. Stabilize prepared-browser binding and keep the canonical matrix and public docs aligned.

Salvaged from #2166

Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>
@f-trycua
f-trycua force-pushed the codex/browser-completeness branch from c71ef4c to 31b4f85 Compare July 18, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant