fix(cua-driver): target page JavaScript exactly by hqhq1025 · Pull Request #2166 · trycua/cua

hqhq1025 · 2026-07-11T20:24:24Z

Summary

allow page.execute_javascript to accept explicit cdp_port and target_url_contains
require an explicit target hint to match exactly one CDP page
reject invalid CDP ports instead of truncating them

Why

Multi-window Electron processes expose multiple page targets on one CDP port. execute_javascript previously discarded the caller's resolved page identity and selected the first page target. An explicit URL hint also fell back to the first page when no target matched, so a successful RPC could execute against the wrong renderer.

The lower-level CDP session code already supported selecting a target by URL. This change threads that identity through the page tool and makes explicit hints fail closed when absent or ambiguous.

Validation on latest upstream main

Rebased without conflict onto trycua/cua@8c921b2b3bf13494724ead4f0a814d80c56a7e8b.

cargo test -p cua-driver-core page::tests
  3 passed

cargo test -p platform-macos browser::cdp_client::tests
  6 passed

cargo test -p cua-driver-core cdp::tests
  2 passed

cargo check -p cua-driver --release
  passed

cargo check -p platform-windows --target x86_64-pc-windows-gnu
  passed

cargo check -p platform-linux  # Debian bookworm / Rust 1.92
  passed

The targeted-page commit is 68a42c06bf3fdfa429d65e9c011efa13c2414548; review feedback is addressed by 5c2b402d, which reuses the existing CDP session cache, and 0e9a8294, which prevents replay after a script has been sent. 35fa5658 completes explicit targeted routing on Windows and Linux. git range-diff reports it equivalent to the originally tested patch commit adef3e87405986cc82df52ae59aef4c32e08a082.

Real multi-window downstream validation

Maka uses cua-driver as the sole page executor. Its real-machine harness runs two independent Electron BrowserWindow pages on one CDP port and passes the exact port + unique URL hint through cua-driver for pointer actions, element inspection, Input.insertText, and post-action readback.

Final batch semantic-targeting-v5:

10/10 complete runs passed
39/39 checks passed in every run
two-window click/type routing and text isolation passed in every run
zero fallback, wrong-target, no-op, or duplicate-effect cases
frontmost application and real pointer remained unchanged

Per-action results:

Action	Pass	Route	p50 / p90 / max
left click	10/10	targeted `page.execute_javascript`	84 / 87 / 102 ms
checkbox click	10/10	targeted `page.execute_javascript`	70 / 77 / 77 ms
range drag	10/10	targeted `page.execute_javascript`	96 / 102 / 127 ms
right click	10/10	targeted `page.execute_javascript`	74 / 80 / 85 ms
double click	10/10	targeted `page.execute_javascript`	74 / 78 / 98 ms

The downstream repo also passed its full typecheck, build, tests, bundle provenance gate, and final 39/39 real-machine E2E after integrating the patched driver.

Compatibility

Repeated targeted evaluations reuse the existing per-port CDP session cache, avoiding repeated Chrome remote-debugging confirmation prompts.
Retry is limited to retarget failures before Runtime.evaluate; a sent script is never replayed after timeout or disconnect.
Windows and Linux consume explicit cdp_port and unique URL hints through the shared CDP helper; calls without explicit targeting retain the existing backend behavior.
Backends that do not implement targeted execution fail closed only when the caller supplies an explicit target.
insert_text and type_keystrokes now share the checked CDP port parser, eliminating the previous u16 truncation behavior.

hqhq1025 · 2026-07-12T04:47:13Z

@f-trycua Would you mind reviewing this targeted page-selection change? You are the primary recent author for the affected cua-driver page/CDP paths.

I rebased it onto current main@8c921b2b; the focused Rust tests pass 7/7 and cargo check -p cua-driver --release passes. Downstream, we exercised two Electron pages sharing one CDP port for 10 complete real-machine runs (39/39 checks each), with zero wrong-target or fallback cases. The detailed evidence and latency table are in the PR description. Thanks!

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68a42c06bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-12T04:47:23Z

+        port: u16,
+        target_url_contains: Option<&str>,
+    ) -> anyhow::Result<String> {
+        let mut session = CdpSession::connect(port, target_url_contains).await?;


Reuse the CDP session for targeted evaluation

When execute_javascript is called with an explicit cdp_port for Chrome's in-browser remote-debugging toggle, this creates a new WebSocket for every evaluation. The CdpSessionCache in this same module exists because that endpoint prompts on each new connection, and insert_text/type_keystrokes already reuse it; targeted execute_javascript will therefore reprompt or stall automation on repeated calls instead of reusing the approved session.

Useful? React with 👍 / 👎.

Fixed in 5c2b402d. Explicit targeted execute_javascript now goes through the existing ToolState.cdp_sessions cache, with the same stale-session eviction and one retry used by insert_text / type_keystrokes. The original untargeted Electron inspector path remains one-shot.

I added targeted_evaluate_reuses_browser_websocket, backed by a local fake CDP browser endpoint. Two targeted evaluations perform one HTTP discovery probe and share exactly one browser WebSocket connection while reattaching on that socket. The full macOS CDP test set is now 5/5 and the release check passes.

hqhq1025 · 2026-07-12T05:02:23Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c2b402d3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-12T05:06:28Z

+        self.evict(port).await;
+        let arc = self.get_or_connect(port, target_url_contains).await?;
+        let mut session = arc.lock().await;
+        session.ensure_target(port, target_url_contains).await?;
+        do_evaluate(&mut session, javascript).await


Avoid replaying timed-out JavaScript

For targeted execute_javascript, do_evaluate has already sent the caller's script before it waits for the response, but any error from that wait now falls through to this retry path. If a slow or side-effectful snippet times out, or the socket drops after Chrome accepted Runtime.evaluate, the same JavaScript is executed a second time; the previous execute path surfaced the error without replaying the script. Please limit the retry to connection/retarget failures that happen before sending the user script, or return evaluate errors directly.

Useful? React with 👍 / 👎.

Fixed in 0e9a8294. The cache now retries only when ensure_target fails before Runtime.evaluate is sent. Once the caller's JavaScript is dispatched, do_evaluate returns its success or error directly, so timeouts / response loss / disconnects cannot replay a side-effectful script.

Added targeted_evaluate_does_not_replay_after_disconnect: the fake CDP endpoint closes immediately after receiving Runtime.evaluate, then watches for a reconnect. The call errors and the accepted connection count remains exactly 2 (one HTTP probe + one browser WebSocket), proving the script was sent once. macOS CDP tests are now 6/6; core page tests 3/3 and release check pass.

hqhq1025 · 2026-07-12T07:25:41Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e9a8294c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-12T07:29:19Z

+        if cdp_port.is_some() || target_url_contains.is_some() {
+            anyhow::bail!(
+                "targeted execute_javascript is not implemented on this platform's page backend"
+            );


Implement cdp_port for non-macOS execute_javascript

On Windows and Linux, PageTool is registered with WindowsPageBackend/LinuxPageBackend, and those impls still only implement execute_javascript while their CDP paths read CUA_DRIVER_CDP_PORT. Because the new dispatcher always calls execute_javascript_targeted and this default bails whenever cdp_port is present, a call using the newly advertised execute_javascript cdp_port never reaches those CDP fallbacks; it fails with “not implemented” even though the platform can evaluate JS via CDP. Please either override this method in those backends to consume cdp_port or avoid advertising/accepting the option there.

Useful? React with 👍 / 👎.

Fixed in 35fa5658. The shared cua_driver_core::cdp helper now supports unique case-insensitive URL targeting, and both Windows and Linux override execute_javascript_targeted to consume explicit cdp_port / target_url_contains. Calls without explicit targeting still use the existing bookmark/env behavior.

Validation:

core CDP selection tests: 2/2

core page tests: 3/3

macOS CDP tests: 6/6

macOS release check: passed

cargo check -p platform-windows --target x86_64-pc-windows-gnu: passed

cargo check -p platform-linux: passed in a clean Debian bookworm / Rust 1.92 container with X11/Wayland build dependencies.

The local macOS-to-Linux cross-check was initially blocked before our code by missing pkg-config sysroot; the native Linux container check above covers the actual backend.

hqhq1025 · 2026-07-12T07:41:08Z

@codex review

chatgpt-codex-connector · 2026-07-12T07:45:52Z

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 35fa565846

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

hqhq1025 · 2026-07-12T07:57:52Z

@f-trycua The PR workflows are currently blocked before job creation: all 8 runs report action_required with jobs=0, which indicates fork-contributor workflow approval is required. Could you please click Approve and run workflows on the Actions prompt?

For reference, I have already run the relevant checks independently: core page 3/3, core CDP 2/2, macOS CDP 6/6, macOS release check, Windows cross-check, and native Linux check in Debian Bookworm / Rust 1.92 all pass.

f-trycua · 2026-07-12T11:40:15Z

Thanks @hqhq1025 for the careful implementation, thorough validation, and quick follow-through on the review feedback. I also checked this against our current cua-driver E2E work: the branches merge cleanly, and the combined CDP code preserves both the readiness polling and exact target selection. Merged!

Salvaged from #2166 Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>

Represent Linux Chromium JavaScript-dialog resolution as an explicit foreground capability with a side-effect-free background refusal. Stabilize prepared-browser binding and keep the canonical matrix and public docs aligned. Salvaged from #2166 Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>

Complete exact browser tab state, typed JavaScript dialogs, file uploads, pointer-action parity, approval-gated downloads, telemetry, documentation, and canonical cross-platform evidence. Salvaged from #2166 Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>

Add exact-tab JavaScript dialog capabilities and ref-targeted CDP file input assignment. Dialog state retains only bounded kind and generation; uploads validate direct regular files and omit paths from results.\n\nSalvaged from trycua#2166\nRefs trycua#2240 trycua#2320\n\nCo-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>

fix(cua-driver): target page JavaScript exactly

68a42c0

hqhq1025 force-pushed the codex/targeted-page-execute branch from adef3e8 to 68a42c0 Compare July 12, 2026 04:42

hqhq1025 marked this pull request as ready for review July 12, 2026 04:43

chatgpt-codex-connector Bot reviewed Jul 12, 2026

View reviewed changes

fix(cua-driver): reuse targeted CDP sessions

5c2b402

chatgpt-codex-connector Bot reviewed Jul 12, 2026

View reviewed changes

fix(cua-driver): avoid replaying page scripts

0e9a829

chatgpt-codex-connector Bot reviewed Jul 12, 2026

View reviewed changes

fix(cua-driver): route targeted JS cross-platform

35fa565

f-trycua merged commit fb5bc19 into trycua:main Jul 12, 2026

This was referenced Jul 12, 2026

Computer Use: post-foundation roadmap for state, lifecycle, and real-model reliability apache/maka#836

Closed

feat(cua-driver): add stable AX node identity tokens #2210

Closed

cua-release-bot Bot mentioned this pull request Jul 17, 2026

chore(main): release cua-driver-rs 0.9.0 #2289

Merged

f-trycua mentioned this pull request Jul 18, 2026

feat(cua-driver): complete the browser action surface #2323

Merged

f-trycua pushed a commit that referenced this pull request Jul 18, 2026

docs(cua-driver): preserve source contribution credit

00f5905

Salvaged from #2166 Co-authored-by: Haoqing Wang <78337154+hqhq1025@users.noreply.github.com>

hqhq1025 mentioned this pull request Jul 28, 2026

fix(cua-driver): carry full 32-bit snapshot generation in element tokens #2608

Merged

cua-release-bot Bot mentioned this pull request Jul 28, 2026

chore(main): release cua-driver-rs 0.13.1 #2643

Merged

cua-release-bot Bot mentioned this pull request Aug 5, 2026

chore(main): release cua-driver-rs 0.19.0 #2912

Merged

Uh oh!

Uh oh!

Conversation

hqhq1025 commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Validation on latest upstream main

Real multi-window downstream validation

Compatibility

Uh oh!

hqhq1025 commented Jul 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 commented Jul 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 commented Jul 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

hqhq1025 commented Jul 12, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 12, 2026

Uh oh!

hqhq1025 commented Jul 12, 2026

Uh oh!

f-trycua commented Jul 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hqhq1025 commented Jul 11, 2026 •

edited

Loading