Skip to content

feat(cua-driver): expose post-action window rebind targets - #2746

Closed
f-trycua wants to merge 1 commit into
mainfrom
codex/2238-popup-rediscovery
Closed

feat(cua-driver): expose post-action window rebind targets#2746
f-trycua wants to merge 1 commit into
mainfrom
codex/2238-popup-rediscovery

Conversation

@f-trycua

@f-trycua f-trycua commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a closed, platform-neutral window_change action-result record with newly discovered native window ids, pids, app names, titles, and foreground-change state
  • recommend rebind after any macOS pointer, keyboard, value, or scroll action discovers a new interaction surface, including an exact target when only one window appeared
  • preserve the existing read-only window diff and wildcard focus suppression; rediscovery never activates or raises the new surface
  • regenerate the portable manifest and Python/TypeScript UniFFI bindings, and document rebind-before-pixel/page/foreground ownership

Why

The macOS detector already observed popups and new windows after native actions, but #2713 narrowed successful actions to the shared ActionResult and retained those target identifiers only in diagnostic prose. Callers therefore remained anchored to a parent window that a modal or popup could block. This change projects the decision-critical topology facts through the same closed contract for every wrapped native action rather than special-casing browser tools.

Malformed producer topology fails closed at the canonical adapter. When multiple windows appear, the result retains every candidate and recommends rebind without guessing an exact target.

Validation

  • cargo test --release -p cua-driver-contract -j 1 (26 unit tests + generator test)
  • cargo test --release -p cua-driver-core action_record::tests --lib -j 1 (20 passed)
  • cargo test --release -p platform-macos window_change_detector::tests --lib -j 1 (10 passed)
  • node scripts/generate-uniffi-bindings.mjs --check
  • npm run typecheck
  • python3 -m py_compile src/cua_driver/__init__.py src/cua_driver/_native_contract.py
  • cargo fmt --all -- --check
  • git diff --check

Scope

Addresses the structured post-action rediscovery and focus-preservation portion of #2238. The separate privacy-sensitive, request-scoped desktop-frame fallback proposed there is not included in this draft; ambiguous structured results remain recoverable through list_windows without widening capture scope.

Relates to #2238.

@f-trycua
f-trycua force-pushed the codex/2238-popup-rediscovery branch from 2247850 to ef1e30a Compare July 31, 2026 23:44
@f-trycua
f-trycua marked this pull request as ready for review August 1, 2026 00:58
injaneity added a commit that referenced this pull request Aug 25, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
injaneity added a commit that referenced this pull request Aug 26, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
injaneity added a commit that referenced this pull request Aug 27, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
@injaneity

Copy link
Copy Markdown
Collaborator

closing as superseded by #3373

@injaneity injaneity closed this Aug 27, 2026
injaneity added a commit that referenced this pull request Aug 28, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
injaneity added a commit that referenced this pull request Aug 28, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
injaneity added a commit that referenced this pull request Sep 4, 2026
Salvaged from #2746.

Co-authored-by: Francesco Bonacci <195596869+f-trycua@users.noreply.github.com>

Co-authored-by: Zane Chee <zanechee@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants