Skip to content

fix(cua-driver)(windows): list empty-/null-title top-level windows (WPF, borderless, custom-chrome) (#2020) - #2021

Open
LaZzyMan wants to merge 2 commits into
trycua:mainfrom
LaZzyMan:fix/windows-list-empty-title-windows
Open

fix(cua-driver)(windows): list empty-/null-title top-level windows (WPF, borderless, custom-chrome) (#2020)#2021
LaZzyMan wants to merge 2 commits into
trycua:mainfrom
LaZzyMan:fix/windows-list-empty-title-windows

Conversation

@LaZzyMan

@LaZzyMan LaZzyMan commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Closes #2020.

Draft / direction check. This changes the Windows list_windows
enumeration contract, and I can't run the Windows test suite locally
(platform-windows needs an MSVC toolchain + a real interactive desktop;
see Verification below). Opening as a draft to confirm the direction —
replace the non-empty-title proxy with an explicit "is this a real window?"
predicate — before asking for a real-hardware regression run.

Problem

On Windows, list_windows silently dropped any visible top-level window whose
title is empty/null. Because get_window_state, click, scroll, etc. all
resolve a pid → windows through crate::win32::list_windows, such a window was
completely untargetable by the agent:

  • debug_window_info {"pid": <pid>} listed the window (e.g. a WPF
    Ruler.Wpf.exe, class HwndWrapper[App.exe;;<guid>], hwnd 393542, title
    null).
  • list_windows {"pid": <pid>} returned an empty array.
  • get_window_state {"window_id": 393542}"No window with window_id … exists".
  • Every following click / scroll"No windows found for pid <pid>".

WPF, borderless, and custom-chrome apps legitimately ship empty-caption
top-level windows, so this is a hard wall for a whole class of apps.

Root cause

Both enumeration sources hard-filtered on a non-empty title before the window
could reach the merged list:

  • EnumWindows path — enum_windows_cb, win32/windows.rs (title_len == 0 → skip).
  • UI Automation path — window_info_from_uia_element, uia/windows_enum.rs
    (title_len == 0 → None).

The module doc-comment even stated it as the contract: "Both sources apply the
same filters (visible, non-iconic, non-empty title)."
This non-empty-title gate
was introduced as "filter parity" in #1542 (whose actual goal was the opposite —
UWP/WebView2 frames returning an empty array), and was never reconsidered for
honest untitled windows. Meanwhile debug_window_info (#1597) enumerated with a
looser filter (visible + no owner, title ignored), which is why the window showed
up there but nowhere else.

Fix

Replace the title proxy with an explicit predicate for "real, targetable
top-level window", shared by both enumeration sources so they can't drift
again (the drift was the root cause):

pub(crate) fn is_listable_top_level(hwnd: HWND) -> bool {
    // visible && !iconic && owner-less (GW_OWNER null) && !DWM-cloaked
}
  • GW_OWNER null → excludes tool-tips / owned pop-ups / transient children
    (the EnumWindows path previously had no owner check — this is what lets us
    safely drop the title gate without admitting noise). Mirrors the top-level
    test debug_window_info already uses, so the two tools now agree.
  • DWMWA_CLOAKED == 0 → excludes suspended-UWP / ApplicationFrameHost
    background frames that keep WS_VISIBLE but aren't on screen (the noise class
    feat(list_windows): UIA-first top-level window enumeration on Windows #1542 cared about).
  • Title is now read for display only via a shared window_title helper; empty
    caption → empty string. The tool layer already renders (no title) for these
    records, so nothing downstream needed to change.

Touches: win32/windows.rs (predicate + helpers + enum_windows_cb),
uia/windows_enum.rs (reuse the predicate, drop the title gate),
PARITY.md (enumeration-source contract).

Tests

Added empty_title_top_level_window_is_listed in win32/windows.rs — creates a
real visible, owner-less, empty-caption top-level window via CreateWindowExW
and asserts it appears in list_windows(Some(own_pid)) with an empty title. It's
#[ignore] (needs an interactive window station), matching the existing Windows
GUI tests; run via the sandbox harness runner or
cargo test -p platform-windows -- --ignored empty_title.

Verification status

  • cargo check -p platform-windows --target x86_64-pc-windows-gnu — clean (lib).
  • cargo check … --tests — clean, including the new CreateWindowExW-based
    test module
    (the part I was least sure about, since it exercises Win32 API
    signatures directly). The only warning is a pre-existing unused import in
    impl_.rs, unrelated to this change.
  • ✅ Static review against existing call sites (overlay.rs for
    CreateWindowExW/RegisterClassExW, impl_.rs for
    GetWindow(GW_OWNER).unwrap_or_default().is_invalid(), get_window_bounds
    for the DwmGetWindowAttribute shape).
  • ⚠️ Not run on real Windows. Cross-check proves it compiles for the
    Windows target, but I have no rig / interactive desktop to execute the new
    #[ignore] test or run a real-desktop regression. The owner/cloaked gates'
    effect on real noise windows still needs a hardware run — ideally the same
    folks helping with Help wanted: validate multi-monitor (negative-X origin) fix on real Windows hardware — we have no local rig #1981.

Happy to adjust the predicate (e.g. add a non-zero-area check, or keep/drop the
cloaked gate) based on what a real run shows.

Summary by CodeRabbit

  • Bug Fixes
    • Top-level windows with empty titles are now included in window listings when they meet the usual visibility and desktop-state checks.
    • Window enumeration is now more consistent across the app, reducing cases where detectable windows were hidden from lists.
    • Displayed window titles can now be blank instead of acting as a filter, improving accuracy for untitled windows.

…rycua#2020)

list_windows dropped any visible top-level window with an empty caption
because both enumeration sources (EnumWindows + UIA) filtered on a
non-empty title. WPF (HwndWrapper[...]), borderless and custom-chrome
apps were therefore untargetable: get_window_state, click and scroll all
resolve windows through list_windows, so they failed with "No window with
window_id" / "No windows found for pid" even though debug_window_info
could still see the window.

Replace the non-empty-title proxy with a shared is_listable_top_level
predicate (visible + non-iconic + owner-less + non-DWM-cloaked) used by
both enumeration sources so they can't drift apart again. The owner check
(GW_OWNER null) is what lets us drop the title gate without admitting
noise; the EnumWindows path previously had no owner check at all. The
title is now read for display only via a shared window_title helper;
empty captions are listed (the tool layer already renders "(no title)").

Add an #[ignore] regression test that creates a real empty-title
top-level window and asserts list_windows enumerates it. Cross-checked
with cargo check --target x86_64-pc-windows-gnu (lib + tests); not yet
run on real Windows hardware.
@vercel

vercel Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@LaZzyMan is attempting to deploy a commit to the Cua Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4d4dd017-4600-4490-be5b-f74d93f34470

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Windows top-level window enumeration now includes empty-caption HWNDs when they pass the shared visibility, ownership, and cloaking checks. The Win32 and UIA paths both use shared helpers, a regression test covers an untitled top-level window, and PARITY.md reflects the updated contract.

Changes

Windows top-level window listability parity

Layer / File(s) Summary
Shared Win32 listability contract
libs/cua-driver/rust/crates/platform-windows/src/win32/windows.rs, libs/cua-driver/rust/PARITY.md
list_windows now uses is_listable_top_level and window_title, empty captions are kept as display data, and a regression test covers an untitled visible top-level window.
UIA listability gate
libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rs
window_info_from_uia_element now gates on is_listable_top_level, reads titles through the shared helper, and updates its comments and imports for empty-caption windows.

Sequence Diagram(s)

sequenceDiagram
  participant list_windows
  participant enum_windows_cb
  participant is_listable_top_level
  participant window_info_from_uia_element
  participant window_title
  list_windows->>enum_windows_cb: enumerate top-level HWNDs
  enum_windows_cb->>is_listable_top_level: filter visible, owner-less, non-cloaked HWNDs
  enum_windows_cb->>window_title: read caption for display
  window_info_from_uia_element->>is_listable_top_level: gate resolved HWND
  window_info_from_uia_element->>window_title: read caption for display
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • trycua/cua#1542: Introduced the Windows UIA/top-level enumeration filtering that this PR revises by allowing empty-caption HWNDs through the shared listability check.

Poem

I hopped where empty titles used to hide,
ახლა they sit in the list with rabbit pride.
One shared gate, one caption path,
and thump-thump windows join the path. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the Windows fix for listing empty/null-title top-level windows and the affected window types.
Linked Issues check ✅ Passed The PR matches #2020 by removing title-based filtering, aligning Win32/UIA enumeration, updating docs, and adding a regression test.
Out of Scope Changes check ✅ Passed The changes are scoped to the Windows window-listing fix, with supporting refactors and tests that directly serve that goal.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@LaZzyMan
LaZzyMan marked this pull request as ready for review June 25, 2026 03:23
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@libs/cua-driver/rust/crates/platform-windows/src/win32/windows.rs`:
- Around line 294-330: The test window teardown in the CreateWindowExW flow is
not panic-safe because the precondition assertions in the window_title and
is_listable_top_level checks can panic before DestroyWindow runs. Add a cleanup
guard around the hwnd lifetime in this test path so the window is always
destroyed even if assert_eq! or assert! fails, and keep the existing
list_windows verification using hwnd as the unique handle reference.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f29cc920-5271-4879-a5d2-cea21f902451

📥 Commits

Reviewing files that changed from the base of the PR and between 9e84c1d and d2027f8.

📒 Files selected for processing (3)
  • libs/cua-driver/rust/PARITY.md
  • libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rs
  • libs/cua-driver/rust/crates/platform-windows/src/win32/windows.rs

Comment thread libs/cua-driver/rust/crates/platform-windows/src/win32/windows.rs Outdated
…-safe

Wrap the test HWND in a RAII guard so DestroyWindow runs even if a precondition assertion panics before the explicit teardown. Addresses CodeRabbit review feedback on trycua#2021.
LaZzyMan added a commit to QwenLM/qwen-code that referenced this pull request Jun 26, 2026
Ports the fix from upstream trycua/cua#2021 into the vendored driver.

list_windows filtered out any top-level window whose title was empty or null,
so legitimate targets (splash screens, some Electron/game windows, tool
windows) were invisible to the agent and unclickable. Include empty-title
windows, using class name / process as a fallback label.

(platform-windows crate is not built on macOS; verified by clean upstream
apply and covered by upstream + release-workflow Windows CI.)
LaZzyMan added a commit to QwenLM/qwen-code that referenced this pull request Jun 26, 2026
The vendored copy is actually at cua-driver-rs-v0.6.7 (workspace version and
all 0.6.7->0.6.8 delta files confirm it), but .vendored-from had drifted to
0.6.8 during an earlier sync-script trial whose code delta was not kept. Left
as-is it would make a future sync diff 0.6.8->newer and silently skip the real
0.6.7->0.6.8 fixes. Correct it back to 0.6.7.

Also record the four not-yet-merged upstream PRs we carry as cherry-picks
(trycua/cua#2021/#2025/#2035/#2036) in .vendored-patches.md, and have
sync-from-upstream.sh point at it so the next sync reconciles them.
shenyankm pushed a commit to shenyankm/qwen-code that referenced this pull request Jun 26, 2026
…coordinates (QwenLM#5896)

* feat(cua-driver): vendor trycua/cua driver with 1000-normalized coordinate support

Vendor libs/cua-driver from trycua/cua into packages/cua-driver as the
basis for qwen-code's computer-use backend, adding an opt-in relative
(1000x1000 normalized) coordinate mode for Qwen-VL clients.

- coord_norm.rs: 0-1000 <-> pixel conversion, per-(pid,window_id) size
  cache, tools/list description rewrite (TDD, 27 tests)
- ToolRegistry: normalized field + invoke input/output hooks
- protocol.rs: system-instruction coordinate wording switched by mode
- serve.rs: daemon list path description rewrite (input_schema aware)
- main.rs: CUA_DRIVER_RS_COORDINATE_SPACE env seed

Default coordinate_space=pixels => zero behavior change for existing
pixel clients. Set CUA_DRIVER_RS_COORDINATE_SPACE=normalized_1000 to
enable. Excludes rust/target build output.

* feat(cua-driver): make normalized coordinate scale configurable

Add CUA_DRIVER_RS_COORDINATE_SCALE (default 1000) so the normalization
full-scale can absorb the Qwen 999-vs-1000 cookbook ambiguity without a
recompile. norm_to_px/px_to_norm now take an explicit scale; denormalize_args
reads the process-wide COORDINATE_SCALE seeded once at startup from env.

* ci(cua-driver): add cross-platform release workflow for vendored driver

Standalone GitHub Action that builds, signs, and releases the vendored
cua-driver under packages/cua-driver. Adapted from upstream trycua/cua
cd-rust-cua-driver.yml:

- macOS: universal binary (lipo arm64+x86_64), codesigned + notarized into
  CuaDriver.app using qwen-code's existing secrets (MAC_CSC_LINK cert +
  App Store Connect API key notarization); Developer ID identity is
  auto-discovered from the imported cert.
- Linux: x86_64 + arm64, built in debian:11 for a glibc 2.31 floor.
- Windows: x86_64 + arm64, unsigned (no EV cert, matches upstream).
- Release: softprops/action-gh-release on cua-driver-rs-v* tags or manual
  dispatch, prerelease.

Triggered by tag push (cua-driver-rs-v*) or workflow_dispatch.

* chore(cua-driver): rebrand vendored driver as qwen-cua-driver

Rename the vendored trycua/cua driver so the fork installs and runs
independently of any upstream trycua install:
- binary cua-driver -> qwen-cua-driver
- bundle CuaDriver.app -> QwenCuaDriver.app
- bundle id com.trycua.driver -> com.qwencode.cua-driver

Updates the cargo/uia manifests, Info.plist, bundle/proxy launch paths,
permission/health-report wording, the install/build scripts, and the
cross-platform release workflow.

* feat(cua-driver): finish relative-coordinate mode — toggle, scale, zoom/move_cursor

- CUA_DRIVER_RS_COORDINATE_SPACE is now a 1/0 toggle (via is_env_truthy);
  default off keeps pixel mode byte-identical to upstream.
- Thread CUA_DRIVER_RS_COORDINATE_SCALE through every coordinate surface
  (was hardcoded 1000): input denormalization already used it; now the
  rewritten screenshot dims, the tool/param descriptions, and the agent
  instructions track the configured scale too.
- Normalize zoom (window basis) and move_cursor (screen basis) inputs and
  rewrite their descriptions, alongside click/double_click/right_click/drag.
- Fix zoom on downscaled (Retina) windows: apply the get_window_state resize
  ratio so the crop lands on the region the agent saw. Normalized mode only;
  pixel-mode zoom unchanged.

All coordinate behavior stays gated on the normalized flag, so the default
(pixels) path is unchanged from upstream.

* chore(cua-driver): add upstream-sync script (git subtree unusable here)

`git subtree split --prefix=libs/cua-driver` hangs on a commit deep in
trycua/cua's history, so the subtree add/pull workflow isn't usable for
the vendored driver (and a pull would re-split + re-hang every time).

Add scripts/sync-from-upstream.sh instead: it git-diffs two upstream refs
(never walks the full history, so it dodges the hang), reprefixes the
libs/cua-driver delta to packages/cua-driver, and `git apply --reject`s it
on top of our local changes — conflicts land as *.rej for manual fixup.
Record the vendored version in .vendored-from and document the migration +
sync method in the design doc.

* chore(cua-driver): exclude vendored driver from qwen-code ESLint

The vendored packages/cua-driver tree carries upstream JS (e.g. the
test-harness Electron app) that doesn't follow qwen-code's lint rules and
fails CI. It is not a workspace package (no package.json) and is not
qwen-code TypeScript, so add it to eslint.config.js global ignores —
alongside packages/desktop/** — the standard treatment for vendored code.

* fix(cua-driver): let start_session revive an idle-reaped session

Ports the fix from upstream trycua/cua#2035 into the vendored driver.

When a session is reaped for idleness, a subsequent start_session with the
same id failed instead of resuming it. Revive the ended session in place so
the agent can continue rather than getting a hard error.

* fix(cua-driver): retry daemon socket writes on EAGAIN

Ports the fix from upstream trycua/cua#2036 into the vendored driver.

A non-blocking daemon socket can return EAGAIN/EWOULDBLOCK mid-write when the
peer's receive buffer is momentarily full. The driver treated that as fatal
and dropped the connection. Add a bounded retry/poll loop (mirror of the
read-side socket_io helper) so transient back-pressure no longer kills the
session; only a real timeout or hard error fails the write.

* fix(cua-driver/linux): stop reporting bare "Clicked" for X11 synthetic clicks

Ports the fix from upstream trycua/cua#2025 into the vendored driver.

On X11, clicks are delivered via XSendEvent synthetic events, which many
toolkits (GTK/SDL/Allegro) ignore because send_event is set. The driver still
reported a flat success ("Clicked"), masking that nothing happened. Report
the synthetic-delivery caveat honestly so the agent can fall back instead of
assuming the click landed.

(platform-linux crate is not built on macOS; verified by clean upstream apply
and covered by upstream + release-workflow Linux CI.)

* fix(cua-driver/windows): list empty-/null-title top-level windows

Ports the fix from upstream trycua/cua#2021 into the vendored driver.

list_windows filtered out any top-level window whose title was empty or null,
so legitimate targets (splash screens, some Electron/game windows, tool
windows) were invisible to the agent and unclickable. Include empty-title
windows, using class name / process as a fallback label.

(platform-windows crate is not built on macOS; verified by clean upstream
apply and covered by upstream + release-workflow Windows CI.)

* chore(cua-driver): track cherry-picked upstream PRs; fix vendored-from

The vendored copy is actually at cua-driver-rs-v0.6.7 (workspace version and
all 0.6.7->0.6.8 delta files confirm it), but .vendored-from had drifted to
0.6.8 during an earlier sync-script trial whose code delta was not kept. Left
as-is it would make a future sync diff 0.6.8->newer and silently skip the real
0.6.7->0.6.8 fixes. Correct it back to 0.6.7.

Also record the four not-yet-merged upstream PRs we carry as cherry-picks
(trycua/cua#2021/QwenLM#2025/QwenLM#2035/QwenLM#2036) in .vendored-patches.md, and have
sync-from-upstream.sh point at it so the next sync reconciles them.

* ci(cua-driver): satisfy repo yamllint on the release workflow

The vendored-driver release workflow tripped 114 quoted-strings violations
under the repo's .yamllint (quote-type: single, required). Single-quote all
string scalars to match every other workflow in .github/workflows.

While reformatting, the release-notes body also got its paragraph blank lines
collapsed and still referenced the old CUA_DRIVER_RS_COORDINATE_SPACE=
normalized_1000 value — restore the blank lines and update it to the current
0/1 toggle (default 0 = off; optional CUA_DRIVER_RS_COORDINATE_SCALE=1000).

* chore(cua-driver): sync vendored driver to cua-driver-rs-v0.6.8

First real run of scripts/sync-from-upstream.sh: it 3-way-applied the upstream
0.6.7->0.6.8 delta onto our local fork. 10/12 files applied cleanly; the 2
rejects (install.ps1, _install-rust.sh) were already-applied baked-version
bumps (0.6.6->0.6.7, our copies were already at 0.6.7), i.e. no real conflict.

0.6.8 brings: Wayland input path (platform-linux), linux health_report +
overlay tweaks, a platform-macos build.rs step, and dependency bumps. Version
moved to 0.6.8 across the workspace.

Verified our work survived the sync untouched: the relative-coordinate shim
(coord_norm/protocol) and all four cherry-picked PRs (socket_io/session +
linux/windows) are intact — in particular the 0.6.8 edit to platform-linux
tools/impl_.rs landed alongside our QwenLM#2025 change with no collision. macOS
cargo check + 132 core tests green. (platform-linux/windows + the binary
integration test build only on their own runners; upstream CI covers those.)

* ci(cua-driver): add a dry_run gate to the release workflow

Mirror the desktop-release / release dry-run pattern: a workflow_dispatch
dry_run boolean input (default true). The cross-platform build + package jobs
always run and upload their artifacts; the GitHub Release job now publishes
only on a tag push or an explicit dry_run=false dispatch.

Lets us rehearse the whole build/package pipeline (dry_run=true, notarize=false)
and inspect the produced artifacts without cutting a release. A branch push
(no tag, not a dispatch) likewise builds without releasing.
@f-trycua

f-trycua commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Real, still-open bug (#2020) — main still drops empty-title top-level windows. The shared is_listable_top_level predicate is the right shape.

Blockers before merge: (1) it's conflicting against 0.7.0 — win32/windows.rs collides with the new UWP-host-resolution block, and PARITY.md was deleted on main (relocate or drop those doc edits); please rebase. (2) The new owner-less (GW_OWNER null) gate is a behavior change the old path never had — it could now drop owned-but-titled windows like modal "Save As" dialogs. That needs a real-Windows regression run to confirm it doesn't break dialog targeting. Rebase + validate and I'll take it. Keeping #2020 open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

3 participants