From e64209be0e77359801d84fb2e277ea72418defba Mon Sep 17 00:00:00 2001 From: Francesco Bonacci Date: Thu, 21 May 2026 14:52:12 +0200 Subject: [PATCH 1/2] docs(cua-driver): Windows behavior notes for the v0.2.9 fix chain + install.ps1 PS 5.1 workaround Tonight's three cua-driver-rs Windows fixes (#1620 Chromium anti-throttling flag auto-inject in `launch_app`, #1621 control-type whitelist for the `click(x, y)` UIA Invoke pre-check, #1623 SendInput routing for Chromium coord clicks) shipped in v0.2.9 without docs updates. This PR closes that gap and documents the install.ps1 PS 5.1 parse bug as a known issue. ## mcp-tools.mdx - New top-level section `## Windows behavior notes` at the end of the reference, gathering the three cross-cutting changes: - `launch_app` Chromium flag list + the 10 detected browser executables - `click(x, y)` control-type whitelist (Button / MenuItem / Hyperlink / TabItem / ListItem / CheckBox / RadioButton / SplitButton / TreeItem) + why canvases / Panes / Customs fall through - SendInput on Chromium with brief foreground swap + cursor jump, the UIAccess requirement, and the `cua-driver-uia.exe` proxy default - `hotkey`'s SendInput-routed delivery + matching UIAccess constraint - Inline cross-references from `click`, `launch_app`, and `hotkey` pointing to the Windows behavior section so callers reading any of those tool entries see the platform-specific notes. ## installation.mdx - Callout under the Windows install one-liner documenting #1626 (PS 5.1 parse error on `install.ps1`) with the manual-zip workaround verbatim from the issue, scoped to PS 5.1 only (PS 7+ parses fine). Closes the standing /docs update obligation for #1619, #1620, #1621, #1623. Co-Authored-By: Claude Opus 4.7 --- .../guide/getting-started/installation.mdx | 17 +++++ .../docs/cua-driver/reference/mcp-tools.mdx | 64 +++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx index 2fb0dcdb0b..344005db7d 100644 --- a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx +++ b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx @@ -70,6 +70,23 @@ irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/in The legacy `libs/cua-driver-rs/scripts/install.{sh,ps1}` URLs continue to work via backward-compat shims that redirect to the canonical installers — Hermes and any other integration that hardcodes the old URLs keep working unchanged. + + **Known issue on Windows PowerShell 5.1**: `install.ps1` currently fails to parse on stock Windows PowerShell 5.1 (the version shipped with Windows 10 / Server 2019 / Server 2025) due to backtick + ampersand escaping in the post-install hint block — see [#1626](https://github.com/trycua/cua/issues/1626). Workaround until that lands: manual zip install: + + ```powershell + $version = "0.2.9" + $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x86_64' } + $url = "https://github.com/trycua/cua/releases/download/cua-driver-rs-v$version/cua-driver-rs-$version-windows-$arch-binary.zip" + iwr $url -OutFile $env:TEMP\cua.zip -UseBasicParsing + $dest = "$env:LOCALAPPDATA\Programs\trycua\cua-driver-rs\bin" + New-Item -ItemType Directory -Force -Path $dest | Out-Null + Expand-Archive -Force -Path $env:TEMP\cua.zip -DestinationPath $dest + [Environment]::SetEnvironmentVariable('Path', "$dest;" + [Environment]::GetEnvironmentVariable('Path','User'), 'User') + ``` + + PowerShell 7+ (`pwsh`) parses `install.ps1` correctly, so the one-liner works there. + + The Windows installer auto-detects host architecture (x64 / arm64) via `[System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture` and downloads the matching `cua-driver-rs--windows-{x86_64,arm64}.zip` from GitHub Releases. It runs **without admin** and **without Developer Mode**. diff --git a/docs/content/docs/cua-driver/reference/mcp-tools.mdx b/docs/content/docs/cua-driver/reference/mcp-tools.mdx index 0cc266a5bb..e7b6077f56 100644 --- a/docs/content/docs/cua-driver/reference/mcp-tools.mdx +++ b/docs/content/docs/cua-driver/reference/mcp-tools.mdx @@ -72,6 +72,14 @@ cache lookup). `action` is only valid with `element_index`; `count` and `modifier` are ignored in the AX path. +**Windows note:** the `x, y` path layers in a control-type +whitelist for the UIA Invoke pre-check (canvases / images +/ custom surfaces fall through to PostMessage to preserve +pixel precision) and routes through `SendInput` with a +brief foreground swap when the target HWND is a Chromium +frame. See [Windows behavior notes](#windows-behavior-notes) +below. + **Arguments:** - `action` (string, optional): AX action name (element_index path only). Default: press. @@ -375,6 +383,11 @@ up/down/left/right, space, delete, home, end, pageup, pagedown, f1-f12, letters, digits). Order: modifiers first, one non-modifier last. +**Windows note:** uses `SendInput` (not PostMessage) so +the OS modifier state is correctly updated for accelerator +dispatch. See [Windows behavior notes](#windows-behavior-notes) +below. + **Arguments:** - `keys` (array of string, required): Modifier(s) and one non-modifier key, e.g. ["cmd", "c"]. @@ -433,6 +446,11 @@ Example: `{"bundle_id": "com.apple.Safari", "urls": ["about:blank"]}`. Electron apps also follow this contract when their entry point depends on a URL argument. +**Windows note:** when launching a Chromium-based browser, +the driver auto-injects three anti-throttling flags so the +hidden renderer doesn't suspend itself. See +[Windows behavior notes](#windows-behavior-notes) below. + Optional `electron_debugging_port` launches an Electron app with `--remote-debugging-port=`, activating its Chrome DevTools Protocol (CDP) on that port. This gives the `page` @@ -1124,3 +1142,49 @@ session so the resize ratio is known. ```json {"pid":844,"x1":100,"x2":100,"y1":200,"y2":200} ``` + +--- + +## Windows behavior notes + +The tool reference above is written against the macOS reference impl. cua-driver-rs (`v0.2.9+`) layers in several Windows-only behaviors that callers need to know about to drive Chromium-based browsers and custom-drawn canvases correctly. + +### `launch_app` — Chromium anti-throttling flags auto-injected + +When `path` / `name` / `launch_path` resolves to a Chromium-based browser (`msedge`, `chrome`, `brave`, `vivaldi`, `opera`, `chromium`, `arc`, `thorium`, `iridium`, `browser` (Yandex), or a `Chrome_WidgetWin_*` / `CefBrowser*`-class Electron app), the daemon transparently prepends these three flags to `additional_arguments`: + +``` +--disable-features=CalculateNativeWinOcclusion +--disable-backgrounding-occluded-windows +--disable-renderer-backgrounding +``` + +Without them, Chromium's occlusion logic suspends the renderer process because the window is launched non-foreground via `SW_SHOWNOACTIVATE` — the UIA tree returns only browser chrome (no page DOM) and `PrintWindow` returns a blank body. With them, the page renders + exposes its DOM via UIA while the window stays hidden. + +If you already pass `--disable-features=...`, the driver merges `CalculateNativeWinOcclusion` into your existing list instead of duplicating the flag. + +**Also worth knowing**: Chromium auto-disables page-level accessibility unless an AT-grade client probes the renderer. For UIA-based callers that want the rendered DOM in `get_window_state`, additionally pass `--force-renderer-accessibility` yourself — that's a page-level concern the driver doesn't auto-inject. + +### `click` — control-type whitelist for the UIA Invoke pre-check (`x, y` path) + +The `x, y` dispatch tries UIA Invoke first (no focus steal), then falls through to PostMessage / SendInput. The UIA Invoke step only fires for elements with a **coord-independent primary action**: + +`Button` · `MenuItem` · `Hyperlink` · `TabItem` · `ListItem` · `CheckBox` · `RadioButton` · `SplitButton` · `TreeItem` + +For canvases, paint surfaces, image maps, and custom widgets (control types `Image` / `Pane` / `Custom` / `Document` / `Group`), `click(x, y)` skips UIA Invoke even when the element advertises `InvokePattern` — invoking those fires the element's default action at the geometric centre, ignoring the requested coords. The click falls through to PostMessage or (for Chromium targets) SendInput, preserving pixel precision. + +### `click(x, y)` on Chromium — SendInput with brief foreground swap + +`PostMessage(WM_LBUTTONDOWN/UP)` on a Chromium HWND doesn't reach the DOM input pipeline (Chromium's input thread requires SendInput-queue origin). When the target HWND class is `Chrome_WidgetWin_*` or `CefBrowser*`, `click(x, y)` routes through `SendInput` with a brief `SetForegroundWindow(target)` + `SetCursorPos(x, y)` + restore-previous-foreground dance. + +Trade-off: **the cursor visibly moves to the click point and the target briefly becomes foreground**. There's no Chromium-native path that delivers coord clicks to the DOM without these side effects short of launching with `--remote-debugging-port` and dispatching via the `page` tool (CDP). + +UIAccess requirement: `SetForegroundWindow` from a non-UIAccess daemon is rejected by Win32, in which case the tool returns the diagnostic `"SendInput inserted only 0 of 3 mouse events..."` and recommends running through the `cua-driver-uia.exe` worker. The MCP proxy auto-prefers the UIA worker's pipe when both daemons are running, so the SendInput path works out-of-the-box on a normal install. + +**When SendInput is unwanted** (zero focus steal required): use `element_index` clicks where the page exposes the target via UIA. Buttons / links / menu items in Chromium pages take the UIA Invoke path and don't incur the focus swap. + +### `hotkey` — SendInput-routed for OS modifier-state propagation + +The hotkey tool synthesises keystrokes via `SendInput` with a brief foreground swap (same mechanism as the Chromium `click`), so the OS modifier state visible to `GetKeyState` / `GetAsyncKeyState` is correctly updated for the duration of the keystroke. PostMessage-based paths put `WM_KEYDOWN(VK_CONTROL)` in the target's message queue but **don't** update the system modifier state — `TranslateAccelerator` then fails to recognise Ctrl+S as a Save accelerator and the `s` arrives as plain text. + +Same UIAccess constraint as Chromium clicks: `SetForegroundWindow` is restricted from non-UIAccess processes, so callers should funnel hotkey calls through the `cua-driver-uia.exe` worker. The MCP proxy already does this when both pipes are up. From 15d9e19bfcce2b7b1e8b1bd9388e9e9afa1f1747 Mon Sep 17 00:00:00 2001 From: Francesco Bonacci Date: Thu, 21 May 2026 16:09:23 +0200 Subject: [PATCH 2/2] docs(cua-driver): make manual-zip PATH update idempotent (CodeRabbit on #1627) Re-running the manual-install workaround duplicated `$dest` in the User PATH because the snippet unconditionally prepended. Guards with a `-notcontains` check before `SetEnvironmentVariable` so the entry is added at most once. Co-Authored-By: Claude Opus 4.7 --- .../docs/cua-driver/guide/getting-started/installation.mdx | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx index 344005db7d..6b6658524f 100644 --- a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx +++ b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx @@ -81,7 +81,12 @@ The legacy `libs/cua-driver-rs/scripts/install.{sh,ps1}` URLs continue to work v $dest = "$env:LOCALAPPDATA\Programs\trycua\cua-driver-rs\bin" New-Item -ItemType Directory -Force -Path $dest | Out-Null Expand-Archive -Force -Path $env:TEMP\cua.zip -DestinationPath $dest - [Environment]::SetEnvironmentVariable('Path', "$dest;" + [Environment]::GetEnvironmentVariable('Path','User'), 'User') + # Idempotent User PATH update — only add $dest if not already present. + $userPath = [Environment]::GetEnvironmentVariable('Path', 'User') + $parts = @($userPath -split ';' | Where-Object { $_ -ne '' }) + if ($parts -notcontains $dest) { + [Environment]::SetEnvironmentVariable('Path', "$dest;$userPath", 'User') + } ``` PowerShell 7+ (`pwsh`) parses `install.ps1` correctly, so the one-liner works there.