docs: update imports and install commands to use cua metapackage - #1228
Merged
Conversation
- Replace `from cua_sandbox import` / `from agent import` / `from agent.tools import` / `from agent.callbacks import` with `from cua import` - Replace `pip install cua-sandbox` / `pip install cua-agent[...]` with `pip install cua[...]` - Replace all-caps CUA (brand name) with Cua in prose (preserving env vars and CUA-Bench) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR updates documentation and configuration across the repository to reflect a unified package consolidation: replacing separate Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
… imports Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gent Those versions were published as cua-agent, not cua. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows `from cua import QEMURuntime, TartRuntime` etc. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cua.runtime re-exports all cua_sandbox.runtime classes (QEMURuntime, TartRuntime, etc.) - cua.tools re-exports agent.tools + ToolError/IllegalArgumentError from agent.types - Remove runtime symbols from cua top-level __init__ - Fix docs: from cua import BrowserTool/BaseTool/ToolError → from cua.tools import ... - Fix docs: from cua import TartRuntime/QEMURuntime → from cua.runtime import ... Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cua.callbacks re-exports all agent.callbacks handlers - Fix docs/cua/guide/fundamentals/callbacks.mdx to use from cua.callbacks import ... Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ndows tabs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (6)
docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdx (1)
97-104:⚠️ Potential issue | 🟡 MinorIncorrect suggested fix: requirements should use
cua[omni], notcua[som].The inconsistency is valid—requirements.txt uses old separate packages (
cua-agent,cua-sandbox,cua-som) while imports use the unifiedcuametapackage API (from cua import ...). However, the suggested fix is incorrect: thecuametapackage does not have asomextra. The correct extra for OmniParser support isomni.Corrected fix
```text -cua-agent -cua-sandbox -cua-som # OmniParser for GUI element detection +cua[omni] # Includes OmniParser for GUI element detection litellm>=1.0.0 python-dotenv>=1.0.0 google-cloud-aiplatform>=1.70.0</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against the current code and only fix it if needed.
In
@docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdx
around lines 97 - 104, Replace the three separate package entries "cua-agent",
"cua-sandbox", and "cua-som" in the requirements block with the unified
metapackage entry "cua[omni]" (which provides OmniParser support) so imports
like "from cua import ..." match the installed package; update the requirements
list around the block containing litellm, python-dotenv, and
google-cloud-aiplatform accordingly.</details> </blockquote></details> <details> <summary>docs/content/docs/cua/examples/automation/form-filling.mdx (1)</summary><blockquote> `27-31`: _⚠️ Potential issue_ | _🟠 Major_ **Requirements snippet should install `cua` to match the imports throughout the file.** Lines 28-29 list `cua-agent` and `cua-sandbox`, but all code examples import from the unified `cua` package. Users following this documentation would install the wrong packages. <details> <summary>Proposed fix</summary> ```diff -cua-agent -cua-sandbox +cua python-dotenv>=1.0.0🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/examples/automation/form-filling.mdx` around lines 27 - 31, The requirements code block currently lists cua-agent and cua-sandbox but the examples import from the unified cua package; update the snippet so it installs the cua package (e.g., add "cua" with an appropriate version specifier) to match the imports used in the examples and remove or adjust the separate cua-agent/cua-sandbox entries so installed packages align with the example import statements referencing "cua".docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdx (1)
63-65:⚠️ Potential issue | 🟠 MajorUpdate dependency snippet to match the unified
cuapackage imports used in the examples.Lines 63-64 still show split packages (
cua-agent,cua-sandbox), while the code examples import from the unifiedcuapackage (lines 154-155, 266-267, 345-346). This inconsistency will cause installation/import mismatches for users.🔧 Proposed fix
-cua-agent -cua-sandbox +cua python-dotenv>=1.0.0🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdx` around lines 63 - 65, The dependency list currently lists split packages `cua-agent` and `cua-sandbox`, which conflicts with the examples that import from the unified `cua` package; update the snippet to replace `cua-agent` and `cua-sandbox` with the single unified dependency `cua` (e.g., `cua>=<appropriate-version>`) while keeping `python-dotenv>=1.0.0`, so the installation matches the import statements used in the examples that import from `cua`.docs/content/docs/cua/guide/sandbox/lifecycle.mdx (1)
146-146:⚠️ Potential issue | 🟠 MajorFix the broken internal link at Line 146.
docs:check-linksis failing because/cua/guide/get-started/self-hosted-sandboxesresolves to not-found.🔧 Proposed docs link fix
-See [Self-Hosted Sandboxes](/cua/guide/get-started/self-hosted-sandboxes) for local setup instructions. +See [Using the Sandbox SDK](/cua/guide/get-started/using-sandbox-sdk) for local setup instructions.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/sandbox/lifecycle.mdx` at line 146, The internal link "Self-Hosted Sandboxes" currently points to a non-existent path (/cua/guide/get-started/self-hosted-sandboxes); update that href to the correct existing doc slug (or use the proper relative path) so docs:check-links passes. Locate the link text "Self-Hosted Sandboxes" in the file and replace the broken target (/cua/guide/get-started/self-hosted-sandboxes) with the correct URL for the self-hosted sandboxes page, then run docs:check-links to verify.docs/content/docs/cua/guide/advanced/custom-tools.mdx (1)
32-32:⚠️ Potential issue | 🟠 MajorUpdate legacy
agent.toolsimport to usecuanamespace.Line 32 uses
from agent.tools.browser_tool import BrowserTool, which conflicts with the metapackage migration. Update to the new namespace.Suggested fix
-from agent.tools.browser_tool import BrowserTool +from cua import BrowserTool🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/advanced/custom-tools.mdx` at line 32, Update the legacy import for BrowserTool to the new cua namespace: replace the old import that references agent.tools.browser_tool with the new module path that exposes the BrowserTool class (i.e., import BrowserTool from cua.tools.browser_tool) so any usage of BrowserTool in this file continues to work with the metapackage migration.docs/content/docs/cua/guide/fundamentals/callbacks.mdx (1)
172-173:⚠️ Potential issue | 🟠 MajorUpdate custom callback examples to use
cua.callbacksimport.Lines 172 and 217 import
AsyncCallbackHandlerfromagent.callbacks.base, inconsistent with other callback imports in this file. The correct import path isfrom cua.callbacks import AsyncCallbackHandlerto match the examples at lines 14, 48, 72, 99, 122, and 149.Corrected fix
-from agent.callbacks.base import AsyncCallbackHandler +from cua.callbacks import AsyncCallbackHandlerApply to both line 172 and line 217.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/fundamentals/callbacks.mdx` around lines 172 - 173, Replace the incorrect imports of AsyncCallbackHandler from agent.callbacks.base with the project-consistent import from cua.callbacks by changing any occurrences that read "from agent.callbacks.base import AsyncCallbackHandler" to "from cua.callbacks import AsyncCallbackHandler"; there are two such occurrences in the custom callback examples (the AsyncCallbackHandler imports used alongside other callback examples) so update both to ensure consistency with the rest of the file.
🧹 Nitpick comments (3)
docs/content/docs/cua/guide/integrations/laminar.mdx (1)
26-27: LGTM!The imports are correctly updated to use the
cuametapackage. Optionally, these could be consolidated into a single import statement.💡 Optional: Consolidate imports
-from cua import ComputerAgent -from cua import Sandbox, Image +from cua import ComputerAgent, Sandbox, Image🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/integrations/laminar.mdx` around lines 26 - 27, The imports for ComputerAgent, Sandbox, and Image are fine but can be consolidated; replace the two separate from cua import lines with a single import that lists the three symbols (ComputerAgent, Sandbox, Image) so use one import statement referencing those exact names to simplify and tidy the top of the module.docs/content/docs/cua/guide/sandbox/scale-out.mdx (1)
130-131: LGTM!The imports are correctly updated to use the
cuametapackage. Optionally, these could be consolidated into a single import statement.💡 Optional: Consolidate imports
-from cua import Sandbox, Image -from cua import ComputerAgent +from cua import Sandbox, Image, ComputerAgent🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/sandbox/scale-out.mdx` around lines 130 - 131, The current file imports Sandbox, Image, and ComputerAgent via two separate statements; consolidate them into a single import from the cua metapackage by combining the symbols (Sandbox, Image, ComputerAgent) into one import statement so the code reads like a single-line import of those three names from cua.docs/content/docs/cua/guide/get-started/what-is-cua.mdx (1)
10-11: Consider combining imports for conciseness.Both imports are from the same
cuamodule. While functionally correct, combining them would be more idiomatic:from cua import Sandbox, Image, ComputerAgent🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/content/docs/cua/guide/get-started/what-is-cua.mdx` around lines 10 - 11, Combine the two separate imports into a single, concise import statement by replacing the two lines that import Sandbox and Image and the separate line that imports ComputerAgent with one line that imports Sandbox, Image, and ComputerAgent together; update the lines referencing Sandbox, Image, and ComputerAgent so they continue to work with the consolidated import.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/content/docs/cua/examples/automation/post-event-contact-export.mdx`:
- Around line 137-138: The requirements section is inconsistent with the
imports: the code now imports ComputerAgent, Sandbox, and Image from the
top-level "cua" module (see imports "from cua import ComputerAgent" and "from
cua import Sandbox, Image"), but the requirements still list "cua-agent" and
"cua-sandbox"; update the requirements snippet to list the correct package(s)
that provide those symbols (replace "cua-agent" and "cua-sandbox" with "cua" or
the single package that exports ComputerAgent/Sandbox/Image), ensuring the
package name(s) match the import surface used in the examples.
In `@docs/content/docs/cua/reference/sandbox-sdk/index.mdx`:
- Line 859: Replace the legacy import "import cua_sandbox as cua" with the new,
consistent "from cua import ..." style used elsewhere (e.g., "from cua import
localhost" and other exported symbols); update any subsequent references that
rely on the old alias (calls like cua.foo) to either import the specific symbols
they use or import the module under a matching name (e.g., "from cua import
sandbox as cua" or "from cua import foo, bar") so all examples use the same
modern import pattern and remain copy-pasteable.
---
Outside diff comments:
In `@docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdx`:
- Around line 97-104: Replace the three separate package entries "cua-agent",
"cua-sandbox", and "cua-som" in the requirements block with the unified
metapackage entry "cua[omni]" (which provides OmniParser support) so imports
like "from cua import ..." match the installed package; update the requirements
list around the block containing litellm, python-dotenv, and
google-cloud-aiplatform accordingly.
In `@docs/content/docs/cua/examples/automation/form-filling.mdx`:
- Around line 27-31: The requirements code block currently lists cua-agent and
cua-sandbox but the examples import from the unified cua package; update the
snippet so it installs the cua package (e.g., add "cua" with an appropriate
version specifier) to match the imports used in the examples and remove or
adjust the separate cua-agent/cua-sandbox entries so installed packages align
with the example import statements referencing "cua".
In `@docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdx`:
- Around line 63-65: The dependency list currently lists split packages
`cua-agent` and `cua-sandbox`, which conflicts with the examples that import
from the unified `cua` package; update the snippet to replace `cua-agent` and
`cua-sandbox` with the single unified dependency `cua` (e.g.,
`cua>=<appropriate-version>`) while keeping `python-dotenv>=1.0.0`, so the
installation matches the import statements used in the examples that import from
`cua`.
In `@docs/content/docs/cua/guide/advanced/custom-tools.mdx`:
- Line 32: Update the legacy import for BrowserTool to the new cua namespace:
replace the old import that references agent.tools.browser_tool with the new
module path that exposes the BrowserTool class (i.e., import BrowserTool from
cua.tools.browser_tool) so any usage of BrowserTool in this file continues to
work with the metapackage migration.
In `@docs/content/docs/cua/guide/fundamentals/callbacks.mdx`:
- Around line 172-173: Replace the incorrect imports of AsyncCallbackHandler
from agent.callbacks.base with the project-consistent import from cua.callbacks
by changing any occurrences that read "from agent.callbacks.base import
AsyncCallbackHandler" to "from cua.callbacks import AsyncCallbackHandler"; there
are two such occurrences in the custom callback examples (the
AsyncCallbackHandler imports used alongside other callback examples) so update
both to ensure consistency with the rest of the file.
In `@docs/content/docs/cua/guide/sandbox/lifecycle.mdx`:
- Line 146: The internal link "Self-Hosted Sandboxes" currently points to a
non-existent path (/cua/guide/get-started/self-hosted-sandboxes); update that
href to the correct existing doc slug (or use the proper relative path) so
docs:check-links passes. Locate the link text "Self-Hosted Sandboxes" in the
file and replace the broken target
(/cua/guide/get-started/self-hosted-sandboxes) with the correct URL for the
self-hosted sandboxes page, then run docs:check-links to verify.
---
Nitpick comments:
In `@docs/content/docs/cua/guide/get-started/what-is-cua.mdx`:
- Around line 10-11: Combine the two separate imports into a single, concise
import statement by replacing the two lines that import Sandbox and Image and
the separate line that imports ComputerAgent with one line that imports Sandbox,
Image, and ComputerAgent together; update the lines referencing Sandbox, Image,
and ComputerAgent so they continue to work with the consolidated import.
In `@docs/content/docs/cua/guide/integrations/laminar.mdx`:
- Around line 26-27: The imports for ComputerAgent, Sandbox, and Image are fine
but can be consolidated; replace the two separate from cua import lines with a
single import that lists the three symbols (ComputerAgent, Sandbox, Image) so
use one import statement referencing those exact names to simplify and tidy the
top of the module.
In `@docs/content/docs/cua/guide/sandbox/scale-out.mdx`:
- Around line 130-131: The current file imports Sandbox, Image, and
ComputerAgent via two separate statements; consolidate them into a single import
from the cua metapackage by combining the symbols (Sandbox, Image,
ComputerAgent) into one import statement so the code reads like a single-line
import of those three names from cua.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b296e620-41ec-44de-b4da-e76dccd33efb
📒 Files selected for processing (55)
.github/workflows/claude-auto-fix.ymlREADME.mddocs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdxdocs/content/docs/cua/examples/automation/form-filling.mdxdocs/content/docs/cua/examples/automation/post-event-contact-export.mdxdocs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdxdocs/content/docs/cua/examples/sandboxes/android.mdxdocs/content/docs/cua/examples/sandboxes/custom-images.mdxdocs/content/docs/cua/examples/sandboxes/linux.mdxdocs/content/docs/cua/examples/sandboxes/macos.mdxdocs/content/docs/cua/examples/sandboxes/windows.mdxdocs/content/docs/cua/guide/advanced/chat-history.mdxdocs/content/docs/cua/guide/advanced/composed-models.mdxdocs/content/docs/cua/guide/advanced/custom-tools.mdxdocs/content/docs/cua/guide/advanced/demonstration-guided-skills.mdxdocs/content/docs/cua/guide/advanced/human-in-the-loop.mdxdocs/content/docs/cua/guide/advanced/interactive-shell.mdxdocs/content/docs/cua/guide/advanced/local-computer-server.mdxdocs/content/docs/cua/guide/advanced/prompt-caching.mdxdocs/content/docs/cua/guide/advanced/sandboxed-python.mdxdocs/content/docs/cua/guide/advanced/telemetry.mdxdocs/content/docs/cua/guide/advanced/tracing.mdxdocs/content/docs/cua/guide/advanced/trajectories.mdxdocs/content/docs/cua/guide/fundamentals/agent-loops.mdxdocs/content/docs/cua/guide/fundamentals/browser-tool.mdxdocs/content/docs/cua/guide/fundamentals/callbacks.mdxdocs/content/docs/cua/guide/fundamentals/cua-vlm-router.mdxdocs/content/docs/cua/guide/fundamentals/vlms.mdxdocs/content/docs/cua/guide/get-started/migrating-from-computer-sdk.mdxdocs/content/docs/cua/guide/get-started/quickstart.mdxdocs/content/docs/cua/guide/get-started/using-agent-sdk.mdxdocs/content/docs/cua/guide/get-started/using-sandbox-sdk.mdxdocs/content/docs/cua/guide/get-started/what-is-cua.mdxdocs/content/docs/cua/guide/integrations/benchmarks/index.mdxdocs/content/docs/cua/guide/integrations/hud.mdxdocs/content/docs/cua/guide/integrations/laminar.mdxdocs/content/docs/cua/guide/sandbox/images.mdxdocs/content/docs/cua/guide/sandbox/interactive-shell.mdxdocs/content/docs/cua/guide/sandbox/lifecycle.mdxdocs/content/docs/cua/guide/sandbox/scale-out.mdxdocs/content/docs/cua/guide/sandbox/secrets.mdxdocs/content/docs/cua/guide/sandbox/tunneling.mdxdocs/content/docs/cua/reference/agent-sdk/changelog.mdxdocs/content/docs/cua/reference/agent-sdk/v0.3/api.mdxdocs/content/docs/cua/reference/agent-sdk/v0.6/api.mdxdocs/content/docs/cua/reference/cli/index.mdxdocs/content/docs/cua/reference/computer-sdk/v0.3/api.mdxdocs/content/docs/cua/reference/computer-sdk/v0.4/api.mdxdocs/content/docs/cua/reference/sandbox-sdk/index.mdxdocs/content/docs/cua/reference/sandbox-sdk/interfaces.mdxdocs/content/docs/cua/vibe-coding-mcp.mdxdocs/content/docs/cuabench/guide/fundamentals/agent-traces.mdxdocs/content/docs/cuabench/guide/getting-started/installation.mdxdocs/content/docs/cuabench/reference/api.mdxdocs/content/docs/lume/reference/changelog.mdx
JiwaniZakir
reviewed
Mar 26, 2026
JiwaniZakir
left a comment
JiwaniZakir
left a comment
There was a problem hiding this comment.
The runtime class imports in custom-images.mdx deserve attention — TartRuntime and QEMURuntime are being moved from from cua_sandbox.runtime import TartRuntime to from cua import TartRuntime. This is a more significant change than the other import updates, since it assumes these classes are explicitly re-exported at the top level of the cua metapackage's __init__.py. If they aren't, those specific examples will silently break for users while the rest of the cua imports work fine.
Additionally, across multiple files (e.g., form-filling.mdx, windows-app-behind-vpn.mdx), ComputerAgent and Sandbox, Image are now imported as two separate from cua import ... statements rather than being consolidated into one. Since all symbols now come from the same package, combining them into from cua import ComputerAgent, Sandbox, Image would be the idiomatic Python style and reduce visual noise in the example code that users will copy.
…cks with cua Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…orts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
f-trycua
added a commit
that referenced
this pull request
Apr 23, 2026
Add two entries to .lycheeignore to unbreak the Check External Links (lychee) job on PR #1359: `https://openai.com/*` (openai.com returns 403 to automated checkers on URLs like /codex but loads fine in a browser — flagged from docs/content/docs/cua-driver/guide/getting-started/comparison.mdx) and the self-referential install.sh raw-content URL used by the CLI reference page (docs/content/docs/cua-driver/reference/cli-reference.mdx), which 404s only because the install script doesn't exist on main yet and will resolve once this PR merges. The internal link check failure (`/cua/guide/get-started/self-hosted-sandboxes` in docs/content/docs/cua/guide/sandbox/lifecycle.mdx, introduced by PR #1228) is pre-existing on main and out of scope for this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
f-trycua
added a commit
that referenced
this pull request
Apr 23, 2026
* cua-driver v0.1 — initial public release
macOS computer-use driver that speaks the Model Context Protocol
over stdio. Drop-in backend for MCP clients (Claude Code, Cursor) or
as part of the cua-computer-server stack; also usable standalone via
a CLI where every MCP tool is a top-level subcommand.
Designed around a strict no-foreground contract: the user's frontmost
app never changes, the real cursor never warps, and the target never
raises or switches Space. You drive a backgrounded macOS app in one
window while your foreground editor keeps typing in another.
Highlights:
- Element-indexed AX actions that work on hidden, off-Space, or
occluded targets — `get_window_state(pid, window_id)` returns a
per-window AX tree (filtered correctly on multi-window apps),
`click({pid, window_id, element_index})` fires the AX action
without cursor move or focus steal.
- Backgrounded pixel clicks via auth-signed SLEventPostToPid with a
yabai-style focus-without-raise primer — same (x, y) addressing
space as the returned screenshot, supports modifiers and count.
- Three capture modes — `vision` (PNG only; default), `ax` (tree
only, no screen-capture hit), `som` (both).
- Chromium / Electron AX support via the private
`_AXObserverAddNotificationAndCheckRemote` SPI so the tree stays
populated without activating the target.
- Pid-mandatory keyboard — every `press_key` / `type_text` routes
through `CGEvent.postToPid` so keys can't leak into the user's
foreground app.
- Agent-cursor overlay that glides to each target before dispatch,
press-in/ripple on landing, idle-hides. Uniform across AX clicks
and pixel clicks.
- Trajectory recording + replay — per-turn folders with app state,
screenshot, action, click marker. Optional video capture with
zoom-on-click render for demos.
- ScreenCaptureKit screenshots defaulting to a 1568-long-side cap
that matches Anthropic's multimodal input limit, so model-picked
pixel coords match the tool's coordinate space.
See libs/cua-driver/README.md for the full feature list plus
comparison against Codex Computer Use and Claude Computer Use, and
libs/cua-driver/Skills/cua-driver/SKILL.md for the canonical action
loop.
Co-Authored-By: Sarina Li <sarinajin.li@gmail.com>
Co-Authored-By: Dillon DuPont <ddupont@mit.edu>
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: bounds-checked narrowing across tool entry points
Wrap every `Int32(raw...)` / `UInt32(raw...)` conversion in
`Int32(exactly:)` / `UInt32(exactly:)` so malformed MCP input
returns a structured error instead of trapping the daemon.
Addresses CodeRabbit #5 + #6 on PR #1359.
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: daemon signal handling via DispatchSourceSignal
Replaces the POSIX signal handler that called Task.detached (not
async-signal-safe per Apple's Swift concurrency docs) with a
DispatchSource-based pattern. SIGINT/SIGTERM routed through
dispatch sources on a user-initiated queue; SIGPIPE continues to
be SIG_IGN'd so write() returns EPIPE cleanly.
Addresses CodeRabbit #4 on PR #1359.
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: make som the default capture_mode
som returns tree + screenshot, so element_index clicks — the
driver's primary addressing mode — work on the first
get_window_state call without any configuration. vision (PNG only)
stays available as opt-in for vision-first VLM pipelines that
specifically don't want the AX walk.
The previous vision default meant a user who installed the driver
and ran the canonical snapshot-then-click loop ended up on the
pixel-click fallback path instead of the element-indexed primary
path, which made the driver's distinguishing feature invisible out
of the box.
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: bounds-check pid narrowing in ListWindowsTool
The bounds-checked-narrowing sweep caught the 11 tools CodeRabbit
called out but missed `ListWindowsTool.swift`, which had the same
`Int32($0)` trap in its `pidFilter` path (filters the window list
against a caller-supplied pid). A malformed `pid` over Int32.max
would crash the daemon instead of returning a structured error.
Adds a `private static func errorResult` alongside the existing
`summary` helper so the same pattern the other tools use is
available here.
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: v0.1 docs
Adds the cua-driver docs subtree to docs/content/docs/cua-driver/,
matching the lume structure (guide + reference sections, no
examples for v0.1). Registers cua-driver in the root docs
meta.json alongside cua/cuabench/cuabot/lume.
- guide/getting-started/{introduction,installation,quickstart,comparison,faq}.mdx
- reference/{cli-reference,mcp-tools,limits}.mdx
- Fumadocs format: frontmatter + Callout imports matching the
lume pages; no VersionHeader yet (only one shipped version).
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: wire cua-driver section-root redirects
Matches the cuabench / lume pattern: bare `/cua-driver`, the
`/guide` section root, and the `/reference` section root each
redirect to their first content page. Before this, hitting
`/docs/cua-driver` returned a 404 — the docs tree existed but
the section landing redirects in middleware.ts hadn't been
added alongside the new content.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(cursor+recording): Dubins-path cursor, window-bbox zoom, variable-speed render (#1360)
**Cursor**
- AgentCursorView/AgentCursorRenderer: rewrite agent cursor rendering with
SwiftUI Canvas + Dubins-path motion engine (arc → straight → arc)
- AgentCursor.animate(): always arrive at 45° (upper-left tip), approaching
targets from the lower-right — consistent visual signature on every click
**Recording**
- ToolRegistry: capture CLOCK_UPTIME_RAW before handler.invoke() so the
recorded span brackets the full animation time (t_start_ms_from_session_start)
- RecordingSession: add lastAutoRenderURL + auto-render to recording_rendered.mp4
on stop; embed display_scale_factor in session.json
- TrajectoryLoader: add loadActionSpans() — walks all turn-*/action.json and
extracts ActionSpan{startMs, endMs, windowBounds, clickPoint}; read
displayScaleFactor from session.json
- ActionSpan (new): ClickPoint, FocusWaypoint; ActionSpanGenerator with
padMs=500, fastSpeed=8×, mergeGapMs=5000; per-span focus waypoints for
smooth camera pan on merged spans
**Video post-processing**
- RecordingRenderer: window-bbox zoom (ZoomRegion from windowBounds, letterbox
aspect ratio, 400ms eased); variable-speed PTS remapping (1× inside spans,
8× outside); Metal-backed CIContext for GPU frame processing; falls back to
legacy click-zoom when no action spans present
- SetRecordingTool: report rendered path in stop confirmation message
Co-authored-by: cua <cua@cua.localdomain>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: add cua-driver to the header nav with its icon
Adds cua-driver alongside Cua / Cua Bench / Lume / Cua-Bot in the
docs header nav. Ships Guide + Reference tabs (no Examples in
v0.1). Icon is a koala-astronaut mascot provided by Francesco,
tinted into black and white PNG variants at build time so both
light and dark themes have a readable contrast.
SVG equivalents would be nicer (crisper at high DPI, smaller
payload) — left as a design follow-up.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: tighten cua-driver header description
"Background computer-use" reads punchier and matches the length
of the other product descriptions in the dropdown (Benchmarking
toolkit, macOS VM CLI and Framework, etc.).
Co-Authored-By: Claude <noreply@anthropic.com>
* cua-driver: fix Swift 6.1 CI build
Restore `nonisolated(unsafe)` on `capturedWriter` in RecordingRenderer;
PR #1360 dropped it and Swift 6.1 (Xcode 16.3) rejects implicit capture
of non-Sendable AVAssetWriter in the @sendable finishWriting closure.
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: fix broken links flagged by link-check CI
Add two entries to .lycheeignore to unbreak the Check External Links
(lychee) job on PR #1359: `https://openai.com/*` (openai.com returns
403 to automated checkers on URLs like /codex but loads fine in a
browser — flagged from docs/content/docs/cua-driver/guide/getting-started/comparison.mdx)
and the self-referential install.sh raw-content URL used by the CLI
reference page (docs/content/docs/cua-driver/reference/cli-reference.mdx),
which 404s only because the install script doesn't exist on main yet
and will resolve once this PR merges. The internal link check failure
(`/cua/guide/get-started/self-hosted-sandboxes` in
docs/content/docs/cua/guide/sandbox/lifecycle.mdx, introduced by
PR #1228) is pre-existing on main and out of scope for this PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* cua-driver: main-actor-isolate AppKitBootstrap.runBlockingAppKitWith
Swift 6.1 on CI flags `NSApplication.shared`, `setActivationPolicy`,
and `.run()` as main-actor-isolated calls from a nonisolated
context. Our local Swift 6.3 toolchain was more permissive here, so
the errors only surfaced on CI. Compiler-suggested fix: annotate the
helper itself with `@MainActor`. All callers are already on the
main actor (this is the AppKit bootstrap path), so isolation is
tightened without behavior change.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Sarina Li <sarinajin.li@gmail.com>
Co-authored-by: Dillon DuPont <ddupont@mit.edu>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: ddupont <3820588+ddupont808@users.noreply.github.com>
Co-authored-by: cua <cua@cua.localdomain>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
from cua_sandbox import/from agent import/from agent.tools import/from agent.callbacks importwithfrom cua importacross all docspip install cua-sandbox/pip install "cua-agent[...]"withpip install cua/pip install "cua[...]"CUA(brand name in prose) withCua— preservingCUA_*env vars andCUA-Benchproduct nameTest plan
from cua_sandboxorfrom agent importreferences remain in docspip install cua-agentorpip install cua-sandboxreferences remain in docs (except changelogs)🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
Documentation
pip install cuaconsolidates previously separatecua-sandboxandcua-agentpackages.cuapackage instead ofcua_sandboxandagentmodules.Chores