Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
527653b
feat(cua-driver): vendor trycua/cua driver with 1000-normalized coord…
LaZzyMan Jun 25, 2026
8252adf
feat(cua-driver): make normalized coordinate scale configurable
LaZzyMan Jun 25, 2026
14f7dff
ci(cua-driver): add cross-platform release workflow for vendored driver
LaZzyMan Jun 25, 2026
0ae031e
chore(cua-driver): rebrand vendored driver as qwen-cua-driver
LaZzyMan Jun 26, 2026
ed543e7
feat(cua-driver): finish relative-coordinate mode — toggle, scale, zo…
LaZzyMan Jun 26, 2026
1c2d8ef
chore(cua-driver): add upstream-sync script (git subtree unusable here)
LaZzyMan Jun 26, 2026
81bf180
chore(cua-driver): exclude vendored driver from qwen-code ESLint
LaZzyMan Jun 26, 2026
7c52306
fix(cua-driver): let start_session revive an idle-reaped session
LaZzyMan Jun 26, 2026
606b688
fix(cua-driver): retry daemon socket writes on EAGAIN
LaZzyMan Jun 26, 2026
9658106
fix(cua-driver/linux): stop reporting bare "Clicked" for X11 syntheti…
LaZzyMan Jun 26, 2026
b9d8614
fix(cua-driver/windows): list empty-/null-title top-level windows
LaZzyMan Jun 26, 2026
758beff
chore(cua-driver): track cherry-picked upstream PRs; fix vendored-from
LaZzyMan Jun 26, 2026
3292c3b
ci(cua-driver): satisfy repo yamllint on the release workflow
LaZzyMan Jun 26, 2026
d682c19
chore(cua-driver): sync vendored driver to cua-driver-rs-v0.6.8
LaZzyMan Jun 26, 2026
97ec02b
ci(cua-driver): add a dry_run gate to the release workflow
LaZzyMan Jun 26, 2026
0cb586d
Merge branch 'main' into lazzy/cua-driver-relative-coords
LaZzyMan Jun 26, 2026
38b0d4c
Merge branch 'main' into lazzy/cua-driver-relative-coords
LaZzyMan Jun 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
353 changes: 353 additions & 0 deletions .github/workflows/cd-cua-driver.yml

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions packages/cua-driver/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Vendored fork of trycua/cua → libs/cua-driver.
#
# Upstream relied on the cua monorepo root .gitignore to ignore Rust build
# output (`target/`). That root is gone in this vendored copy, so re-declare
# the generated artifacts here to keep `cargo build` output out of git.
target/
*.pyc
__pycache__/
.DS_Store
1 change: 1 addition & 0 deletions packages/cua-driver/.vendored-from
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
cua-driver-rs-v0.6.8
23 changes: 23 additions & 0 deletions packages/cua-driver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Cua Driver

Background computer-use driver for any agents. Speaks MCP over stdio; drives native macOS apps without stealing focus.

**[Documentation](https://cua.ai/docs/cua-driver)** - Installation, guides, and API reference.

## Claude Code computer-use compatibility

Standard Claude Code MCP registration:

```bash
claude mcp add --transport stdio cua-driver -- qwen-cua-driver mcp
```

If you want Claude Code's vision/computer-use-style flow to ground on QwenCuaDriver window screenshots, register the compatibility mode:

```bash
claude mcp add --transport stdio cua-computer-use -- qwen-cua-driver mcp --claude-code-computer-use-compat
```

This keeps QwenCuaDriver's normal MCP tools and changes only `screenshot`, which requires `pid` and `window_id` and captures that window only.

Use MCP for this Claude Code vision/computer-use-style path. CLI screenshots still work as QwenCuaDriver calls, but they do not expose the `mcp__cua-computer-use__screenshot` tool name that Claude Code appears to use as the image-grounding cue.
265 changes: 265 additions & 0 deletions packages/cua-driver/docs/relative-coordinates-design.md

Large diffs are not rendered by default.

39 changes: 39 additions & 0 deletions packages/cua-driver/docs/tool-output-format.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# cua-driver MCP Tool Output Format

Every tool call returns a ✅ checkmark + concise summary. No structured JSON output.

## screenshot
```
✅ Screenshot — 1920x1080 png

On-screen windows:
- Terminal (pid 7476) "cua — Claude Code" [window_id: 2102]
- Blender (pid 6808) "* Untitled - Blender 5.1.1" [window_id: 2129]
- Google Chrome (pid 13313) "Download — Blender" [window_id: 1941]
→ Call get_window_state(pid, window_id) to inspect a window's UI.
```

## get_window_state
```
✅ Blender — 11 elements, turn 3 + screenshot
⚠️ Small AX tree (11 elements) — this app likely uses custom rendering
(e.g. Blender, games, Electron). Use pixel clicks: click(pid, x, y)
with coordinates from the screenshot.

- AXApplication "Blender"
- [0] AXWindow "* Untitled - Blender 5.1.1" actions=[AXRaise]
- [1] AXButton
...
```

## click
```
✅ Posted click to pid 6808.
```

## zoom
```
✅ Zoomed region captured at native resolution. To click a target in
this image, use `click(pid, x, y, from_zoom=true)` where x,y are pixel
coordinates in THIS zoomed image — the driver maps them back automatically.
```
16 changes: 16 additions & 0 deletions packages/cua-driver/python/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Build artifacts
dist/
build/
*.egg-info/
__pycache__/
*.pyc
*.pyo

# Downloaded binaries and archives
downloads/
src/cua_driver/bin/

# Virtual environments
venv/
.venv/
env/
2 changes: 2 additions & 0 deletions packages/cua-driver/python/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
include README.md
recursive-include src/cua_driver/bin *
74 changes: 74 additions & 0 deletions packages/cua-driver/python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# cua-driver

Python wrapper for [cua-driver](https://github.com/trycua/cua/tree/main/libs/cua-driver) - a cross-platform MCP (Model Context Protocol) server for computer-use automation.

## Installation

```bash
pip install cua-driver
```

## Usage

The package provides a `cua-driver` command that wraps the native Rust binary:

```bash
# Start MCP server over stdio
cua-driver mcp

# Check version
cua-driver --version

# View help
cua-driver --help
```

### Python API

You can also use the Python API directly:

```python
from cua_driver import run_cua_driver, get_binary_path

# Run with custom args
exit_code = run_cua_driver(["mcp"])

# Get path to bundled binary
binary_path = get_binary_path()
```

## Features

- **Cross-platform**: Works on macOS (universal), Linux (x86_64), and Windows (x86_64/ARM64)
- **Zero dependencies**: Pure Python wrapper with no external dependencies
- **Stdio passthrough**: Transparent piping for MCP protocol communication
- **Bundled binary**: No separate installation required - the Rust binary is included in the wheel

## What is cua-driver?

`cua-driver` is a cross-platform MCP server that provides 40+ tools for:
- Screen capture and window management
- Mouse and keyboard automation
- Accessibility tree interaction
- Application launching and control

It enables AI agents to interact with desktop applications through the Model Context Protocol.

## Platform Support

| Platform | Architecture | Status |
|----------|-------------|---------|
| macOS | Universal (ARM64 + x86_64) | ✅ Supported |
| Linux | x86_64 | ✅ Supported |
| Windows | x86_64 | ✅ Supported |
| Windows | ARM64 | ✅ Supported |

## License

MIT License - see [LICENSE](https://github.com/trycua/cua/blob/main/LICENSE.md)

## Links

- [GitHub Repository](https://github.com/trycua/cua)
- [Documentation](https://github.com/trycua/cua/tree/main/libs/cua-driver)
- [Issue Tracker](https://github.com/trycua/cua/issues)
Loading
Loading