diff --git a/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx index c266bb3417..8561144b51 100644 --- a/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx +++ b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx @@ -127,6 +127,8 @@ Register the stdio server: openclaw mcp set cua-driver '{"command":"cua-driver","args":["mcp"]}' ``` +This registers the driver as a normal gateway-spawned MCP server; on macOS it does **not** inherit OpenClaw.app's permission grants. For that, the app process must spawn `cua-driver --embedded` directly; see [Embedding](/reference/cua-driver/embedding). + Verify: restart OpenClaw and confirm `cua-driver` is available in the MCP server list. ## OpenCode diff --git a/docs/content/docs/reference/cua-driver/cli-reference.mdx b/docs/content/docs/reference/cua-driver/cli-reference.mdx index a797ff3bd9..d7f3988659 100644 --- a/docs/content/docs/reference/cua-driver/cli-reference.mdx +++ b/docs/content/docs/reference/cua-driver/cli-reference.mdx @@ -71,6 +71,7 @@ On macOS, shell-spawned MCP processes can auto-launch and proxy through a CuaDri | Name | Type | Default | Description | | ---- | ---- | ------- | ----------- | | `--socket` | String | — | Override the daemon socket or named-pipe path used by the proxy fallback. | +| `--host-bundle-id` | String | — | Advisory host bundle id label echoed in check_permissions output (embedded mode). | **Flags:** @@ -78,6 +79,7 @@ On macOS, shell-spawned MCP processes can auto-launch and proxy through a CuaDri | ---- | ----------- | | `--no-daemon-relaunch` | Stay in-process instead of proxying through a daemon. | | `--claude-code-computer-use-compat` | Expose the Claude Code computer-use compatibility screenshot surface. | +| `--embedded` | Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1. | ### `cua-driver serve` @@ -91,12 +93,14 @@ The daemon owns per-process state such as element-index caches, recording state, | ---- | ---- | ------- | ----------- | | `--socket` | String | — | Override the daemon socket or named-pipe path. | | `--pid-file` | String | — | Override the pid-file path on Unix targets. | +| `--host-bundle-id` | String | — | Advisory host bundle id label echoed in check_permissions output (embedded mode). | **Flags:** | Name | Description | | ---- | ----------- | | `--no-permissions-gate` | Skip the macOS first-launch permissions gate. | +| `--embedded` | Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1. | ### `cua-driver stop` diff --git a/docs/content/docs/reference/cua-driver/embedding.mdx b/docs/content/docs/reference/cua-driver/embedding.mdx new file mode 100644 index 0000000000..02bd8d7c6b --- /dev/null +++ b/docs/content/docs/reference/cua-driver/embedding.mdx @@ -0,0 +1,80 @@ +--- +title: Embedding +description: Run cua-driver as a direct child of your host app instead of handing off to a standalone daemon. +--- + +Embedding runs `cua-driver` as a direct child of your host app instead of relaunching or proxying through a standalone daemon. On macOS, this also lets the driver inherit the host app's Accessibility and Screen Recording grants, so users only approve your app. On Windows and Linux, embedded mode keeps driver execution inside the host process tree and avoids daemon handoff. + +A complete macOS reference host and demo live in the repo at `libs/cua-driver/rust/examples/embedded-host-macos`. + +## Launch embedded + +Set embedded mode on the driver process your host spawns: + +```sh +CUA_DRIVER_EMBEDDED=1 \ +CUA_DRIVER_HOST_BUNDLE_ID=com.yourco.yourapp \ +cua-driver mcp +``` + +Or use the equivalent flags: + +```sh +cua-driver mcp --embedded --host-bundle-id com.yourco.yourapp +``` + +Only the exact value `CUA_DRIVER_EMBEDDED=1` enables embedded mode. The host bundle id is an advisory label echoed in `check_permissions`; trust still comes from macOS's responsibility chain. + +## Host requirements + +- Spawn `cua-driver` directly from your app, for example with `Process` / `NSTask`, `posix_spawn`, or `fork` / `exec`. +- Speak MCP over the child's stdin/stdout. +- On macOS, do not launch the driver with `open(1)` or `NSWorkspace.open`; LaunchServices makes the launched app its own responsible process and breaks permission inheritance. +- On macOS, request Accessibility and Screen Recording from the host app with `AXIsProcessTrustedWithOptions` and `CGRequestScreenCaptureAccess`. + +If macOS grants are added after the driver child has started, restart the child so TCC is re-queried with a fresh per-process cache. + +## App + gateway architectures + +`--embedded` does not transfer a GUI app's grants to the driver; it only keeps the driver inside its spawner's macOS responsibility chain. If a separate gateway, daemon, or Node process spawns MCP servers, registering `cua-driver mcp --embedded` there makes the driver inherit the gateway's identity, not the app's. Spawn the driver from the app process, or bridge MCP from the gateway to an app-spawned child. + +```text +Wrong (inherits the gateway's identity): Right: + +gateway / node daemon YourApp.app + └─ cua-driver --embedded └─ cua-driver --embedded +``` + +## What changes + +| Behavior | Standalone | Embedded | +| --- | --- | --- | +| Process model | May proxy through a daemon | Direct child / in-process path | +| Daemon relaunch | May proxy through app daemon | Disabled | +| macOS TCC identity | `com.trycua.driver` or caller | Host app | +| macOS permission prompts | Driver may prompt | Driver never prompts | +| macOS Settings entries | CuaDriver | Host app only | +| `check_permissions` attribution | `driver-daemon` or `caller` | `host` on macOS embedded runs | + +Driver tools, screenshots, AX tree reads, background input, and the agent cursor overlay otherwise behave the same. + +## macOS permission check + +Call the `check_permissions` MCP tool after starting the embedded driver. On macOS, embedded mode ignores prompt requests and should return `source.attribution: "host"`: + +```json +{ + "accessibility": true, + "screen_recording": true, + "screen_recording_capturable": true, + "source": { + "attribution": "host", + "host_bundle_id": "com.yourco.yourapp", + "embedded": true + } +} +``` + +If `source.attribution` is not `host` on macOS, embedded mode is not active for the process handling your MCP calls. Check that `CUA_DRIVER_EMBEDDED=1` is passed to the child, that the child was spawned directly, and that you are not accidentally talking to an old standalone daemon. + +`source.attribution: "host"` means the driver process is running in embedded mode; it does not prove that your GUI app is the responsible process. If a gateway, daemon, or Node process spawned the child, the reported grant state still belongs to that spawner. Spawn `cua-driver` from the app process that owns the macOS grants, or bridge MCP to an app-spawned child. diff --git a/docs/content/docs/reference/cua-driver/macos-permissions.mdx b/docs/content/docs/reference/cua-driver/macos-permissions.mdx index 844510ca61..64bf7b62de 100644 --- a/docs/content/docs/reference/cua-driver/macos-permissions.mdx +++ b/docs/content/docs/reference/cua-driver/macos-permissions.mdx @@ -7,7 +7,7 @@ description: The macOS-only `cua-driver permissions` command for inspecting and ## `cua-driver permissions` (macOS) -Inspect or request the macOS TCC grants the driver needs (Accessibility and Screen Recording). +Inspect or request the macOS TCC grants the driver needs (Accessibility and Screen Recording). Embedded-mode hosts do not use `cua-driver permissions grant`; the host app requests these grants itself, as described in [Embedding](/reference/cua-driver/embedding). ```bash cua-driver permissions status # report grant status; read-only, no prompt diff --git a/docs/content/docs/reference/cua-driver/meta.json b/docs/content/docs/reference/cua-driver/meta.json index d567181229..c0804e3322 100644 --- a/docs/content/docs/reference/cua-driver/meta.json +++ b/docs/content/docs/reference/cua-driver/meta.json @@ -1 +1 @@ -{ "title": "Cua Driver", "pages": ["cli-reference", "macos-permissions", "mcp-tools", "mcp-tool-notes", "contracts", "limits", "modality-test-suite"] } +{ "title": "Cua Driver", "pages": ["cli-reference", "macos-permissions", "embedding", "mcp-tools", "mcp-tool-notes", "contracts", "limits", "modality-test-suite"] } diff --git a/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md b/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md new file mode 100644 index 0000000000..1f5bad14f2 --- /dev/null +++ b/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md @@ -0,0 +1,407 @@ +# Embedding cua-driver in your agent harness without introducing new permissions + +This guide is for teams shipping a macOS app (an "agent harness") that wants +cua-driver's background computer-use and agent-cursor overlay **inside their +own app**, without shipping a second app bundle and without their users ever +seeing a second macOS permission prompt. Your app requests Accessibility and +Screen Recording once; the embedded driver inherits those grants. + +A working reference host lives in the cua repo at +`libs/cua-driver/rust/examples/embedded-host-macos/` +(https://github.com/trycua/cua). This doc ships standalone in the skill +pack, so the path is given rather than a relative link. + +## How macOS attributes these permissions (what you must know) + +macOS TCC (the privacy system behind System Settings → Privacy & Security) +does not attribute Accessibility or Screen Recording to an executable path. +It attributes them to the **responsible process**: the app at the top of the +process's launch chain, as tracked by the kernel/LaunchServices. When your +signed app spawns a child with `posix_spawn`, `NSTask`/`Process`, or plain +`fork`/`exec`, that child stays inside *your* responsibility chain — TCC +checks made by the child are answered with **your app's** grants, and any +prompt it triggered would name **your app**. This is exactly the behavior +embedding relies on: grant once to the host, and every well-behaved child +inherits. (Apple documents the attribution chain; you can watch it live with +`log stream --debug --predicate 'subsystem == "com.apple.TCC" AND eventMessage BEGINSWITH "AttributionChain"'`.) + +Two things break the chain, and both are things the embedded driver must +*not* do (and, in embedded mode, does not do). First, launching via +LaunchServices (`open -a …`, `NSWorkspace.open`) makes the launched app its +own responsible process. Second, a process can explicitly *disclaim* +responsibility for a child (`responsibility_spawnattrs_setdisclaim`), making +the child its own responsible process — standalone cua-driver does this on +purpose so its permissions attach to a stable `com.trycua.driver` identity +instead of whatever terminal launched it. Embedded mode turns that off. + +Note this is TCC **responsibility** inheritance — it is unrelated to App +Sandbox inheritance (`com.apple.security.inherit`). This guide assumes a +non-sandboxed host, which is typical for agent harnesses; a sandboxed host +spawning a non-sandboxed helper raises separate App Sandbox questions that +embedded mode does not address. + +## Launching in embedded mode + +```sh +# env var form — set by the host on the child process +CUA_DRIVER_EMBEDDED=1 CUA_DRIVER_HOST_BUNDLE_ID=com.yourco.yourapp cua-driver mcp + +# flag form — equivalent (the flags just set the env vars) +cua-driver mcp --embedded --host-bundle-id com.yourco.yourapp +``` + +Requirements on the host side: + +- **Spawn the driver directly** as a child process (`Process`/`NSTask`, + `posix_spawn`, `exec` from your own code). Do **not** launch it via + `open(1)` or `NSWorkspace` — that hands it to LaunchServices and breaks + inheritance. +- Speak MCP over the child's stdin/stdout (line-delimited JSON-RPC, the + driver's native transport). The reference host does exactly this. +- Request Accessibility and Screen Recording **from your app** before (or + after — the driver just reports "not granted" until then) starting the + driver, using `AXIsProcessTrustedWithOptions([kAXTrustedCheckOptionPrompt: true])` + and `CGRequestScreenCaptureAccess()`. + +Only the exact value `CUA_DRIVER_EMBEDDED=1` enables embedded mode; anything +else is ignored (fail-safe). `--host-bundle-id` is an advisory label echoed +in `check_permissions` output and logs — it is **not** a trust signal; trust +comes from the OS responsibility chain, so there is nothing to spoof by +setting it. + +## What embedded mode changes (and what it doesn't) + +| | Standalone | Embedded (`CUA_DRIVER_EMBEDDED=1`) | +| ------------------------------ | ----------------------------------- | ---------------------------------------- | +| Responsibility disclaim re-exec| ON (owns its TCC identity) | OFF (stays in the host's chain) | +| Daemon auto-relaunch via `open -a CuaDriver` | Yes, when installed | Never (would leave the host's chain) | +| TCC identity | `com.trycua.driver` | the host app | +| Permission prompts / startup gate | May prompt once | **Never prompts** | +| Settings → Privacy & Security entries | CuaDriver | your app only | +| `check_permissions` `source.attribution` | `driver-daemon` (or `caller`) | `host` | +| Overlay, background input, capture, all tools | full | full — identical | + +Everything else — the agent-cursor overlay, background (no-focus-steal) +clicking and typing, AX tree reads, per-window screenshots — is unchanged. +When embedded mode is off, nothing in this feature is active: standalone +behavior is byte-for-byte what it was. + +## The responsibility-chain requirement, exactly + +The host must be the responsible process for the driver. That holds +automatically when you spawn the driver directly and embedded mode is on. If +the driver were allowed to disclaim (standalone behavior), macOS would treat +it as its own responsible process: your user would get a *second* prompt +attributed to the driver binary, a second Settings entry, and capture/AX +would fail until that second grant — the exact experience embedding exists +to eliminate. Embedded mode short-circuits the disclaim re-exec +(`responsibility.rs`) and the `open -a CuaDriver` daemon relaunch, which are +the only two places the driver would otherwise leave your chain. + +### App + gateway architectures + +`--embedded` does not transfer a GUI app's permissions to the driver; it +only keeps the driver inside its **spawner's** TCC responsibility chain. If +your product has a GUI app that owns the macOS grants and a separate +gateway, daemon, or Node process that spawns MCP servers, registering +`cua-driver mcp --embedded` with the gateway makes the driver inherit the +gateway's identity, not the app's. Spawn the driver from the GUI app itself, +or bridge MCP from the gateway to an app-spawned child. + +```text +Wrong (inherits the gateway's identity): Right: + +gateway / node daemon YourApp.app + └─ cua-driver --embedded └─ cua-driver --embedded +``` + +Note `check_permissions` cannot detect this: `source.attribution` reports +`host` whenever `CUA_DRIVER_EMBEDDED=1` is set, even if a gateway spawned +the driver. The symptoms are grant booleans that track the *gateway's* TCC +state and prompts/Settings entries naming the gateway process; see +Troubleshooting below. + +## Reading `check_permissions` from the host + +Call the `check_permissions` tool over MCP. In embedded mode it never raises +a dialog (the `prompt` argument is ignored) and returns: + +```json +{ + "accessibility": true, + "screen_recording": true, + "screen_recording_capturable": true, + "source": { + "attribution": "host", + "host_bundle_id": "com.yourco.yourapp", + "embedded": true, + "pid": 12345, + "responsible_ppid": 12300, + "executable": "/path/to/cua-driver", + "disclaim_env": false, + "note": "Embedded mode: these booleans reflect the HOST app's TCC grant…" + } +} +``` + +- `accessibility` / `screen_recording` — the live TCC state *of your app's + grant*, answered from inside the driver process (which shares your + identity). If both are true, it is safe to drive the desktop. +- `screen_recording_capturable` — a live ScreenCaptureKit probe + (`SCShareableContent`), the authoritative signal. If it disagrees with + `screen_recording`, the preflight boolean is stale or belongs to a + different identity — see troubleshooting. +- `source.attribution` values: + - `host` — embedded mode; booleans reflect the host's grant. What you + should always see when embedding. + - `driver-daemon` — standalone daemon owning `com.trycua.driver`. If you + see this while embedding, embedded mode is not actually set. + - `caller` — a non-embedded, non-bundle launch (e.g. someone ran the + binary from a terminal); booleans reflect the terminal's grants. + +If a permission is missing, the correct reaction is: **the host requests +it** (the two API calls above), then re-calls `check_permissions`. The +driver will never pop its own dialog in embedded mode. + +Heads-up on grant timing: macOS caches TCC answers per process. If your app +requests/receives the grants *after* the driver child is already running, +restart the driver child so it re-queries with a fresh cache. + +## Minimal host example (copy-paste) + +The file below is the complete reference host — mirrored verbatim from +`libs/cua-driver/rust/examples/embedded-host-macos/ExampleAgentHarness.swift` +in the cua repo (which also has a build-and-run `demo.sh` covering the +TCC-reset flow). +It requests the two grants as the host, spawns cua-driver embedded, and +runs the whole demo sequence: attribution check, background screenshot, +background AX read, agent-cursor glide. + +`ExampleAgentHarness.swift`: + +```swift +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 Cua AI, Inc. + +// ExampleAgentHarness — minimal reference host for embedding cua-driver. +// Mirrored verbatim in Skills/cua-driver/EMBEDDING.md ("Minimal host +// example") — keep the two in sync. +// +// Runs the one-grant demo sequence from EMBEDDING.md end to end: +// 1. Requests Accessibility + Screen Recording AS THE HOST (the only +// prompts the user ever sees), then +// 2. spawns cua-driver as a direct child in embedded mode and, over +// stdio MCP, verifies attribution, takes a background screenshot, +// reads a background app's window state, and glides the agent-cursor +// overlay — with zero driver-side prompts. +// +// Launched via `open` (see demo.sh) the app has no terminal, so all +// output also goes to /tmp/cua-embedded-demo.log. + +import Foundation +import ApplicationServices +import CoreGraphics + +let logPath = "/tmp/cua-embedded-demo.log" +FileManager.default.createFile(atPath: logPath, contents: nil) +let logFile = FileHandle(forWritingAtPath: logPath)! +func log(_ line: String) { + print(line) + logFile.write((line + "\n").data(using: .utf8)!) +} + +// 1. Request both grants AS THE HOST — the only prompts in the whole flow. +let axOpts = ["AXTrustedCheckOptionPrompt": true] as CFDictionary +let ax = AXIsProcessTrustedWithOptions(axOpts) +let sr = CGRequestScreenCaptureAccess() +log("host grants — accessibility: \(ax), screen recording: \(sr)") +// Keep going even without grants: the run registers BOTH rows in one pass +// (the AX request above, plus — on newer macOS, where the app only appears +// in the Screen Recording pane after a real ScreenCaptureKit attempt — the +// embedded driver's live probe below, registered as THE HOST, which is the +// point of embedding). Grant both in one Settings visit, then re-run. +if !ax || !sr { + log("after this run: grant the missing item(s) in System Settings, then re-run") +} + +// 2. Spawn cua-driver as a DIRECT child (never via `open`/NSWorkspace — +// that breaks responsibility inheritance) in embedded mode. +let driverPath = ProcessInfo.processInfo.environment["CUA_DRIVER_PATH"] + ?? "/usr/local/bin/cua-driver" +let driver = Process() +driver.executableURL = URL(fileURLWithPath: driverPath) +driver.arguments = ["mcp"] +var env = ProcessInfo.processInfo.environment +env["CUA_DRIVER_EMBEDDED"] = "1" +env["CUA_DRIVER_HOST_BUNDLE_ID"] = Bundle.main.bundleIdentifier ?? "" +driver.environment = env +let toDriver = Pipe(), fromDriver = Pipe() +driver.standardInput = toDriver +driver.standardOutput = fromDriver +try driver.run() + +// 3. Line-delimited JSON-RPC 2.0 over the child's stdio. +var buffer = Data() +func send(_ obj: [String: Any]) { + var data = try! JSONSerialization.data(withJSONObject: obj) + data.append(0x0A) + toDriver.fileHandleForWriting.write(data) +} +func readMessage() -> [String: Any] { + while true { + if let nl = buffer.firstIndex(of: 0x0A) { + let line = buffer.subdata(in: buffer.startIndex.. [String: Any] { + nextId += 1 + send(["jsonrpc": "2.0", "id": nextId, "method": "tools/call", + "params": ["name": tool, "arguments": args]]) + while true { + let msg = readMessage() + if msg["id"] as? Int == nextId { + if let error = msg["error"] as? [String: Any] { + log("RPC error for \(tool): \(error)") + } + return msg["result"] as? [String: Any] ?? [:] + } + } +} + +nextId += 1 +send(["jsonrpc": "2.0", "id": nextId, "method": "initialize", "params": [ + "protocolVersion": "2024-11-05", "capabilities": [:], + "clientInfo": ["name": "ExampleAgentHarness", "version": "0.1"]]]) +_ = readMessage() +send(["jsonrpc": "2.0", "method": "notifications/initialized"]) +log("embedded cua-driver started (\(driverPath)) — no driver prompt should have appeared") + +// 4. check_permissions must report attribution "host" and never prompt. +let perms = call("check_permissions") +let structured = perms["structuredContent"] as? [String: Any] ?? [:] +let source = structured["source"] as? [String: Any] ?? [:] +let attribution = source["attribution"] as? String ?? "?" +log("check_permissions — attribution: \(attribution) (want: host), " + + "capturable: \(structured["screen_recording_capturable"] ?? "?")") + +// 5. Background AX read + window screenshot — proves both grants +// inherited without focusing anything. launch_app resolves pid + +// windows without foregrounding; get_window_state returns the AX +// element tree AND a screenshot of the (background) window. +let launch = call("launch_app", ["bundle_id": "com.apple.finder"]) +let launched = launch["structuredContent"] as? [String: Any] ?? [:] +let pid = launched["pid"] as? Int ?? 0 +let windows = launched["windows"] as? [[String: Any]] ?? [] +let windowId = windows.first?["window_id"] as? Int ?? 0 +log("launch_app(Finder) — pid: \(pid), windows: \(windows.count)") + +let state = call("get_window_state", ["pid": pid, "window_id": windowId]) +let images = (state["content"] as? [[String: Any]] ?? []) + .filter { $0["type"] as? String == "image" } +let hasTree = (state["structuredContent"] as? [String: Any])?["elements"] != nil +log("get_window_state(Finder) — tree: \(hasTree ? "ok" : "EMPTY"), " + + "screenshot: \(images.count) image(s) (want: ≥1)") + +// 6. Agent-cursor glide — shows the overlay, no real-pointer move. +log("watch the agent cursor glide now (no real-pointer move)…") +let cursor1 = call("move_cursor", ["x": 200, "y": 200]) +Thread.sleep(forTimeInterval: 2) +let cursor2 = call("move_cursor", ["x": 900, "y": 500]) +Thread.sleep(forTimeInterval: 2) +let cursorOk = (cursor1["isError"] as? Bool) != true && + (cursor2["isError"] as? Bool) != true +log("move_cursor — \(cursorOk ? "ok" : "FAILED")") + +let pass = attribution == "host" && !images.isEmpty && hasTree && cursorOk +log(pass ? "DEMO COMPLETE: PASS" : "DEMO COMPLETE: FAIL") +driver.terminate() +exit(pass ? 0 : 1) +``` + +Build it as a signed app bundle (a stable signing identity is what keys +the TCC grant rows to your app): + +```sh +mkdir -p ExampleAgentHarness.app/Contents/MacOS +swiftc -O ExampleAgentHarness.swift \ + -o ExampleAgentHarness.app/Contents/MacOS/ExampleAgentHarness \ + -framework ApplicationServices +printf '%s\n' '' \ + '' \ + '' \ + 'CFBundleExecutableExampleAgentHarness' \ + 'CFBundleIdentifiercom.trycua.example-agent-harness' \ + 'CFBundlePackageTypeAPPL' \ + '' > ExampleAgentHarness.app/Contents/Info.plist +codesign --force --sign - ExampleAgentHarness.app # use your Developer ID in production +open ExampleAgentHarness.app # `open` is correct HERE: the HOST must be its own responsible process +tail -f /tmp/cua-embedded-demo.log +``` + +## Troubleshooting + +**"I still get a second permission prompt / a second Settings entry."** +Embedded mode is not in effect for the process doing the TCC check. Causes, +in order of likelihood: (a) `CUA_DRIVER_EMBEDDED` is not exactly `1`, or was +set on your app but not passed into the child's environment; (b) the driver +was launched via `open(1)` / `NSWorkspace` instead of spawned directly, so +it is its own responsible process; (c) an old standalone `CuaDriver.app` +daemon is running and your MCP calls are being answered by *it* — check +`check_permissions` → `source.attribution` (must be `host`) and stop the +daemon (`cua-driver stop`). To see exactly which identity macOS is charging, +run: `log stream --debug --predicate 'subsystem == "com.apple.TCC" AND +eventMessage BEGINSWITH "AttributionChain"'` and trigger the action again. + +**"Screenshots come back black (or `screen_recording: true` but +`screen_recording_capturable: false`)."** +The preflight boolean and the live probe disagree, which means the Screen +Recording grant TCC found does not belong to the driver's current +responsible identity. Either the host never actually got the grant (check +System Settings), the grant was reset (`tccutil reset ScreenCapture`) after +the app cached a `true`, or the driver escaped the host's chain (see the +previous item). Restart the driver child after any grant change — TCC +answers are cached per process. + +**"The AX tree comes back empty / clicks do nothing."** +`AXIsProcessTrusted()` is false for the effective identity. The host hasn't +been granted Accessibility, or was granted it *after* the driver child +started (per-process cache again — restart the child), or the app was +re-signed/moved so the existing grant row no longer matches it (remove and +re-add it in System Settings, or `tccutil reset Accessibility ` +and re-grant). + +**"It worked, then stopped after I updated/re-signed my app."** +TCC grant rows are keyed to the app's code-signing identity. A signature +change can orphan the old row. Reset and re-grant: +`tccutil reset Accessibility com.yourco.yourapp && tccutil reset ScreenCapture com.yourco.yourapp`. + +## Platform notes (Windows / Linux) + +Embedding already works on Windows and Linux (X11) with **no permission +ceremony at all**: neither platform gates screen capture, tree reads, or +input injection behind per-app grants, so a directly-spawned driver child +inherits everything relevant (session, desktop, integrity level) by plain +process inheritance. Set the flag anyway — it keeps the driver answering +in-process as the host's child instead of proxying to an out-of-session +daemon, and it makes intent explicit in `check_permissions` output. The +one-grant inheritance story this guide describes is macOS-specific because +macOS is the only platform where a grant exists to inherit. + +Two known exceptions: + +- **Windows, elevated / UWP targets**: injecting into higher-integrity + windows needs the uiAccess-signed worker (`cua-driver-uia`). Embedded + mode forces in-process execution, so it does NOT route through a running + uia worker — an embedded host that must drive elevated apps has to manage + that worker itself. +- **Linux Wayland** (opt-in/preview): capture goes through XDG desktop + portals, which prompt per-session at capture time and cannot be + pre-granted by the host. X11, the supported path, has no gate. diff --git a/libs/cua-driver/rust/Skills/cua-driver/README.md b/libs/cua-driver/rust/Skills/cua-driver/README.md index 5ce2345def..2fad020679 100644 --- a/libs/cua-driver/rust/Skills/cua-driver/README.md +++ b/libs/cua-driver/rust/Skills/cua-driver/README.md @@ -19,6 +19,11 @@ platform: no focus steal, no cursor warp. - `LINUX.md` — Linux carve-out (X11 background input via AT-SPI + XSendEvent, recording, Wayland opt-in/preview). Read this when driving on Linux. +- `EMBEDDING.md` — embedding cua-driver inside another macOS app + (agent harness) so the driver inherits the host's Accessibility + + Screen Recording grants with zero extra prompts. Read this when + integrating the driver into your own app rather than running it + standalone. ## What the skill covers diff --git a/libs/cua-driver/rust/Skills/cua-driver/SKILL.md b/libs/cua-driver/rust/Skills/cua-driver/SKILL.md index 23fa3f0f20..77ddcccf2b 100644 --- a/libs/cua-driver/rust/Skills/cua-driver/SKILL.md +++ b/libs/cua-driver/rust/Skills/cua-driver/SKILL.md @@ -146,7 +146,7 @@ on-screen a short distance from the target and plays a brief glide + pulse** — not the long Bezier sweep a cursor already on-screen would trace from its previous spot. It's subtle and easy to miss in a recording. If you want a clearly *gliding* cursor for a demo or screen -recording, do a pixel click (`click({pid,x,y})`) or a `move_agent_cursor` +recording, do a pixel click (`click({pid,x,y})`) or a `move_cursor` first to put the cursor on-screen; subsequent AX actions then glide the full path normally. diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs index 257c5ba171..18b3d4ef72 100644 --- a/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs +++ b/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs @@ -12,6 +12,26 @@ pub const RESPONSIBILITY_DISCLAIMED_ENV: &str = "CUA_DRIVER_RS_RESPONSIBILITY_DISCLAIMED"; +/// Embedded mode (`CUA_DRIVER_EMBEDDED=1` / `--embedded`): the driver runs +/// as a direct child of a host app and stays in its TCC responsibility +/// chain — no disclaim re-exec, no daemon relaunch, no permission prompts. +/// See `Skills/cua-driver/EMBEDDING.md`. +/// +/// Caller-controlled, which is safe only because embedded mode strictly +/// REMOVES capability claims; it must never feed into the `driver-daemon` +/// attribution decision (`permission_source` in platform-macos). +pub const EMBEDDED_ENV: &str = "CUA_DRIVER_EMBEDDED"; + +/// Advisory label for the embedding host's bundle id, echoed in +/// `check_permissions` output. NOT a trust signal — trust comes from the +/// OS responsibility chain. +pub const HOST_BUNDLE_ID_ENV: &str = "CUA_DRIVER_HOST_BUNDLE_ID"; + +/// Only the exact value `1` counts — fail-safe for anything else. +pub fn embedded_mode() -> bool { + std::env::var_os(EMBEDDED_ENV).is_some_and(|v| v == "1") +} + pub mod capture_mode; pub mod cdp; pub mod element_cache; diff --git a/libs/cua-driver/rust/crates/cua-driver/src/cli.rs b/libs/cua-driver/rust/crates/cua-driver/src/cli.rs index 6faa203dfb..ffc9501013 100644 --- a/libs/cua-driver/rust/crates/cua-driver/src/cli.rs +++ b/libs/cua-driver/rust/crates/cua-driver/src/cli.rs @@ -140,6 +140,7 @@ const VALUE_FLAGS: &[&str] = &[ "--cursor-icon", "--cursor-id", "--cursor-palette", "--cursor-shape", "--glide-ms", "--dwell-ms", "--idle-hide-ms", "--screenshot-out-file", "--client", "--socket", "--pid-file", "--type", + "--host-bundle-id", // Experimental PiP preview — value flag for the optional geometry // override (--experimental-pip itself is a bare flag and doesn't // need to be listed here). @@ -201,6 +202,10 @@ pub fn parse_command() -> Command { println!("mcp options (macOS):"); println!(" --no-daemon-relaunch Stay in-process; skip auto-launching the CuaDriver daemon."); println!(" Also: CUA_DRIVER_RS_MCP_NO_RELAUNCH=1"); + println!(" --embedded Run embedded inside a host app (also: CUA_DRIVER_EMBEDDED=1)."); + println!(" Inherits the host's TCC grants; never prompts or relaunches."); + println!(" See Skills/cua-driver/EMBEDDING.md."); + println!(" --host-bundle-id Advisory host bundle id label for check_permissions output."); println!(" --socket Override the daemon UDS path used by the proxy fallback."); println!(" --claude-code-computer-use-compat"); println!(" Select the Claude Code computer-use compat surface."); @@ -216,7 +221,7 @@ pub fn parse_command() -> Command { println!(" cursor (keyed by session id) that shows where the agent acts without moving the"); println!(" real pointer. It is removed when the session ends. A pure accessibility (AX)"); println!(" action snaps the cursor with a brief pulse on its first action instead of a long"); - println!(" glide, so it can be easy to miss — do a pixel click or move_agent_cursor first"); + println!(" glide, so it can be easy to miss — do a pixel click or move_cursor first"); println!(" for a visibly gliding demo. These flags tune the overlay on `serve`/`mcp`:"); println!(" --no-overlay Disable the cursor overlay entirely for this daemon."); println!(" --cursor-id Name the default cursor instance (default: 'default')."); @@ -254,6 +259,16 @@ pub fn parse_command() -> Command { let mcp_client = flag_value(&args, "--client"); let socket = flag_value(&args, "--socket"); + // `--embedded` / `--host-bundle-id` export to the environment rather + // than threading through `Command`: all consumers read + // `cua_driver_core::embedded_mode()` and children inherit the mode. + if args.iter().any(|a| a == "--embedded") { + std::env::set_var(cua_driver_core::EMBEDDED_ENV, "1"); + } + if let Some(id) = flag_value(&args, "--host-bundle-id") { + std::env::set_var(cua_driver_core::HOST_BUNDLE_ID_ENV, id); + } + // Strip cursor-overlay flags (and their values) to expose the subcommand. let mut positionals: Vec<&str> = Vec::new(); let mut i = 0; @@ -540,6 +555,12 @@ pub fn run_describe(registry: &ToolRegistry, name: &str) { #[cfg(target_os = "macos")] pub fn should_use_daemon_proxy(no_daemon_relaunch: bool) -> bool { use crate::bundle::{is_env_truthy, is_executable_inside_cuadriver_app, parent_is_not_launchd}; + // Embedded mode stays in-process: relaunching via `open -a CuaDriver` + // would leave the host's TCC responsibility chain and could prompt + // for com.trycua.driver. + if cua_driver_core::embedded_mode() { + return false; + } if no_daemon_relaunch { return false; } @@ -590,6 +611,10 @@ pub fn should_use_daemon_proxy(no_daemon_relaunch: bool) -> bool { #[cfg(not(target_os = "macos"))] pub fn should_use_daemon_proxy(no_daemon_relaunch: bool) -> bool { use crate::bundle::is_env_truthy; + // Same rule as macOS: an embedded driver answers in-process. + if cua_driver_core::embedded_mode() { + return false; + } if no_daemon_relaunch { return false; } @@ -846,14 +871,18 @@ pub fn build_manifest() -> serde_json::Value { "args": [ { "name": "--no-daemon-relaunch", "type": "flag", "description": "Skip the bundle-based TCC auto-relaunch and stay in-process." }, { "name": "--socket", "type": "string", "description": "Override the daemon proxy UDS path." }, - { "name": "--claude-code-computer-use-compat", "type": "flag", "description": "Select the Claude Code computer-use compat tool surface." } + { "name": "--claude-code-computer-use-compat", "type": "flag", "description": "Select the Claude Code computer-use compat tool surface." }, + { "name": "--embedded", "type": "flag", "description": "Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1." }, + { "name": "--host-bundle-id", "type": "string", "description": "Advisory host bundle id label echoed in check_permissions output." } ] }, { "name": "serve", "description": "Run the long-lived daemon — backs the proxy/auto-relaunch path on macOS and the autostart Session 1+ daemon on Windows.", "args": [ { "name": "--socket", "type": "string", "description": "Override the listen socket path." }, { "name": "--no-permissions-gate", "type": "flag", "description": "Skip the macOS TCC first-launch gate." }, - { "name": "--claude-code-computer-use-compat", "type": "flag", "description": "Forwarded by the MCP proxy when the client asked for the compat surface." } + { "name": "--claude-code-computer-use-compat", "type": "flag", "description": "Forwarded by the MCP proxy when the client asked for the compat surface." }, + { "name": "--embedded", "type": "flag", "description": "Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1." }, + { "name": "--host-bundle-id", "type": "string", "description": "Advisory host bundle id label echoed in check_permissions output." } ] }, { "name": "stop", "description": "Stop a running daemon by sending it a shutdown request.", @@ -2019,11 +2048,13 @@ fn cli_docs_json() -> serde_json::Value { "discussion": "On macOS, shell-spawned MCP processes can auto-launch and proxy through a CuaDriver.app daemon so TCC grants attach to the bundle. On Windows and Linux, MCP proxies through an already-running daemon when one is listening.", "arguments": no_args, "options": [ - {"name":"socket","short_name":null,"help":"Override the daemon socket or named-pipe path used by the proxy fallback.","type":"String","default_value":null,"is_optional":true} + {"name":"socket","short_name":null,"help":"Override the daemon socket or named-pipe path used by the proxy fallback.","type":"String","default_value":null,"is_optional":true}, + {"name":"host-bundle-id","short_name":null,"help":"Advisory host bundle id label echoed in check_permissions output (embedded mode).","type":"String","default_value":null,"is_optional":true} ], "flags": [ {"name":"no-daemon-relaunch","short_name":null,"help":"Stay in-process instead of proxying through a daemon.","default_value":false}, - {"name":"claude-code-computer-use-compat","short_name":null,"help":"Expose the Claude Code computer-use compatibility screenshot surface.","default_value":false} + {"name":"claude-code-computer-use-compat","short_name":null,"help":"Expose the Claude Code computer-use compatibility screenshot surface.","default_value":false}, + {"name":"embedded","short_name":null,"help":"Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1.","default_value":false} ], "subcommands": no_subcommands }, @@ -2067,10 +2098,12 @@ fn cli_docs_json() -> serde_json::Value { "arguments": no_args, "options": [ {"name":"socket","short_name":null,"help":"Override the daemon socket or named-pipe path.","type":"String","default_value":null,"is_optional":true}, - {"name":"pid-file","short_name":null,"help":"Override the pid-file path on Unix targets.","type":"String","default_value":null,"is_optional":true} + {"name":"pid-file","short_name":null,"help":"Override the pid-file path on Unix targets.","type":"String","default_value":null,"is_optional":true}, + {"name":"host-bundle-id","short_name":null,"help":"Advisory host bundle id label echoed in check_permissions output (embedded mode).","type":"String","default_value":null,"is_optional":true} ], "flags": [ - {"name":"no-permissions-gate","short_name":null,"help":"Skip the macOS first-launch permissions gate.","default_value":false} + {"name":"no-permissions-gate","short_name":null,"help":"Skip the macOS first-launch permissions gate.","default_value":false}, + {"name":"embedded","short_name":null,"help":"Run embedded inside a host app: inherit the host's TCC grants, never prompt or relaunch. Also CUA_DRIVER_EMBEDDED=1.","default_value":false} ], "subcommands": no_subcommands }, diff --git a/libs/cua-driver/rust/crates/cua-driver/src/responsibility.rs b/libs/cua-driver/rust/crates/cua-driver/src/responsibility.rs index 2ab7ac0435..c09a89bfb9 100644 --- a/libs/cua-driver/rust/crates/cua-driver/src/responsibility.rs +++ b/libs/cua-driver/rust/crates/cua-driver/src/responsibility.rs @@ -17,12 +17,22 @@ pub fn already_disclaimed() -> bool { std::env::var_os(cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV).is_some() } +/// Split out from [`reexec_disclaimed_if_needed`] so the decision is +/// testable without spawning. Embedded mode must skip the disclaim: +/// disclaiming would make the driver its own responsible process and +/// break TCC inheritance from the host. +#[cfg(target_os = "macos")] +fn should_skip_disclaim(embedded: bool, already_disclaimed: bool, inside_bundle: bool) -> bool { + embedded || already_disclaimed || inside_bundle +} + #[cfg(target_os = "macos")] pub fn reexec_disclaimed_if_needed() { - if already_disclaimed() { - return; - } - if crate::bundle::is_executable_inside_cuadriver_app() { + if should_skip_disclaim( + cua_driver_core::embedded_mode(), + already_disclaimed(), + crate::bundle::is_executable_inside_cuadriver_app(), + ) { return; } @@ -164,6 +174,15 @@ pub fn reexec_disclaimed_if_needed() {} mod tests { use super::*; + #[test] + fn embedded_mode_skips_disclaim_reexec() { + assert!(should_skip_disclaim(true, false, false)); + // A bare standalone binary must still disclaim. + assert!(!should_skip_disclaim(false, false, false)); + assert!(should_skip_disclaim(false, true, false)); + assert!(should_skip_disclaim(false, false, true)); + } + #[test] fn already_disclaimed_reflects_env_var() { let name = cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV; diff --git a/libs/cua-driver/rust/crates/platform-macos/src/permissions/gate.rs b/libs/cua-driver/rust/crates/platform-macos/src/permissions/gate.rs index d386354c29..6c27f72324 100644 --- a/libs/cua-driver/rust/crates/platform-macos/src/permissions/gate.rs +++ b/libs/cua-driver/rust/crates/platform-macos/src/permissions/gate.rs @@ -165,9 +165,11 @@ impl Default for GateOpts { impl GateOpts { /// Construct from the standard env-var /// (`CUA_DRIVER_RS_PERMISSIONS_GATE` set to `0` / `false` / `no` / - /// `off`, case-insensitive, disables the gate) and an explicit - /// `--no-permissions-gate` flag. Either signal is sufficient to opt - /// out. + /// `off`, case-insensitive, disables the gate), an explicit + /// `--no-permissions-gate` flag, and embedded mode + /// (`CUA_DRIVER_EMBEDDED=1`). Any signal is sufficient to opt out. + /// Embedded mode opts out because the host app owns the grant flow; + /// the driver must never raise its own prompts. pub fn from_env_and_flag(no_gate_flag: bool) -> Self { // Match the standard list of "off" sentinels case-insensitively so // CI scripts can use any of `0`, `false`, `no`, `off`, `FALSE`, @@ -181,7 +183,7 @@ impl GateOpts { }) .unwrap_or(false); Self { - opt_out: no_gate_flag || env_disabled, + opt_out: no_gate_flag || env_disabled || cua_driver_core::embedded_mode(), ..Self::default() } } @@ -595,22 +597,11 @@ fn fmt_missing(missing: &[MissingPermission]) -> String { #[cfg(test)] mod tests { use super::*; - use std::sync::{Mutex, OnceLock}; - - /// Serializes every test that mutates `CUA_DRIVER_RS_PERMISSIONS_GATE`. - /// `cargo test` runs tests in parallel by default and `std::env::set_var` - /// / `remove_var` touch a process-global table — without this lock the - /// env-var tests race and produce flaky failures. - static TEST_ENV_MUTEX: OnceLock> = OnceLock::new(); + /// Crate-wide env-var test lock — `from_env_and_flag` reads + /// `CUA_DRIVER_EMBEDDED`, which the `check_permissions` tests mutate. fn env_lock() -> std::sync::MutexGuard<'static, ()> { - // `lock()` can only fail if a previous holder panicked. Recover the - // guard and keep going — the env var will be re-set/cleared by this - // test anyway, so a poisoned mutex carries no stale invariant. - TEST_ENV_MUTEX - .get_or_init(|| Mutex::new(())) - .lock() - .unwrap_or_else(|e| e.into_inner()) + crate::permissions::test_env_lock() } #[test] @@ -652,10 +643,23 @@ mod tests { fn neither_flag_nor_env_does_not_opt_out() { let _guard = env_lock(); std::env::remove_var("CUA_DRIVER_RS_PERMISSIONS_GATE"); + std::env::remove_var(cua_driver_core::EMBEDDED_ENV); let opts = GateOpts::from_env_and_flag(false); assert!(!opts.opt_out); } + #[test] + fn embedded_mode_opts_out_of_gate() { + let _guard = env_lock(); + std::env::remove_var("CUA_DRIVER_RS_PERMISSIONS_GATE"); + std::env::set_var(cua_driver_core::EMBEDDED_ENV, "1"); + assert!(GateOpts::from_env_and_flag(false).opt_out); + // Only the exact value "1" enables embedded mode. + std::env::set_var(cua_driver_core::EMBEDDED_ENV, "true"); + assert!(!GateOpts::from_env_and_flag(false).opt_out); + std::env::remove_var(cua_driver_core::EMBEDDED_ENV); + } + #[test] fn env_var_truthy_values_do_not_opt_out() { let _guard = env_lock(); diff --git a/libs/cua-driver/rust/crates/platform-macos/src/permissions/mod.rs b/libs/cua-driver/rust/crates/platform-macos/src/permissions/mod.rs index 6d96a019f9..c5061e689c 100644 --- a/libs/cua-driver/rust/crates/platform-macos/src/permissions/mod.rs +++ b/libs/cua-driver/rust/crates/platform-macos/src/permissions/mod.rs @@ -24,3 +24,15 @@ pub mod panel; pub use status::{PermissionsStatus, current_status}; pub use gate::{GateOpts, MissingPermission, run_if_needed}; + +/// Crate-wide lock serializing tests that mutate process-global env vars. +/// Per-module locks are not enough: `gate` and `check_permissions` tests +/// share `CUA_DRIVER_EMBEDDED`. +#[cfg(test)] +pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> { + static LOCK: std::sync::OnceLock> = std::sync::OnceLock::new(); + // Poison carries no stale invariant (tests restore the vars they touch). + LOCK.get_or_init(|| std::sync::Mutex::new(())) + .lock() + .unwrap_or_else(|e| e.into_inner()) +} diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/check_permissions.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/check_permissions.rs index 51ad0a53e2..60d3f550b2 100644 --- a/libs/cua-driver/rust/crates/platform-macos/src/tools/check_permissions.rs +++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/check_permissions.rs @@ -44,6 +44,31 @@ fn permission_source() -> serde_json::Value { .and_then(|p| std::fs::canonicalize(p).ok()) .and_then(|p| p.to_str().map(str::to_owned)) .unwrap_or_default(); + let disclaimed = + std::env::var_os(cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV).is_some(); + // Embedded mode: the driver is a child in a host app's responsibility + // chain, so the probes already answer for the host's TCC identity. + // This branch only ever downgrades attribution (host, never + // driver-daemon), so the caller-controlled env var can't spoof an + // elevated identity. `host_bundle_id` is advisory, not a trust signal. + if cua_driver_core::embedded_mode() { + let host_bundle_id = std::env::var(cua_driver_core::HOST_BUNDLE_ID_ENV) + .unwrap_or_default(); + return serde_json::json!({ + "attribution": "host", + "host_bundle_id": host_bundle_id, + "embedded": true, + "pid": pid, + "responsible_ppid": ppid, + "executable": exe, + "disclaim_env": disclaimed, + "note": "Embedded mode: these booleans reflect the HOST app's TCC \ + grant (the driver is a child in the host's responsibility \ + chain). No separate driver grant exists or is needed. If a \ + permission is NOT granted, the host app must request it — \ + the driver never raises its own prompt.", + }); + } // The trustworthy, non-spoofable signal is the executable path: a caller // can't run from inside the code-signed `CuaDriver.app` bundle without // controlling that install. The disclaim env var is caller-controlled, so @@ -54,8 +79,6 @@ fn permission_source() -> serde_json::Value { // a caller could pre-set it and spoof the TCC source. Fail closed to // "caller" whenever the bundle signal is absent. let inside_bundle = exe.contains("/CuaDriver.app/Contents/MacOS/"); - let disclaimed = - std::env::var_os(cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV).is_some(); let is_driver_daemon = inside_bundle && (ppid == 1 || disclaimed); let (attribution, note) = if is_driver_daemon { @@ -81,6 +104,7 @@ fn permission_source() -> serde_json::Value { "pid": pid, "responsible_ppid": ppid, "executable": exe, + "disclaim_env": disclaimed, "note": note, }) } @@ -129,7 +153,12 @@ impl Tool for CheckPermissionsTool { async fn invoke(&self, args: Value) -> ToolResult { use cua_driver_core::tool_args::ArgsExt; // Default to prompting — same default + rationale as Swift. - let should_prompt = args.bool_or("prompt", true); + // Embedded mode hard-disables prompting regardless of the arg (the + // host owns the grant flow). This and the startup gate are the only + // `request_*` call sites, so both being gated makes prompts + // unreachable when embedded. + let should_prompt = + args.bool_or("prompt", true) && !cua_driver_core::embedded_mode(); if should_prompt { let _ = request_accessibility(); let _ = request_screen_recording(); @@ -158,7 +187,15 @@ impl Tool for CheckPermissionsTool { the grant likely belongs to a different process, not this one.", ); } - // Make the attribution explicit when answering for the caller (not the daemon). + // Make the attribution explicit when answering for a host or caller + // (not the daemon). + if source.get("attribution").and_then(|v| v.as_str()) == Some("host") { + summary.push_str( + "\nℹ️ Embedded mode: status reflects the HOST app's TCC grant. \ + If a permission is missing, the host must request it — the \ + driver will not prompt.", + ); + } if is_caller { summary.push_str( "\nℹ️ Status reflects the launching app's TCC identity, not the CuaDriver \ @@ -180,6 +217,28 @@ impl Tool for CheckPermissionsTool { mod tests { use super::*; + fn env_lock() -> std::sync::MutexGuard<'static, ()> { + crate::permissions::test_env_lock() + } + + /// Set/remove `var`, returning the original for restore. Callers must + /// hold `env_lock()`. + fn swap_env(var: &str, value: Option<&str>) -> Option { + let original = std::env::var_os(var); + match value { + Some(v) => std::env::set_var(var, v), + None => std::env::remove_var(var), + } + original + } + + fn restore_env(var: &str, original: Option) { + match original { + Some(value) => std::env::set_var(var, value), + None => std::env::remove_var(var), + } + } + #[test] fn disclaim_env_var_alone_does_not_grant_daemon_attribution() { // The disclaim env var is caller-controlled, so on its own it must not @@ -187,10 +246,11 @@ mod tests { // identity. Daemon attribution additionally requires the binary to live // inside the code-signed `CuaDriver.app` bundle — the test runner does // not, so even with the env var present we must fail closed to "caller". + let _guard = env_lock(); let name = cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV; - let original = std::env::var_os(name); + let original = swap_env(name, Some("1")); + let embedded = swap_env(cua_driver_core::EMBEDDED_ENV, None); - std::env::set_var(name, "1"); let source = permission_source(); assert_eq!( source.get("attribution").and_then(|v| v.as_str()), @@ -198,9 +258,59 @@ mod tests { "env-var presence alone must not yield daemon attribution" ); - match original { - Some(value) => std::env::set_var(name, value), - None => std::env::remove_var(name), - } + restore_env(cua_driver_core::EMBEDDED_ENV, embedded); + restore_env(name, original); + } + + #[test] + fn embedded_mode_reports_host_attribution() { + let _guard = env_lock(); + let embedded = swap_env(cua_driver_core::EMBEDDED_ENV, Some("1")); + let host = swap_env(cua_driver_core::HOST_BUNDLE_ID_ENV, Some("com.example.host")); + + let source = permission_source(); + assert_eq!( + source.get("attribution").and_then(|v| v.as_str()), + Some("host"), + ); + assert_eq!( + source.get("host_bundle_id").and_then(|v| v.as_str()), + Some("com.example.host"), + ); + assert_eq!(source.get("embedded").and_then(|v| v.as_bool()), Some(true)); + + restore_env(cua_driver_core::HOST_BUNDLE_ID_ENV, host); + restore_env(cua_driver_core::EMBEDDED_ENV, embedded); + } + + #[test] + fn embedded_plus_disclaim_env_never_yields_daemon_attribution() { + // Both caller-controlled env vars together must still not produce + // "driver-daemon" — embedded mode may only DOWNGRADE attribution. + let _guard = env_lock(); + let embedded = swap_env(cua_driver_core::EMBEDDED_ENV, Some("1")); + let disclaim = swap_env(cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV, Some("1")); + + let source = permission_source(); + assert_eq!( + source.get("attribution").and_then(|v| v.as_str()), + Some("host"), + ); + + restore_env(cua_driver_core::RESPONSIBILITY_DISCLAIMED_ENV, disclaim); + restore_env(cua_driver_core::EMBEDDED_ENV, embedded); + } + + #[test] + fn embedded_env_requires_exact_value_one() { + let _guard = env_lock(); + let embedded = swap_env(cua_driver_core::EMBEDDED_ENV, Some("true")); + let source = permission_source(); + assert_ne!( + source.get("attribution").and_then(|v| v.as_str()), + Some("host"), + "only CUA_DRIVER_EMBEDDED=1 may enable embedded mode" + ); + restore_env(cua_driver_core::EMBEDDED_ENV, embedded); } } diff --git a/libs/cua-driver/rust/examples/embedded-host-macos/.gitignore b/libs/cua-driver/rust/examples/embedded-host-macos/.gitignore new file mode 100644 index 0000000000..699c18a9b4 --- /dev/null +++ b/libs/cua-driver/rust/examples/embedded-host-macos/.gitignore @@ -0,0 +1 @@ +ExampleAgentHarness.app/ diff --git a/libs/cua-driver/rust/examples/embedded-host-macos/ExampleAgentHarness.swift b/libs/cua-driver/rust/examples/embedded-host-macos/ExampleAgentHarness.swift new file mode 100644 index 0000000000..4522ad2240 --- /dev/null +++ b/libs/cua-driver/rust/examples/embedded-host-macos/ExampleAgentHarness.swift @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 Cua AI, Inc. + +// ExampleAgentHarness — minimal reference host for embedding cua-driver. +// Mirrored verbatim in Skills/cua-driver/EMBEDDING.md ("Minimal host +// example") — keep the two in sync. +// +// Runs the one-grant demo sequence from EMBEDDING.md end to end: +// 1. Requests Accessibility + Screen Recording AS THE HOST (the only +// prompts the user ever sees), then +// 2. spawns cua-driver as a direct child in embedded mode and, over +// stdio MCP, verifies attribution, takes a background screenshot, +// reads a background app's window state, and glides the agent-cursor +// overlay — with zero driver-side prompts. +// +// Launched via `open` (see demo.sh) the app has no terminal, so all +// output also goes to /tmp/cua-embedded-demo.log. + +import Foundation +import ApplicationServices +import CoreGraphics + +let logPath = "/tmp/cua-embedded-demo.log" +FileManager.default.createFile(atPath: logPath, contents: nil) +let logFile = FileHandle(forWritingAtPath: logPath)! +func log(_ line: String) { + print(line) + logFile.write((line + "\n").data(using: .utf8)!) +} + +// 1. Request both grants AS THE HOST — the only prompts in the whole flow. +let axOpts = ["AXTrustedCheckOptionPrompt": true] as CFDictionary +let ax = AXIsProcessTrustedWithOptions(axOpts) +let sr = CGRequestScreenCaptureAccess() +log("host grants — accessibility: \(ax), screen recording: \(sr)") +// Keep going even without grants: the run registers BOTH rows in one pass +// (the AX request above, plus — on newer macOS, where the app only appears +// in the Screen Recording pane after a real ScreenCaptureKit attempt — the +// embedded driver's live probe below, registered as THE HOST, which is the +// point of embedding). Grant both in one Settings visit, then re-run. +if !ax || !sr { + log("after this run: grant the missing item(s) in System Settings, then re-run") +} + +// 2. Spawn cua-driver as a DIRECT child (never via `open`/NSWorkspace — +// that breaks responsibility inheritance) in embedded mode. +let driverPath = ProcessInfo.processInfo.environment["CUA_DRIVER_PATH"] + ?? "/usr/local/bin/cua-driver" +let driver = Process() +driver.executableURL = URL(fileURLWithPath: driverPath) +driver.arguments = ["mcp"] +var env = ProcessInfo.processInfo.environment +env["CUA_DRIVER_EMBEDDED"] = "1" +env["CUA_DRIVER_HOST_BUNDLE_ID"] = Bundle.main.bundleIdentifier ?? "" +driver.environment = env +let toDriver = Pipe(), fromDriver = Pipe() +driver.standardInput = toDriver +driver.standardOutput = fromDriver +try driver.run() + +// 3. Line-delimited JSON-RPC 2.0 over the child's stdio. +var buffer = Data() +func send(_ obj: [String: Any]) { + var data = try! JSONSerialization.data(withJSONObject: obj) + data.append(0x0A) + toDriver.fileHandleForWriting.write(data) +} +func readMessage() -> [String: Any] { + while true { + if let nl = buffer.firstIndex(of: 0x0A) { + let line = buffer.subdata(in: buffer.startIndex.. [String: Any] { + nextId += 1 + send(["jsonrpc": "2.0", "id": nextId, "method": "tools/call", + "params": ["name": tool, "arguments": args]]) + while true { + let msg = readMessage() + if msg["id"] as? Int == nextId { + if let error = msg["error"] as? [String: Any] { + log("RPC error for \(tool): \(error)") + } + return msg["result"] as? [String: Any] ?? [:] + } + } +} + +nextId += 1 +send(["jsonrpc": "2.0", "id": nextId, "method": "initialize", "params": [ + "protocolVersion": "2024-11-05", "capabilities": [:], + "clientInfo": ["name": "ExampleAgentHarness", "version": "0.1"]]]) +_ = readMessage() +send(["jsonrpc": "2.0", "method": "notifications/initialized"]) +log("embedded cua-driver started (\(driverPath)) — no driver prompt should have appeared") + +// 4. check_permissions must report attribution "host" and never prompt. +let perms = call("check_permissions") +let structured = perms["structuredContent"] as? [String: Any] ?? [:] +let source = structured["source"] as? [String: Any] ?? [:] +let attribution = source["attribution"] as? String ?? "?" +log("check_permissions — attribution: \(attribution) (want: host), " + + "capturable: \(structured["screen_recording_capturable"] ?? "?")") + +// 5. Background AX read + window screenshot — proves both grants +// inherited without focusing anything. launch_app resolves pid + +// windows without foregrounding; get_window_state returns the AX +// element tree AND a screenshot of the (background) window. +let launch = call("launch_app", ["bundle_id": "com.apple.finder"]) +let launched = launch["structuredContent"] as? [String: Any] ?? [:] +let pid = launched["pid"] as? Int ?? 0 +let windows = launched["windows"] as? [[String: Any]] ?? [] +let windowId = windows.first?["window_id"] as? Int ?? 0 +log("launch_app(Finder) — pid: \(pid), windows: \(windows.count)") + +let state = call("get_window_state", ["pid": pid, "window_id": windowId]) +let images = (state["content"] as? [[String: Any]] ?? []) + .filter { $0["type"] as? String == "image" } +let hasTree = (state["structuredContent"] as? [String: Any])?["elements"] != nil +log("get_window_state(Finder) — tree: \(hasTree ? "ok" : "EMPTY"), " + + "screenshot: \(images.count) image(s) (want: ≥1)") + +// 6. Agent-cursor glide — shows the overlay, no real-pointer move. +log("watch the agent cursor glide now (no real-pointer move)…") +let cursor1 = call("move_cursor", ["x": 200, "y": 200]) +Thread.sleep(forTimeInterval: 2) +let cursor2 = call("move_cursor", ["x": 900, "y": 500]) +Thread.sleep(forTimeInterval: 2) +let cursorOk = (cursor1["isError"] as? Bool) != true && + (cursor2["isError"] as? Bool) != true +log("move_cursor — \(cursorOk ? "ok" : "FAILED")") + +let pass = attribution == "host" && !images.isEmpty && hasTree && cursorOk +log(pass ? "DEMO COMPLETE: PASS" : "DEMO COMPLETE: FAIL") +driver.terminate() +exit(pass ? 0 : 1) diff --git a/libs/cua-driver/rust/examples/embedded-host-macos/demo.sh b/libs/cua-driver/rust/examples/embedded-host-macos/demo.sh new file mode 100755 index 0000000000..7ece8cd9f7 --- /dev/null +++ b/libs/cua-driver/rust/examples/embedded-host-macos/demo.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# Build + run the one-grant embedding demo on a freshly reset TCC state. +# MANUAL — needs a real Mac with a user at the keyboard; TCC prompts cannot +# be granted from CI. See ../../Skills/cua-driver/EMBEDDING.md for the pass +# criteria. +# +# ./demo.sh # ad-hoc signature (fine for local demos) +# SIGN_IDENTITY="Developer ID Application: …" ./demo.sh +set -euo pipefail +cd "$(dirname "$0")" + +APP=ExampleAgentHarness.app +BUNDLE_ID=${BUNDLE_ID:-com.trycua.example-agent-harness} +SIGN_IDENTITY=${SIGN_IDENTITY:--} # "-" = ad-hoc +LOG=/tmp/cua-embedded-demo.log + +# Rebuilding re-signs the bundle; a new (ad-hoc) signature can orphan an +# existing TCC grant row, so keep the same build across the grant + re-run +# flow. `rm -rf ExampleAgentHarness.app` to force a rebuild. +if [ -d "$APP" ]; then + echo "1/4 reusing existing ${APP} (delete it to force a rebuild)" +else +echo "1/4 building ${APP}…" +mkdir -p "$APP/Contents/MacOS" +swiftc -O ExampleAgentHarness.swift -o "$APP/Contents/MacOS/ExampleAgentHarness" \ + -framework ApplicationServices +cat > "$APP/Contents/Info.plist" < + + + + CFBundleExecutableExampleAgentHarness + CFBundleIdentifier${BUNDLE_ID} + CFBundleNameExampleAgentHarness + CFBundlePackageTypeAPPL + CFBundleShortVersionString0.1 + LSMinimumSystemVersion13.0 + + +PLIST +# A stable code-signing identity is what keys the TCC grant rows to this +# app; re-signing later orphans existing grants (see EMBEDDING.md). +codesign --force --sign "$SIGN_IDENTITY" "$APP" +fi + +# Re-runs after granting must skip the reset or they wipe the grant again. +if [ -z "${SKIP_RESET:-}" ]; then + echo "2/4 resetting TCC state for ${BUNDLE_ID} (expect re-prompts)…" + tccutil reset Accessibility "$BUNDLE_ID" || true + tccutil reset ScreenCapture "$BUNDLE_ID" || true +else + echo "2/4 SKIP_RESET set — keeping existing grants" +fi + +echo "3/4 launching the host — grant BOTH prompts, then re-run with" +echo " SKIP_RESET=1 ./demo.sh (TCC answers are cached per process)." +echo " (open(1) is correct HERE: the HOST must be its own responsible" +echo " process; it then spawns cua-driver directly as its child.)" +# `open` does not forward shell env; route a custom driver path through +# launchd so the app can see it. +if [ -n "${CUA_DRIVER_PATH:-}" ]; then + launchctl setenv CUA_DRIVER_PATH "$CUA_DRIVER_PATH" + trap 'launchctl unsetenv CUA_DRIVER_PATH' EXIT +fi +rm -f "$LOG" +open "$APP" + +echo "4/4 following $LOG — expect attribution 'host', a screenshot," +echo " Finder window state, a visible agent-cursor glide, and" +echo " 'DEMO COMPLETE: PASS'. Then verify System Settings → Privacy &" +echo " Security lists ONLY ExampleAgentHarness (no CuaDriver)." +# Portable wait (stock macOS has no timeout(1)). +for _ in $(seq 1 120); do + grep -q "DEMO COMPLETE\|grant the missing item(s)" "$LOG" 2>/dev/null && break + sleep 1 +done +cat "$LOG" 2>/dev/null + +grep -q "DEMO COMPLETE: PASS" "$LOG" && echo "✅ one-grant demo passed" || { + echo "❌ demo failed — debug attribution with:" + echo " log stream --debug --predicate 'subsystem == \"com.apple.TCC\" AND eventMessage BEGINSWITH \"AttributionChain\"'" + exit 1 +}