feat(cua-driver): implement macOS LaunchAgent autostart by x7peeps · Pull Request #2990 · trycua/cua

x7peeps · 2026-08-08T00:11:48Z

feat(cua-driver): macOS LaunchAgent autostart (enable/disable/status/kick)

Story: from a broken local session to a working autostart

This PR is the upstream contribution that closes
#2196 — and it carries the full
story of how we got here, because the debugging journey is the proof that the
design works.

1. The local session that would not run

We integrate cua-driver as the computer-use backend of Hermes Agent
(NousResearch/hermes-agent). On a macOS 26.5.2 (Apple Silicon) machine the
computer-use toolset failed in a very specific way:

capture could identify 358 accessibility elements and AXPress clicks
succeeded ("✅ Performed AXPress"), but every SOM bound came back as
(0,0,0,0) — so coordinate-based clicking was impossible.
The installed cua-driver was 0.2.0, whose get_window_state does not
return structuredContent.elements. The Hermes backend therefore fell back
to a markdown regex parser and every element frame was 0.

Diagnosis chain (what we ruled out before the real cause):

❌ "Missing macOS Accessibility permission" — the AX tree was readable, so
permission was fine (a permission failure yields an empty tree).
✅ Real root cause: the old driver version. Upgrading to 0.19.1
(structured frames landed upstream, see feat(cua-driver): cursor overhaul + Hermes-decoupling MCP surface #1961) fixed bounds.

2. Discovering the project while debugging

While chasing the (0,0,0,0) bounds we found this repo — trycua/cua, the
open-source computer-use 2.0 driver stack. The debugging trail ran straight
through its code:

tools/computer_use/cua_backend.py in Hermes already knew about
structuredContent.elements[].frame (the fix from feat(cua-driver): cursor overhaul + Hermes-decoupling MCP surface #1961);
cua-driver autostart turned out to be Windows-only; macOS/Linux are
stubbed with NOT_YET in autostart.rs;
the manual recipe (libs/cua-driver/scripts/install-local.sh --autostart)
writes a LaunchAgent, but with the legacy launchctl load/unload API.

3. Research + capability build-out (what we actually ran)

We upgraded and verified the full stack on our machine:

Step	What	Evidence
Upgrade	cua-driver 0.2.0 → 0.19.1 (39MB universal asset via ghfast.top mirror; GitHub direct was SSL-blocked from CN)	`cua-driver --version`
Rebuild app	TCC integrity protection blocks overwriting an authorized app binary (`cp`/`mv` → EPERM, `rm` allowed) → deleted and rebuilt `CuaDriver.app` with the 0.19.1 binaries, ad-hoc re-signed	`codesign -dv`
TCC	New binary = new cdhash = grants invalidated (by design) → re-granted Accessibility + Screen Recording for the driver identity	`cua-driver permissions status`
Daemon identity	`~/.local/bin/cua-driver serve` (CLI identity) matches the TCC grant; `open -a CuaDriver --args serve` does not	`ps aux` + capture bounds
Autostart	Wrote `~/Library/LaunchAgents/com.trycua.driver.plist` (RunAtLoad + KeepAlive) → launchd manages the daemon at every logon	`launchctl print gui/501/com.trycua.driver` → `state = running`
E2E	After a fresh logon (daemon auto-started by launchd), capture returned 782 elements with real coordinates, and coordinate/AXPress clicks switched Docker pages	capture + click logs

Key insight from the field: launchd-started daemons are the supported TCC
path. A launchd-managed cua-driver serve carries the stable
com.trycua.driver responsibility identity, so Accessibility / Screen
Recording grants persist across logons — exactly what #2196 describes wanting.

4. Landing it: this PR

This PR implements the macOS (and by extension, the Unix) autostart path
natively in autostart.rs, replacing the stubs with a real LaunchAgent
implementation:

enable — writes a LaunchAgent plist under ~/Library/LaunchAgents/
(label derived from the bundle identity, matching the Windows task-name
pattern), then launchctl bootstrap gui/<uid> <plist> (modern API; the
legacy install script uses load, we use bootstrap/bootout).
disable — launchctl bootout gui/<uid>/<label> + removes the plist
(no-op when absent).
status — reports not-registered / registered (not running) /
registered (running), reusing the existing daemon-socket liveness check
(serve::is_daemon_listening) exactly like the Windows path does.
kick — launchctl kickstart -k gui/<uid>/<label> to (re)start the
autostart entry for the current session.

Platform mapping now:

Platform	Mechanism	Entry name
Windows	Scheduled Task (RunLevel=Highest, hidden console)	`cua-driver-serve` (release) / `cua-driver-local-serve` (local)
macOS	LaunchAgent (RunAtLoad + KeepAlive)	`com.trycua.driver` (release) / `com.trycua.cua-driver-local` (local)
Linux	systemd user unit (unchanged stub behavior → actionable error pointing at install-local.sh)	—

Why this design

KeepAlive instead of RestartCount: macOS launchd's KeepAlive=true
gives the same crash-restart semantics as the Windows RestartCount 3 in
one key, and is the native idiom.
bootstrap/bootout over load/unload: the modern launchctl API
(macOS 13+); the legacy load used by the install script still works but
is deprecated on current macOS.
Status reuses the socket probe: the same is_daemon_listening call the
Windows status uses, so behavior is consistent across platforms and we
don't add a second liveness mechanism.
Exe path is canonicalized (already the non-Windows behavior of
current_exe_for_autostart) so the LaunchAgent keeps working after
upgrades that flip symlinks — the mirror of the Windows junction
preservation (fix(cua-driver): keep Windows autostart on the junction path #2809).

Verification (run on this machine, macOS 26.5.2)

cargo test -p cua-driver autostart        # unit tests
cua-driver autostart status                # not-registered (before)
cua-driver autostart enable                # Registered autostart entry
launchctl print gui/501/com.trycua.driver  # state = running, pid present
cua-driver autostart status                # registered (running)
cua-driver autostart disable               # removed; plist gone

Known gaps

Linux (systemd --user) remains a stub; the error message now points at
the same manual recipe. Implementing it is a natural follow-up with the
same enable/disable/status/kick shape.
We verified on Apple Silicon (aarch64); Intel macOS should behave the same
(pure POSIX + launchctl), but has not been exercised.
TCC grants must still be given once per binary identity (cdhash changes on
upgrade invalidate them — upstream behavior, documented in the
troubleshooting guide).

Who should enable this

Anyone integrating cua-driver as a background computer-use daemon on macOS
(Hermes, Codex, Cursor, OpenClaw, ...) who currently has to hand-write a
LaunchAgent or paste a startup one-liner.
Users of the Hermes computer-use toolset on macOS who want the daemon to
survive reboots without manual cua-driver serve.

Quick-start

cua-driver autostart enable     # register LaunchAgent + start now
cua-driver autostart status     # verify
cua-driver autostart disable    # remove

Troubleshooting

Symptom	Cause	Fix
`autostart status` says `not-registered` right after `enable`	plist not bootstrapped	re-run `enable`; check `~/Library/LaunchAgents/com.trycua.driver.plist` exists
daemon runs but `permissions status` reports `daemon_running: false`	CLI probe identity differs from the launchd identity (known upstream quirk)	ignore; verify via `launchctl print gui/$(id -u)/com.trycua.driver` and real capture bounds
daemon starts at logon but can't read the screen	TCC grant lost (binary upgraded → cdhash changed)	re-grant Accessibility + Screen Recording for the driver identity once
`launchctl bootstrap` exits non-zero	entry already bootstrapped	`cua-driver autostart disable` then `enable`

Platform compatibility

macOS 13+ (uses launchctl bootstrap/kickstart, unavailable before that;
macOS 12 and older would need load — out of scope, the manual recipe
still covers them).
Linux: stub preserved, actionable error.
Windows: untouched (existing Scheduled Task path).

Closes trycua#2196. ## Summary Implements the macOS autostart path natively in `autostart.rs`, replacing the Windows-only stub with a real LaunchAgent implementation: - `enable` — writes `~/Library/LaunchAgents/com.trycua.driver.plist` (RunAtLoad + KeepAlive), then `launchctl bootstrap gui/<uid> <plist>` (modern API) and `kickstart -k` to start it for the current session. - `disable` — `launchctl bootout gui/<uid>/<label>` + removes the plist (no-op when absent). - `status` — reports `not-registered` / `registered (not running)` / `registered (running)` using the existing daemon-socket liveness probe (`serve::is_daemon_listening`), mirroring the Windows path. - `kick` — `launchctl kickstart -k gui/<uid>/<label>`. The LaunchAgent label is the bundle identity (`com.trycua.driver` release / `com.trycua.cua-driver-local` local), so the launchd-started daemon carries the same TCC responsibility identity as the signed app — Accessibility and Screen Recording grants survive logons. This is the exact problem described in trycua#2196. ## Why - **Hermes and other integrations** currently have no supported way to ensure the daemon is running before background clients initialize computer-use MCP on macOS. The only path today is the manual `install-local.sh --autostart` recipe, which uses the legacy `launchctl load/unload` API and does not expose `enable/disable/status/kick` subcommands. - The `status`/`disable` semantics now match Windows: query failures are reported distinctly from "not registered", and removing an absent entry is a no-op. ## Design notes - **KeepAlive** gives the same crash-restart semantics as the Windows `RestartCount 3` in a single plist key, the native launchd idiom. - **`bootstrap`/`bootout` over `load`/`unload`**: modern launchctl API (macOS 13+); the legacy `load` still works but is deprecated. - **Status reuses the socket probe** — one liveness mechanism across platforms. - **`current_exe_for_autostart` already canonicalizes on non-Windows** so the LaunchAgent keeps working after upgrades that flip symlinks (mirror of the Windows junction preservation, trycua#2809). - Linux remains a stub (unchanged), with an error pointing at the manual systemd user-unit recipe — a natural follow-up with the same shape. ## Verification - `cargo check -p cua-driver --tests` — clean. - `cargo test -p cua-driver --bin cua-driver autostart::` — 11 passed (7 existing + 4 new macOS `xml_escape` tests). - E2E on macOS 26.5.2 (Apple Silicon): `status` → `registered (running)`; `disable` → plist removed, `not-registered`; `enable` → plist written with correct Label/ProgramArguments/KeepAlive, launchd `state = running`, pid present; `kick` → entry restarted; then cleanup and the production daemon was restored and verified. - `cargo fmt -- --check` clean (ci-rust-format).

x7peeps · 2026-08-08T17:10:23Z

Companion Hermes-side diagnostic: NousResearch/hermes-agent#81845 adds a daemon_autostart check to hermes computer-use doctor that probes cua-driver autostart status — it degrades to skip until this implementation ships in a release. This PR provides the upstream capability; #81845 provides the verification.

…bleshooting ## Story: from a broken local session to a diagnostic While integrating cua-driver 0.19+ as the Hermes computer-use backend on macOS, we hit a wall: `capture` saw 358 accessibility elements and AXPress clicks succeeded, but every SOM bound came back `(0,0,0,0)` — coordinate clicks were impossible. The root cause was an old driver (pre-0.19 `get_window_state` has no structured frames), and after upgrading we discovered the real operational gap: **nothing told us whether the daemon would survive a reboot.** This PR turns that debugging journey into a permanent diagnostic: ## What changed ### `tools/computer_use/doctor.py` — new `daemon_autostart` check `hermes computer-use doctor` now probes `cua-driver autostart status` and reports whether the serve daemon is registered to start at every logon: - `pass` — autostart registered (daemon running, or idle but registered) - `fail` — `not-registered`: daemon must be started manually after each logon - `skip` — binary unresolved, or driver too old to support the subcommand (pre-0.19.2 returns a "Windows-only" stub error); never fatal This is the check that would have told us immediately why the daemon kept disappearing after reboots, before we ever had to dig through launchd. ### `skills/.../hermes-agent/references/troubleshooting.md` — Computer use section New "Computer use not working (cua-driver)" section capturing the field knowledge: - `(0,0,0,0)` bounds / AXPress-works-but-coordinates-fail → old driver, upgrade path included - daemon dies on reboot → `cua-driver autostart enable` (macOS LaunchAgent / Windows Scheduled Task), and why the LaunchAgent label matching the TCC identity matters for grants persisting across logons - macOS grants granted but still can't capture → daemon must run under the CLI identity, not `open -a CuaDriver --args serve` - `permissions status` reports `daemon_running: false` while capture works → known probe quirk with launchd-managed daemons; trust launchctl + real bounds instead - `cp/mv` of CuaDriver.app → "Operation not permitted" → TCC app-integrity protection: rm + rebuild, then re-grant ## Why this shape - **CLI probe, not a new core tool**: the check reuses the existing `resolve_cua_driver_cmd()` and runs one subprocess. No new MCP surface, no schema change, no new env var — the "smallest footprint" path. - **Graceful degradation**: on drivers where `autostart` is a stub (anything pre-0.19.2), the check reports `skip` with context instead of failing doctor. Nothing about the existing 0.10 health_report fallback changes. - **Docs over code where possible**: the troubleshooting knowledge is deliberately in the skill reference so it loads into any session that hits the symptom, not buried in a code comment. ## Verification - `pytest tests/computer_use/test_doctor.py` — 22 passed (15 existing + 7 new for `_autostart_probe`). The new tests caught two real substring bugs during development (`not-registered` contains `registered`; `not running` contains `running`) — order of checks matters. - `pytest tests/computer_use/` — 51 passed. - Live on macOS 26.5.2 with cua-driver 0.19.1: doctor reports `skip` (old stub) gracefully; with a 0.19.2 debug build the probe reports `pass: autostart registered and daemon running` and `fail: not-registered` when disabled — both exercised against a real launchd-managed daemon. ## Platform compatibility - macOS: probes LaunchAgent registration via the CLI (works with cua-driver 0.19.2+, which implements `autostart` for macOS). - Windows: same CLI probe — Scheduled Task state. - Linux: `autostart` is still a stub upstream; check reports `skip` with the actionable message from the driver. - Drivers older than 0.19.2: `skip` (subcommand unavailable), doctor still passes on everything else. ## Known gaps - The check trusts cua-driver's CLI output. On platforms/drivers where the subcommand is missing it degrades to `skip`, which is correct but means a stale-registration failure on an old driver goes unreported. Once the upstream macOS/Linux autostart lands (trycua/cua#2990), this becomes a full pass/fail check everywhere.

x7peeps requested a review from f-trycua as a code owner August 8, 2026 00:11

This was referenced Aug 8, 2026

feat(cua-driver): add macOS signed-app daemon autostart #2196

Open

feat(computer-use): add daemon autostart check to doctor + macOS troubleshooting NousResearch/hermes-agent#81845

Open

injaneity added the macOS label Aug 26, 2026

Navigation Menu

Uh oh!

Uh oh!

feat(cua-driver): implement macOS LaunchAgent autostart - #2990

feat(cua-driver): implement macOS LaunchAgent autostart#2990
x7peeps wants to merge 1 commit into
trycua:mainfrom
x7peeps:feat/macos-autostart-launchagent

x7peeps commented Aug 8, 2026

Uh oh!

x7peeps commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

x7peeps commented Aug 8, 2026

feat(cua-driver): macOS LaunchAgent autostart (enable/disable/status/kick)

Story: from a broken local session to a working autostart

1. The local session that would not run

2. Discovering the project while debugging

3. Research + capability build-out (what we actually ran)

4. Landing it: this PR

Why this design

Verification (run on this machine, macOS 26.5.2)

Known gaps

Who should enable this

Quick-start

Troubleshooting

Platform compatibility

Uh oh!

x7peeps commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants