Skip to content

feat(computer-use): user-facing authorization for cua-driver browser attachment (browser-approve token + bounded manifest mode) - #86342

Merged
teknium1 merged 2 commits into
mainfrom
feat/cua-browser-integration
Aug 15, 2026
Merged

feat(computer-use): user-facing authorization for cua-driver browser attachment (browser-approve token + bounded manifest mode)#86342
teknium1 merged 2 commits into
mainfrom
feat/cua-browser-integration

Conversation

@teknium1

@teknium1 teknium1 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

The typed cua_browser_* route (landed in #74166) can now actually attach to a user's signed-in browser and run repeatable bounded automation: this PR adds the user-facing authorization surface, live-verified against a real cua-driver 0.19.3 binary — a one-time computer_use.grant_existing_profile config opt-in (cua's trusted-launcher --grant existing-profile), a bounded permission mode driven by a reviewed session-policy manifest, and a hermes computer-use browser-approve token passthrough for driver builds where the legacy token path is enabled.

Until now, existing_profile attachment always failed closed in standard mode and the only escape hatch was full session YOLO. This wires up cua-driver's intended non-YOLO authorization paths end to end.

Changes

  • tools/computer_use/cua_backend.py:
    • computer_use.grant_existing_profile: true appends --grant existing-profile to the standard-mode MCP spawn — the supported route for signed-in-browser attachment on current drivers (live-verified: MCP initialize accepts the flag). Default false keeps attachment failing closed. Never applied to bounded/unrestricted daemons.
    • _EmbeddedCuaDaemon supports bounded: private per-session daemon launched with --session-policy <path> --approve-session-policy (live-verified flag names — the docs' --capability-manifest spelling is rejected by the real binary); missing manifest fails loudly at session start.
  • hermes_cli/config_defaults.py: computer_use.permission_mode (standard | bounded), capability_manifest, grant_existing_profile. unrestricted is deliberately NOT a config value — it stays bound to the explicit per-session YOLO toggle; session YOLO still overrides bounded.
  • hermes_cli/main.py: hermes computer-use browser-approve --pid --window-id --profile-mode interactive passthrough. Live-verified: the verb exists, requires --pid, and refuses non-interactive terminals (genuinely un-mintable by a model). Current drivers treat its token as a disabled legacy compatibility path, so it's surfaced as secondary.
  • tools/computer_use/browser_route.py + tool.py + schema.py: approval_token accepted on cua_browser_prepare, forwarded only for existing_profile and only as a non-empty string.
  • Guidance: bundled computer-use skill, system-prompt typed-browser section, and docs describe the verified ladder — config grant → bounded manifest → YOLO — with isolated-profile-first as the default posture and refusals reported to the user by config key.
  • Tests: tests/tools/test_computer_use_browser_authorization.py (26 new) covering the grant flag, token passthrough + non-string rejection, bounded daemon spawn args, no-bypass-env invariant, config resolution incl. unrestricted-in-config rejection, and backend construction.

Validation

Check Result
Live: real cua-driver 0.19.3 binary, browser-approve verb exists; requires --pid; refuses non-TTY
Live: bounded daemon with real session-policy file starts, status reports running; wrong flag names / policy formats rejected loudly
Live: cua-driver mcp --grant existing-profile MCP initialize succeeds with the grant
Live: browser_prepare schema on 0.19.3 approval_token = legacy path; trusted-launcher grant = supported route (drove the design)
Unit: new + sibling suites 68 + 118 passed
E2E: temp HERMES_HOME — bounded config resolution, loud no-manifest failure, real argparse browser-approve path, standard default all passed
ruff on touched files clean

Not live-testable here: the final GUI attach to a running signed-in Chrome (headless box, no display). Flags, daemon startup, schema contract, and grant acceptance are all verified against the real driver; the display-dependent hop needs one run on a desktop box.

Infographic

Browser attachment authorization — approve token, bounded manifest, isolated profile, exact window-to-tab binding

…attachment

Completes the typed cua_browser_* route (PR #74166 lineage) with the
authorization surface that makes existing-profile attachment and
repeatable bounded automation reachable by real users:

- hermes computer-use browser-approve: CLI passthrough that mints
  cua-driver's five-minute single-use attachment token for one exact
  (pid, window_id). The user, never the model, is the token source.
- approval_token passthrough on cua_browser_prepare (schema + dispatch +
  browser_route), forwarded only for existing_profile and only as a
  non-empty string.
- computer_use.permission_mode: bounded + capability_manifest config:
  private per-session embedded daemon launched with
  --capability-manifest/--approve-capability-manifest; missing manifest
  fails loudly. 'unrestricted' is deliberately NOT a config value —
  it stays bound to the explicit per-session YOLO toggle.
- Skill + system-prompt + docs guidance for the three authorization
  rungs and the isolated-profile-first default.

E2E-verified against a temp HERMES_HOME: real config resolution to
bounded, loud failure without a manifest, real argparse path driving a
fake cua-driver binary, standard default preserved.
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@teknium1, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 17 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 42d288c2-ce8e-4694-a999-82d2d73ef046

📥 Commits

Reviewing files that changed from the base of the PR and between 367f0c2 and 54fbe24.

📒 Files selected for processing (11)
  • agent/prompt_builder.py
  • hermes_cli/config_defaults.py
  • hermes_cli/main.py
  • skills/autonomous-ai-agents/computer-use/SKILL.md
  • tests/tools/test_computer_use_browser_authorization.py
  • tests/tools/test_computer_use_cua_0_9.py
  • tools/computer_use/browser_route.py
  • tools/computer_use/cua_backend.py
  • tools/computer_use/schema.py
  • tools/computer_use/tool.py
  • website/docs/user-guide/features/computer-use.md

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on 9e1822c — fix(computer-use): align browser authorization with live-ver

⚠️ Warnings

OSV vulnerability scan · View job

5 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.


debug info

CI timings

CI timings · View report · View job

Wall time 3m32s vs 6m6s (-42.1%). 15 job(s) slower, 9 faster, 1 unchanged.

  • Python tests / Run tests slice 3/12: -29.0s
  • Python tests / Run tests slice 11/12: +29.0s
  • Python tests / Run tests slice 6/12: +22.0s
  • OSV scan / Scan lockfiles / osv-scan: +21.0s
  • Docs Site / docs-site-checks: -16.0s

@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard tool/browser Browser automation (CDP, Playwright) area/config Config system, migrations, profiles sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 14, 2026
…-driver 0.19.3 contract

Live-tested against the real cua-driver 0.19.3 binary (Linux x86_64):

- bounded serve flags corrected: the daemon accepts
  --session-policy/--approve-session-policy, not the docs'
  --capability-manifest names (which it rejects). Verified end-to-end:
  a bounded daemon with a real policy file starts and reports running.
- browser-approve verified real but interactive-only (refuses without a
  TTY) and its token is a legacy compatibility path disabled by default
  on current drivers (per the live browser_prepare schema). Kept as a
  passthrough; no longer presented as the primary route.
- NEW primary standard-mode route, verified live: launch the runtime
  with cua-driver's trusted-launcher grant. config opt-in
  computer_use.grant_existing_profile: true appends
  --grant existing-profile to the standard-mode MCP spawn (MCP
  initialize verified accepting the flag). Default false = attachment
  keeps failing closed. Never applied to bounded/unrestricted daemons.
- Skill, system prompt, tool schema, and docs updated to the verified
  ladder: config grant > bounded manifest > YOLO; token = legacy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data tool/browser Browser automation (CDP, Playwright) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants