Skip to content

Use Cua for Local VM and include voice mode - #100

Merged
milind-soni merged 6 commits into
mainfrom
codex/computer-routines
Aug 14, 2026
Merged

Use Cua for Local VM and include voice mode#100
milind-soni merged 6 commits into
mainfrom
codex/computer-routines

Conversation

@milind-soni

Copy link
Copy Markdown
Owner

What changed

  • replace the Local VM's custom desktop proxy with the official Cua Driver/MCP transport
  • pin both the packaged SDK and native executable to Cua 0.19.3
  • build a checksum-pinned Linux desktop image for Intel and Apple Silicon hosts
  • keep the VM isolated, loopback-only, resource-limited, and exclusive to one bot at a time
  • safely recreate legacy desktop containers that cannot resume, while new Cua containers support stop/start
  • include voice settings, call mode, speech helper packaging, narration, and TTS server support from current main
  • preserve the clearer Box credential errors: 401/403 for an invalid credential and 402 for a valid account without a paid plan
  • reconcile current structured computer observations and browser-state support

Verification

  • 271 tests passed, 8 skipped
  • TypeScript checks passed
  • Electron checks passed
  • production client and server builds passed
  • signed speech helper build passed
  • Cua 0.19.3 SDK/executable staging passed
  • live Docker VM: 58 MCP tools, screenshots, input, apps, viewer, stop/start/reconnect
  • full signed macOS app, DMG, and ZIP packaging passed
  • packaged app signature verified and embedded Cua executable reports 0.19.3

Notes

milind-soni and others added 5 commits August 14, 2026 09:10
The cloud box needs a paid plan and "This computer" points the agent at
the desktop the human is using. Local VM is the third option: a Linux
desktop in a container on this machine — free, disposable, and separate
from the user's own desktop and files.

It costs almost no new code because the computer tools were never really
coupled to the cloud. Every tool runs shell and reads a file back, so
only two functions had to learn a second transport: commands go through
the runtime's exec instead of a REST endpoint, and a frame is read
straight out of the container instead of over HTTP — which is also
faster, since there is no network hop at all.

- Runs on: Cloud box · Local VM · This computer · Off
- Both drivers ask one helper what env the proxy needs, so the transport
  decision lives in exactly one place and box credentials can never leak
  into a container turn (there's a test for that)
- Choosing Local VM when it isn't ready explains which step is missing —
  no runtime, runtime not started, image not pulled, container stopped —
  instead of failing inside the agent's turn
- The box-only Computer engine says so plainly rather than throwing

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@milind-soni, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 8 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 93d05ab1-b6d0-4596-b187-07bb47dd6191

📥 Commits

Reviewing files that changed from the base of the PR and between e0f8e86 and 54e2801.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (34)
  • .gitignore
  • dist-server/container-computer.js
  • dist-server/container-mcp.js
  • dist-server/drivers/acp/core.js
  • dist-server/drivers/acp/grok.js
  • dist-server/drivers/antigravity.js
  • dist-server/drivers/boxagent.js
  • dist-server/drivers/claude.js
  • dist-server/drivers/codex.js
  • dist-server/env-path.js
  • dist-server/harness/registry.js
  • dist-server/index.js
  • dist-server/procs.js
  • electron-builder.yml
  • electron/cua.mjs
  • package.json
  • scripts/prepare-cua.mjs
  • scripts/smoke-cua-container.mjs
  • scripts/smoke-cua.mjs
  • server/container-computer.test.ts
  • server/container-computer.ts
  • server/container-mcp.test.ts
  • server/container-mcp.ts
  • server/contracts.ts
  • server/drivers/acp/core.ts
  • server/drivers/boxagent.ts
  • server/drivers/claude.ts
  • server/harness/registry.ts
  • server/index.ts
  • server/store.ts
  • src/components/ComputerPanel.tsx
  • src/components/LocalComputerSection.tsx
  • src/components/SettingsModal.tsx
  • src/state/store.tsx

Comment @coderabbitai help to get the list of available commands.

@milind-soni
milind-soni merged commit 70d18b7 into main Aug 14, 2026
5 checks passed
@milind-soni
milind-soni deleted the codex/computer-routines branch August 14, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant