feat(desktop-use): APP-052 Desktop Use Settings, CLI, and AppShot capture migration by AruNi-01 · Pull Request #199 · AruNi-01/atmos

AruNi-01 · 2026-08-03T17:41:36Z

Summary

APP-052 Desktop Use specs (BRAINSTORM / PRD / TECH / TEST): local desktop capture + optional control engine; orthogonal to APP-016 atmos computer; no MCP; no public third-party branding.
crates/desktop-use + atmos desktop-use CLI: status, driver ensure/status/stop/uninstall, capture, drive screenshot/click/type with vendor-free user strings.
Settings → Desktop Use: engine lifecycle controls + embedded AppShot permissions panel; permission CTAs open Settings Desktop Use instead of standalone window as primary path.
AppShot capture migration: production appshot/ no longer shells osascript/screencapture; in-process Desktop Use capture under Electron keeps TCC on Atmos Desktop; metadata-only frontmost for animation preflight.

Spec

specs/APP/APP-052_desktop-use/

Test plan

cargo test -p desktop-use (12 tests)
atmos desktop-use --help|status|drive click offline
Bun: settings wiring tests + frontmost pure helpers + desktop-use client path
Static: no production osascript/screencapture under appshot/ (owned by desktop-use/capture.ts)
Manual: Settings → Desktop Use permissions grant on signed Desktop (TCC)
Manual: dual-shift AppShot still writes records

Notes

Control engine auto-download package not published yet: ensure soft-fails as not_installed with clear copy (dev install via local source still supported).
Merge may wait on required reviews / branch protection.

Summary by cubic

Implements APP-052 Desktop Use: local desktop capture and an optional control engine managed via a new Rust crate, CLI, and Settings. AppShot capture now runs in-process through Desktop Use (no osascript/screencapture), keeping macOS TCC on Atmos Desktop; adds Linux-safe cfg/dead_code for non‑macOS builds and TypeScript settings search/keyword maps for typecheck completeness.

New Features
- crates/desktop-use: capture (frontmost metadata + PNG/base64), vendor-free strings, driver lifecycle, and drive actions (screenshot/click/type).
- CLI atmos desktop-use: status, driver ensure|status|stop|uninstall, capture, drive click|type|screenshot (JSON output).
- Settings → Desktop Use: engine status/actions and an embedded AppShot permissions panel; CTAs open this section instead of a standalone window.
- Electron: appshot/frontmost now delegates to desktop-use/capture; new IPC handlers and a CLI-backed client.
- i18n/tests: added en/zh copy; settings wiring and client tests.
Migration
- AppShot uses Desktop Use for capture; existing records/UI stay the same. Verify dual‑shift still writes.
- Dev checks: cargo test -p desktop-use, atmos desktop-use status, atmos desktop-use capture. driver ensure will report not_installed until the engine package is published.
- macOS permissions: open Settings → Desktop Use from the app; legacy permissions window is a fallback only.

^{Written for commit e672ca7. Summary will update on new commits.}

…apture migration Add Atmos-owned Desktop Use (orthogonal to APP-016 Computer): crates/desktop-use lifecycle + vendor-free strings, atmos desktop-use CLI, Settings section with engine ensure/stop and reused AppShot permissions panel, and AppShot capture via in-process Desktop Use surface (TCC stays on Atmos Desktop). No MCP and no public third-party branding.

vercel · 2026-08-03T17:41:43Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
atmos-landing	Ready	Preview	Aug 3, 2026 5:58pm

Address needless_question_mark, derivable_impls, and format nits so Format Check and Clippy jobs pass on the APP-052 PR.

coderabbitai · 2026-08-03T17:42:42Z

Warning

Review limit reached

@AruNi-01, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2655a6be-8114-4b92-8560-4878d71bad60

📥 Commits

Reviewing files that changed from the base of the PR and between bf47ade and e672ca7.

⛔ Files ignored due to path filters (1)

Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (33)

apps/cli/Cargo.toml
apps/cli/src/commands/desktop_use.rs
apps/cli/src/commands/mod.rs
apps/cli/src/main.rs
apps/cli/src/output.rs
apps/desktop-electron/src/appshot/frontmost.ts
apps/desktop-electron/src/desktop-use/capture.ts
apps/desktop-electron/src/desktop-use/client.test.ts
apps/desktop-electron/src/desktop-use/client.ts
apps/desktop-electron/src/ipc/handlers.ts
apps/web/messages/en.json
apps/web/messages/zh.json
apps/web/src/features/appshot/components/AppshotPermissionsPanel.tsx
apps/web/src/features/appshot/components/AppshotPermissionsWindow.tsx
apps/web/src/features/appshot/lib/appshot-client.ts
apps/web/src/features/appshot/lib/open-desktop-use-settings.ts
apps/web/src/features/settings/components/DesktopUseSettingsSection.tsx
apps/web/src/features/settings/components/SettingsModalSections.tsx
apps/web/src/features/settings/components/__tests__/desktop-use-settings-section.test.ts
apps/web/src/features/settings/components/settings-modal-data.ts
apps/web/src/shared/lib/nuqs/searchParams.ts
crates/desktop-use/Cargo.toml
crates/desktop-use/src/capture.rs
crates/desktop-use/src/config.rs
crates/desktop-use/src/control.rs
crates/desktop-use/src/lib.rs
crates/desktop-use/src/manager.rs
crates/desktop-use/src/strings.rs
specs/APP/APP-052_desktop-use/BRAINSTORM.md
specs/APP/APP-052_desktop-use/PRD.md
specs/APP/APP-052_desktop-use/TECH.md
specs/APP/APP-052_desktop-use/TEST.md
specs/SPEC-INVENTORY.md

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/app-052-desktop-use

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Gate macOS-only Command/base64 imports behind cfg so Linux Clippy builds, and apply workspace rustfmt for Format Check.

github-actions · 2026-08-03T17:54:36Z

E2E report (expired)

This report has been superseded by a newer CI - E2E run.

Field	Value
Superseded by run	`30838953856`
Previous HTML report	Open report

Allow capture helpers on non-macOS for Clippy, and add desktop-use entries to settings keyword/search maps for TypeScript completeness.

github-actions · 2026-08-03T18:05:06Z

E2E report: ✅ Passed

30 passed · 0 failed · 2 flaky · 0 skipped · 4m 9s · 93.8% pass rate

Run

Field	Value
Workflow run	`30838953856`
Selected suites	`smoke-routes`, `smoke-onboarding`, `smoke-app-shell`, `smoke-project`, `smoke-settings`, `smoke-workspace`, `specs`
HTML report	Open report
Artifacts	`playwright-html-report`, `playwright-json-report` (14d) · videos/traces in artifacts only

Overview

Metric	Count
Passed	30
Failed	0
Flaky	2
Skipped	0
Duration	4m 9s
Pass rate	93.8%

By file

Name	Passed	Flaky
`APP-043_workspace-surface-cache.e2e.ts`	2	0
`APP-044_project-workspace-groups.e2e.ts`	2	0
`APP-048_049_api-client-ws.e2e.ts`	4	2
`app-shell-navigation.e2e.ts`	1	0
`app-shell-preferences-menu.e2e.ts`	1	0
`app-shell-workspace-composer.e2e.ts`	1	0
`app-shell-ws-session.e2e.ts`	1	0
`local-onboarding.e2e.ts`	2	0
`private-relay-onboarding.e2e.ts`	1	0
`project-context-route.e2e.ts`	2	0
`remote-onboarding.e2e.ts`	1	0
`route-boot.e2e.ts`	7	0
`settings-modal.e2e.ts`	3	0
`workspace-sidebar-routes.e2e.ts`	2	0

By project

Name	Passed	Failed	Flaky	Skipped
`chromium`	26	0	1	0
`mobile-chromium`	4	0	1	0

All selected E2E suites passed.

fix(desktop-use): satisfy rustfmt and clippy for CI

008ec9a

Address needless_question_mark, derivable_impls, and format nits so Format Check and Clippy jobs pass on the APP-052 PR.

fix(desktop-use): Linux-safe capture imports and rustfmt for CI

37f33cf

Gate macOS-only Command/base64 imports behind cfg so Linux Clippy builds, and apply workspace rustfmt for Format Check.

fix(desktop-use): Linux dead_code allows and Settings typecheck maps

e672ca7

Allow capture helpers on non-macOS for Clippy, and add desktop-use entries to settings keyword/search maps for TypeScript completeness.

vercel Bot deployed to Preview August 3, 2026 17:58 View deployment

AruNi-01 merged commit 3b55c77 into main Aug 3, 2026
31 of 32 checks passed

This was referenced Aug 4, 2026

feat(desktop-use): Inspect (AX tree) as first-class Desktop Use capability #200

Closed

feat(desktop-use): control engine pin, Atmos Desktop Use host, unified AppShot TCC #202

Merged

Navigation Menu

feat(desktop-use): APP-052 Desktop Use Settings, CLI, and AppShot capture migration - #199

feat(desktop-use): APP-052 Desktop Use Settings, CLI, and AppShot capture migration#199
AruNi-01 merged 4 commits into
mainfrom
feat/app-052-desktop-use

AruNi-01 commented Aug 3, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

vercel Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 3, 2026 •

edited

Loading

Review limit reached

Uh oh!

github-actions Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

AruNi-01 commented Aug 3, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Spec

Test plan

Notes

Summary by cubic

Uh oh!

vercel Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

github-actions Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E report (expired)

Uh oh!

github-actions Bot commented Aug 3, 2026

E2E report: ✅ Passed

Run

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

AruNi-01 commented Aug 3, 2026 •

edited by cubic-dev-ai Bot

Loading

vercel Bot commented Aug 3, 2026 •

edited

Loading

coderabbitai Bot commented Aug 3, 2026 •

edited

Loading

github-actions Bot commented Aug 3, 2026 •

edited

Loading