Skip to content

fix(computer-use): align onboarding visual grid - #11820

Open
austinywang wants to merge 4 commits into
mainfrom
issue-11802-computer-use-onboarding-optical-grid
Open

austinywang wants to merge 4 commits into
mainfrom
issue-11802-computer-use-onboarding-optical-grid

Conversation

@austinywang

@austinywang austinywang commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Closes #11802

Context

The Computer Use onboarding artwork, permission cards, compact companion, and completion state had several independently tuned insets, corner radii, type sizes, and spacings. The asymmetric blue cursor also appeared off-center because it was positioned from its ink centroid rather than its visible bounding box.

Implementation

  • Added Sources/App/ComputerUseOnboardingVisualTokens.swift as the shared measured baseline for the expanded overview, permission cards, completion state, compact companion, typography, rounded-tile ratio, and brand colors.
  • Centered the cursor from the transformed path bounding box on the 1,024 × 1,024 icon canvas (251.0085, 251.0085 translation), then regenerated Resources/ComputerUseHelperIcon.icns and synchronized the SVG artwork.
  • Applied the same tokens to the 600 × 440 overview (64-point hero, 72-point permission cards), the 472 × 112 companion, and completion artwork/spacing while preserving existing callbacks, accessibility labels/hints, appearance handling, localization keys, and Dynamic Type safeguards.
  • Extracted the companion view/layout into dedicated files so ComputerUseOnboardingView.swift remains 764 lines; the extracted files are 216, 27, and 280 lines.

Validation

  • git diff --check origin/main...HEAD — pass.
  • python3 scripts/normalize-pbxproj.py --check cmux.xcodeproj/project.pbxproj — pass.
  • ./scripts/check-pbxproj.sh — pass.
  • ./scripts/lint-pbxproj-test-wiring.sh — pass (759 test files checked).
  • swiftc -parse for all changed onboarding/icon sources and cmuxTests/ComputerUseUXTests.swift — pass.
  • Added Swift Testing coverage: helperIconCursorBoundingBoxStaysCenteredInItsTile (light/dark), computerUseOnboardingUsesOneMeasuredVisualGrid, and computerUseHelperIconCentersTheMeasuredCursorBounds; the test source parses cleanly, but the app test target was not built or run under the constraint below.
  • Standalone optical-token and icon-renderer harnesses — pass; the raster blue-cursor bounds moved from center 554 to 511.5 on the 1,024-point canvas.
  • Before/after raster evidence: /tmp/cmux-onboarding-analysis/before-after.png (with /tmp/cmux-onboarding-analysis/current-icon.png and after-icon.png).
  • Localization audit — existing onboarding keys retained; all 22 touched keys have complete English/Japanese catalog units.

Limitations

No local app build, launch, screenshot/video session, or UI dogfood was run, per the no-build/launch constraint. Live light/dark, localized, and Dynamic Type layouts therefore remain unexercised here; validation is limited to parsing, project wiring, geometry/icon harnesses, and static raster evidence. The repository's scripts/swift_file_length_budget.py is not present in this checkout, so that script could not be run.

Summary by CodeRabbit

  • New Features

    • Added a companion window that guides users through Computer Use permission setup, including instructions, navigation, and a draggable helper app tile.
    • Improved consistency across onboarding screens with shared visual sizing, spacing, colors, and corner treatments.
  • Bug Fixes

    • Refined the helper icon’s cursor positioning and geometry for more accurate visual centering.
    • Improved alignment between the onboarding window and permission companion layout.
  • Tests

    • Added regression coverage for onboarding layout consistency and helper icon positioning.

@cursor

cursor Bot commented Sep 3, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@vercel

vercel Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cmux166 Canceled Canceled Sep 3, 2026 6:07pm UTC
cmux41 Canceled Canceled Sep 3, 2026 6:07pm UTC

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: cf3da19c-580e-4398-9efa-e4a1d0d2e621

📥 Commits

Reviewing files that changed from the base of the PR and between 4e30be8 and 193618a.

⛔ Files ignored due to path filters (1)
  • Resources/ComputerUseHelperIcon.svg is excluded by !**/*.svg
📒 Files selected for processing (10)
  • Resources/ComputerUseHelperIcon.icns
  • Sources/App/AgentCursorPointerView.swift
  • Sources/App/ComputerUseOnboardingView.swift
  • Sources/App/ComputerUseOnboardingVisualTokens.swift
  • Sources/App/ComputerUseOnboardingWindowController.swift
  • Sources/App/ComputerUsePermissionCompanionLayout.swift
  • Sources/App/ComputerUsePermissionCompanionView.swift
  • cmux.xcodeproj/project.pbxproj
  • cmuxTests/ComputerUseUXTests.swift
  • scripts/generate-computer-use-helper-icon.swift

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

Changes

The change adds shared visual tokens for Computer Use onboarding. Expanded onboarding surfaces and permission companion layouts now use those tokens. The permission companion view moves to a dedicated file. Icon centering and token synchronization receive regression tests.

Computer Use onboarding visuals

Layer / File(s) Summary
Shared visual baseline and icon geometry
Sources/App/ComputerUseOnboardingVisualTokens.swift, scripts/generate-computer-use-helper-icon.swift, Sources/App/AgentCursorPointerView.swift, cmuxTests/ComputerUseUXTests.swift
Adds shared geometry, typography, color, corner-radius, and cursor-centering tokens. Updates helper-icon generation and rendering to use the measured values.
Expanded onboarding token integration
Sources/App/ComputerUseOnboardingView.swift
Replaces expanded onboarding layout, typography, colors, artwork, permission-card, and action constants with shared tokens.
Permission companion view and wiring
Sources/App/ComputerUsePermissionCompanionLayout.swift, Sources/App/ComputerUsePermissionCompanionView.swift, Sources/App/ComputerUseOnboardingWindowController.swift, cmux.xcodeproj/project.pbxproj, cmuxTests/ComputerUseUXTests.swift
Adds the dedicated permission companion layout and view, wires token-based window sizing, registers the new files, and validates companion metrics.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 19361

This change centralizes Computer Use onboarding visual measurements and applies them across expanded and companion layouts, including synchronized helper-icon geometry. No current merge-blocking behavior or production risk has been identified.

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 8 files. (1 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the primary change: aligning the Computer Use onboarding visual grid.
Description check ✅ Passed The description provides detailed context, implementation, validation results, and limitations. It does not use all template headings, include a demo video, or complete the checklist, but it remains s…
Linked Issues check ✅ Passed The changes satisfy the coding objectives in issue [#11802]. They add documented visual tokens, apply them across onboarding states, center the cursor by measured bounds, preserve existing UI behavior…
Out of Scope Changes check ✅ Passed The changes remain scoped to Computer Use onboarding visuals, related icon generation, project wiring, and regression tests. No unrelated app functionality or chrome changes are evident.
Cmux Swift Actor Isolation ✅ Passed No actor-isolation failure was introduced. The new token type is an immutable top-level Sendable value model. The companion layout is also immutable geometry. The new companion view, icon renderer, …
Cmux Swift Blocking Runtime ✅ Passed PASS: The pull-request diff adds no blocking or timing-based synchronization to production Swift. The changed production files contain no new semaphores, waits, sleeps, delayed dispatch, polling, main…
Cmux Browser Automation Off-Main ✅ Passed PASS: The PR changes only Computer Use onboarding visuals, icon assets, project wiring, and related tests. The scoped files Sources/TerminalController.swift and `Packages/macOS/CmuxControlSocket/...…
Cmux Expensive Synchronous Load ✅ Passed PASS. The production diff adds visual-token constants, SwiftUI layout changes, companion-view extraction, and icon rendering changes. A focused audit of all added production Swift lines found no `Rest…
Cmux Cache Substitution Correctness ✅ Passed PASS: The diff does not replace an authoritative persistence, history, undo, or snapshot read with a cache. The changed production code only centralizes onboarding geometry and icon-rendering tokens. …
Cmux No Hacky Sleeps ✅ Passed PASS: The PR changes Swift sources, Swift test code, image resources, and the Xcode project file. It introduces no TypeScript, JavaScript, shell, or non-Swift build/runtime script changes. The only ti…
Cmux Algorithmic Complexity ✅ Passed No algorithmic-complexity failure was introduced. The production changes add token lookups and constant-time geometry calculations, with no scalable collection scans, sorting, or nested loops. The exi…
Cmux Swift Concurrency ✅ Passed PASS. The diff adds no background Dispatch queues, DispatchGroup, Combine state, completion-handler API, or untracked fire-and-forget Task. Existing Task, ObservableObject, @Published, and completion-…
Cmux Swift @Concurrent ✅ Passed PASS. The diff adds no @concurrent, nonisolated async, or asynchronous helper. The new companion view is a synchronous @MainActor SwiftUI view, and its @MainActor callbacks are UI-bound. The e…
Cmux Swift Package Boundaries ✅ Passed PASS. The production diff is Computer Use onboarding UI and AppKit composition. The new ComputerUsePermissionCompanionView imports AppKit and SwiftUI, the icon renderer uses AppKit, and the wind…
Full details: Description check

Explanation

The description provides detailed context, implementation, validation results, and limitations. It does not use all template headings, include a demo video, or complete the checklist, but it remains substantially complete and relevant.

Full details: Linked Issues check

Explanation

The changes satisfy the coding objectives in issue [#11802]. They add documented visual tokens, apply them across onboarding states, center the cursor by measured bounds, preserve existing UI behavior, and add regression tests. Full live UI verification was not performed, but manual testing requirements are outside this coding assessment.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 8 files. (1 skipped: 1 unsupported.)

Full details: Cmux Swift Actor Isolation

Explanation

No actor-isolation failure was introduced. The new token type is an immutable top-level Sendable value model. The companion layout is also immutable geometry. The new companion view, icon renderer, and onboarding controller are explicitly @MainActor UI types, which the rule allows. The diff adds no service protocols, shared mutable Sendable reference types, or background access to UI-bound stores. The app project has no SWIFT_DEFAULT_ACTOR_ISOLATION setting that would implicitly isolate the new value types.

Full details: Cmux Swift Blocking Runtime

Explanation

PASS: The pull-request diff adds no blocking or timing-based synchronization to production Swift. The changed production files contain no new semaphores, waits, sleeps, delayed dispatch, polling, main-queue sync, or manual locks. The three ContinuousClock.sleep calls in ComputerUseOnboardingWindowController.swift exist in both origin/main and the pull-request revision without semantic changes. Any sleep usage added in cmuxTests/ComputerUseUXTests.swift is deterministic test scaffolding, which the rule allows.

Full details: Cmux Browser Automation Off-Main

Explanation

PASS: The PR changes only Computer Use onboarding visuals, icon assets, project wiring, and related tests. The scoped files Sources/TerminalController.swift and Packages/macOS/CmuxControlSocket/.../ControlCommandExecutionPolicy.swift have no PR diff, and the patch adds no browser.* command, WebKit wait, socket-worker routing, or policy test change. Existing browser automation routing is therefore not introduced or worsened.

Full details: Cmux Expensive Synchronous Load

Explanation

PASS. The production diff adds visual-token constants, SwiftUI layout changes, companion-view extraction, and icon rendering changes. A focused audit of all added production Swift lines found no RestorableAgentSessionIndex.load(), SharedLiveAgentIndex load, transcript/trajectory/workstream store access, directory scan, per-record syscall, or large JSON/file read. The @MainActor code is UI code and only reads in-memory tokens or renders cached artwork. Existing expensive loaders are unchanged and not worsened by this pull request.

Full details: Cmux Cache Substitution Correctness

Explanation

PASS: The diff does not replace an authoritative persistence, history, undo, or snapshot read with a cache. The changed production code only centralizes onboarding geometry and icon-rendering tokens. The existing in-memory cachedImages remains unchanged and is used only for transient UI artwork. The existing UserDefaults and permission-snapshot logic in ComputerUseOnboardingWindowController is unchanged.

Full details: Cmux No Hacky Sleeps

Explanation

PASS: The PR changes Swift sources, Swift test code, image resources, and the Xcode project file. It introduces no TypeScript, JavaScript, shell, or non-Swift build/runtime script changes. The only timing values found are existing Swift onboarding values, and the diff adds no sleep, timer, polling, fixed-delay, or retry synchronization code covered by this check.

Full details: Cmux Algorithmic Complexity

Explanation

No algorithmic-complexity failure was introduced. The production changes add token lookups and constant-time geometry calculations, with no scalable collection scans, sorting, or nested loops. The existing controller scans and polling loop are unchanged. The generator loop processes the explicit five-size list, which is a tiny fixed-size collection. Pixel loops are test-only and exempt.

Full details: Cmux Swift Concurrency

Explanation

PASS. The diff adds no background Dispatch queues, DispatchGroup, Combine state, completion-handler API, or untracked fire-and-forget Task. Existing Task, ObservableObject, @Published, and completion-handler counts remain unchanged in the affected app files. The extracted companion view only preserves existing SwiftUI callback boundaries and observed state. The new visual-token and layout files contain no asynchronous work.

Full details: Cmux Swift `@Concurrent`

Explanation

PASS. The diff adds no @concurrent, nonisolated async, or asynchronous helper. The new companion view is a synchronous @MainActor SwiftUI view, and its @MainActor callbacks are UI-bound. The existing onboarding and window-controller async calls remain unchanged apart from line shifts and the companion view extraction. The token and layout files are synchronous. Therefore, the PR introduces no violation of the stated Swift concurrency conditions.

Full details: Cmux Swift Package Boundaries

Explanation

PASS. The production diff is Computer Use onboarding UI and AppKit composition. The new ComputerUsePermissionCompanionView imports AppKit and SwiftUI, the icon renderer uses AppKit, and the window controller handles app-window lifecycle. ComputerUseOnboardingVisualTokens contains onboarding geometry, colors, and small tile-bound calculations, and it imports SwiftUI; it is not domain, provider, persistence, protocol, or cross-surface state logic. ComputerUsePermissionCompanionLayout only aliases companion UI metrics. The changed symbols are used only by the onboarding views, renderer, controller, and tests. These cases match the rule's allowed small UI/AppKit glue and app-lifecycle composition, so the diff does not require a new SwiftPM package target.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-11802-computer-use-onboarding-optical-grid

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@teamleaderleo

Copy link
Copy Markdown
Collaborator

Superseded by #12265, which shipped the Computer Use onboarding and permission companion lifecycle changes covering this optical-grid work.

1 similar comment
@teamleaderleo

Copy link
Copy Markdown
Collaborator

Superseded by #12265, which shipped the Computer Use onboarding and permission companion lifecycle changes covering this optical-grid work.

@teamleaderleo

Copy link
Copy Markdown
Collaborator

Current main has since added an opt-in skill-link reconciliation path, so this branch now conflicts in the wrapper, docs, metadata, and tests. The duplicate remains worth tracking, but it needs a focused refresh against that policy before landing.

@teamleaderleo teamleaderleo reopened this Sep 30, 2026
@teamleaderleo teamleaderleo added area: layout Splits, panes, tabs, windows, resizing, full screen review: needs-attention Actionable automated review finding needs an author reply S3: minor Wrong behavior with a workaround labels Sep 30, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator

Reopening this: merged #12265 explicitly leaves #11820 as independent open optical-grid work. It remains live for a focused landing decision.

This branch was successfully deployed

2 active deployments
Preview – cmux41 — 193618a6 Deployed Sep 3, 2026 by vercel[bot]
Preview – cmux166 — 193618a6 Deployed Sep 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: layout Splits, panes, tabs, windows, resizing, full screen review: needs-attention Actionable automated review finding needs an author reply S3: minor Wrong behavior with a workaround

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Computer Use onboarding artwork and permission cards need a shared optical grid

2 participants