Skip to content

Cloud: resume paused machines on attach, add Pause When Idle (app) - #23713

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
cloud-idle-auto-pause
Oct 22, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
cloud-idle-auto-pause

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Pilot blocker 1a, Mac app half: attaching to a Cloud machine the idle auto-pause parked resumes it, a paused machine never reads as Failed, and Pause When Idle… sets each machine's timeout.

This is the second of three PRs. The server (the idle detector, the cron and PUT /api/vm/:id/idle-pause) is #23834: merge #23834 first, since this PR's setting calls its route. Until #23834 is deployed, the setting answers with the server's error and everything else here works against today's server (resume uses the existing Resume Machine API). The "Paused (idle)" labels and the Resume button title are #23836, which is stacked on this PR.

What changes

  • Attach resumes. Each of these resumes a paused machine through CloudService.resumeMachine, then connects: open machine; new terminal (for the CLI, the answer is pending if the resume outlasts 30 s); ssh; new Cloud workspace; moving a chat to the machine; a terminal's Connect; Retry. Two attaches during one resume start one resume.
  • Paused is not Failed. A paused machine's stage is paused, including when its app link ended because the machine paused. It reads "Paused" in the progress view, whose Retry button resumes it; the sidebar header keeps its link state instead of "Failed to start". A failed resume says why.
  • Pause When Idle… is in the machine's menu, with CLI parity: cmux cloud set-idle-pause --target machine:<id> --minutes off|15|30|60|120. A non-person caller gets {machine_id, idle_pause_minutes} back, following cmux NEXT: Cloud machine actions from the CLI need --target and return their result #23155. cloud status reports pause_reason and idle_pause_minutes. The default (30 minutes, on) is the server's.
  • A stale record is read again. The app reads the machine list on launch and activation only, so a machine paused while cmux stayed in front can still read running here. A legacy machine with no daemon connection reads its record again before an attach (open, ssh, new terminal, moving a chat) and before Retry decides, and the list is also read after the Mac wakes and after a link start fails (throttled to 30 s).
  • The machine record decodes pauseReason and idlePauseMinutes from /api/vm and from the cmux-next backend's record; an older server, or an unknown value, decodes as absent.
  • plans/cmux-next/action-surfaces.json and actions.md are regenerated for the new action (fleet steps ca18e44f6afe4541045e6c31 and 4556682f68f808ea7aa08207, where ActionSurfaceParityTests passed).
  • New strings are localized in all 21 catalog locales. Catalog changes are additions only; existing keys keep their base order.

Tests

CloudIdlePauseTests: the record decodes pauseReason/idlePauseMinutes (and tolerates an older server); a paused machine's stage is paused, not failed; an ended app link on a paused machine is paused, not failed, while a live machine's ended link still fails; the sidebar header of a paused machine is not Failed; the progress view shows its buttons; attach resumes, once; attaching to a live machine resumes nothing; a failed resume says why; a failed resume does not outlive the next pause; an attach to a stale running record reads it again and resumes (through CloudHandlers.attached); an attach to a paused machine waits for its resume and fails with its reason; the cloud set-idle-pause descriptor has exactly the five choices; CLI, MCP and script runs get the stored value back; invalid choices are refused.

Red proof: fleet step 7b72e086d58ff3002a4c1ad6 at the test-only scratch commit 1a4e38ce939ca804a713a04820a1f2f35d124d67 ran the original set (all three PRs' tests) red before the fix. The ended-app-link, stale-record and stale-failure cases were added after that run, from review.

Observed / Not run

Not run locally: no Swift compile or tests on this Mac (agents don't build here). scripts/cmux-next/prepush-gate.py passed. CI runs the Mac compile and tests.

No capture: the visible change is a paused machine reading "Paused" instead of "Failed" and one menu item; the tests above cover both. No real Cloud VM was created.

Changelog

Added: opening, attaching to or running a command on a paused Cloud machine resumes it; Pause When Idle… sets a machine's idle timeout.

🤖 Generated with Claude Code
Agent session: cosmic-gray

@teamleaderleo teamleaderleo added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 21, 2026
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review d6ba959: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr23713 -f ref=cloud-idle-auto-pause -f platform=macos -f backend_mode=local

Passes: CI passes on d6ba9597d2.

CI passes on d6ba9597d2 (run 39711442902 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

red: cmux-next checks (god files, concurrency, crash safety, l10n); base red: the same failure on the base ab0ad6a; does not block

CI fast guards failed on 1c225a2603 (https://github.com/manaflow-ai/cmux/actions/runs/39685392153). It does not block the merge; a red guard merged into main breaks it for every open PR.
The log named no failed step; see the run.

Agents: python3 scripts/ci/guard_attribution.py fix applies the mechanical fixes locally.

Tour of ce72f7d1c3 (run)

The path-selected recorder failed for core-after, core-before; see the run artifact and failed manifests.

Path-selected interactions

Interaction Changed paths
core Packages/macOS/CmuxNext/Sources/CmuxNextActions/ActionCatalog.xcstrings
Packages/macOS/CmuxNext/Sources/CmuxNextActions/ActionSurfaceCatalog\+Exemptions.swift
Packages/macOS/CmuxNext/Sources/CmuxNextActions/ActionSurfaceCatalog\+MenusContent.swift
Packages/macOS/CmuxNext/Sources/CmuxNextActions/Catalog/CloudActionCatalog.swift
Packages/macOS/CmuxNext/Sources/CmuxNextActions/CatalogArgument.swift
Packages/macOS/CmuxNext/Sources/CmuxNextActions/Localizable.xcstrings
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudAppOps.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudMachineProgressView.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudMachineSession.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudMachineStage.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudMove.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudService.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudStrings\+CLI.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Cloud/CloudStrings\+Progress.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Control/AppControl\+Cloud.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Handlers/CloudHandlers\+Account.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Handlers/CloudHandlers\+Machine.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Handlers/CloudHandlers.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Remote/RemoteTerminalService.swift
Packages/macOS/CmuxNext/Sources/CmuxNextApp/Resources/Cloud.xcstrings
Packages/macOS/CmuxNext/Sources/CmuxNextCloud/API/CloudAPIClient.swift
Packages/macOS/CmuxNext/Sources/CmuxNextCloud/API/CloudMachine.swift
Packages/macOS/CmuxNext/Tests/CmuxNextAppTests/CloudIdlePauseTests.swift

Core UI (before): FAILED — 16 ok, 3 failed, 1 unavailable

tour

Video (mp4)

# Step Result Command
1 Launch ok
2 New workspace ok cmux workspace new --name Tour
3 Type in the terminal ok cmux workspace name:Tour screen current pane current tab current terminal current write --text "echo hello from the cmux-next tour\r"
4 A command reports progress: tab loading spinner ok cmux workspace name:Tour screen current pane current tab current terminal current write --text "clear; printf '\\033]9;4;3\\007'; sleep 900\r"
5 Split right failed: exited 1 cmux action run splitRight
6 Split down failed: exited 1 cmux action run splitDown
7 New terminal tab failed: exited 1 cmux action run newSurface
8 Browser tab ok cmux tab new-browser --url https://example.com
9 Hide the sidebar ok cmux action run toggleSidebar
10 Show the sidebar ok cmux action run toggleSidebar
11 Notification ok cmux notify --title Tour --body "Hello from CI"
12 Notifications panel ok cmux action run showNotifications
13 Agent chat ok cmux agent new-chat
14 Agent workspace ok cmux workspace new --name Agent
15 Agent starts ok cmux workspace name:Agent screen current pane current tab current terminal current write --text "clear; printf '\\033]7501;state=working\\033\\\\'; sleep 8; printf '\\033]7501;state=done\\033\\\\'; sleep 900\r"
16 An agent works: row working spinner ok cmux workspace new --name Notes
17 The agent finishes: done mark in the spinner's place ok
18 Session tree: keycaps on the first rows ok cmux settings set sidebar.sessionTree true
19 Session tree back off for later tours ok cmux settings set sidebar.sessionTree false
20 Profile menu unavailable: exited 1 (the menu shot needs a screen capture route) cmux action run sidebar.profileMenu

Failed steps:

5. Split right
Split right

6. Split down
Split down

7. New terminal tab
New terminal tab

Screenshots (12)

1. Launch
Launch

2. New workspace
New workspace

3. Type in the terminal
Type in the terminal

4. A command reports progress: tab loading spinner
A command reports progress: tab loading spinner

8. Browser tab
Browser tab

9. Hide the sidebar
Hide the sidebar

11. Notification
Notification

12. Notifications panel
Notifications panel

13. Agent chat
Agent chat

16. An agent works: row working spinner
An agent works: row working spinner

17. The agent finishes: done mark in the spinner's place
The agent finishes: done mark in the spinner's place

18. Session tree: keycaps on the first rows
Session tree: keycaps on the first rows

### Core UI \(after\): FAILED — 16 ok, 3 failed, 1 unavailable

tour

Video (mp4)

# Step Result Command
1 Launch ok
2 New workspace ok cmux workspace new --name Tour
3 Type in the terminal ok cmux workspace name:Tour screen current pane current tab current terminal current write --text "echo hello from the cmux-next tour\r"
4 A command reports progress: tab loading spinner ok cmux workspace name:Tour screen current pane current tab current terminal current write --text "clear; printf '\\033]9;4;3\\007'; sleep 900\r"
5 Split right failed: exited 1 cmux action run splitRight
6 Split down failed: exited 1 cmux action run splitDown
7 New terminal tab failed: exited 1 cmux action run newSurface
8 Browser tab ok cmux tab new-browser --url https://example.com
9 Hide the sidebar ok cmux action run toggleSidebar
10 Show the sidebar ok cmux action run toggleSidebar
11 Notification ok cmux notify --title Tour --body "Hello from CI"
12 Notifications panel ok cmux action run showNotifications
13 Agent chat ok cmux agent new-chat
14 Agent workspace ok cmux workspace new --name Agent
15 Agent starts ok cmux workspace name:Agent screen current pane current tab current terminal current write --text "clear; printf '\\033]7501;state=working\\033\\\\'; sleep 8; printf '\\033]7501;state=done\\033\\\\'; sleep 900\r"
16 An agent works: row working spinner ok cmux workspace new --name Notes
17 The agent finishes: done mark in the spinner's place ok
18 Session tree: keycaps on the first rows ok cmux settings set sidebar.sessionTree true
19 Session tree back off for later tours ok cmux settings set sidebar.sessionTree false
20 Profile menu unavailable: exited 1 (the menu shot needs a screen capture route) cmux action run sidebar.profileMenu

Failed steps:

5. Split right
Split right

6. Split down
Split down

7. New terminal tab
New terminal tab

Screenshots (12)

1. Launch
Launch

2. New workspace
New workspace

3. Type in the terminal
Type in the terminal

4. A command reports progress: tab loading spinner
A command reports progress: tab loading spinner

8. Browser tab
Browser tab

9. Hide the sidebar
Hide the sidebar

11. Notification
Notification

12. Notifications panel
Notifications panel

13. Agent chat
Agent chat

16. An agent works: row working spinner
An agent works: row working spinner

17. The agent finishes: done mark in the spinner's place
The agent finishes: done mark in the spinner's place

18. Session tree: keycaps on the first rows
Session tree: keycaps on the first rows

Tours are data in scripts/cmux-next/media/tours: add a step or a tour file to show your feature.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by cloud-team-ready on cosmic-gray · session

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Found 12 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp566ok04i/summary-fail.json (live):/summary-fail.json (live): View Logs
/tmp/tmp5720wl74/summary-missing.json (live):/summary-missing.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
convention violations found (ERROR lines above)./
convention violations found (ERROR lines above).
View Logs
nightly cmux-tui resolve/nightly cmux-tui resolve View Logs
test_actual_workflow_commands_leave_failed_build_receipts_in_upload_directory (main
.SwiftPMBuildMetricsTests.test_actual_workflow_commands_leave_failed_build_receipts_in_
upload_directory)/
test_actual_workflow_commands_leave_failed_build_receipts_in_upload_directory (main
.SwiftPMBuildMetricsTests.test_actual_workflow_commands_leave_failed_build_receipts_in_
upload_directory)
View Logs
test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse
dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo
dules)/
test_every_submodule_free_checkout_on_an_owned_runner_resets_submodules (main.Reuse
dWorkspaceSubmodules.test_every_submodule_free_checkout_on_an_owned_runner_resets_submo
dules)
View Logs

...and 2 more test failures. View all on Blacksmith: run 39717782191, run 39717782302

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-jxe3r created for this PR by the cmux coordinator.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

cmux-next guards / nightly-guards / workflow-guard-tests / release-notary is red on 13 of 19 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

The app side of the idle auto-pause (the server side is its own PR).
Attaching to a paused machine resumes it first through Resume Machine's
path: open machine, new terminal (CLI answers pending past 30 s), ssh,
new Cloud workspace, moving a chat, a terminal's Connect and Retry. A
paused machine's stage is paused, never failed, including after its
app link ended because the machine paused; a failed resume says why.

Pause When Idle... sets the machine's timeout (Off, 15, 30, 60 or 120
minutes; the server default is 30), with CLI parity:
cmux cloud set-idle-pause --target machine:<id> --minutes off|15|30|60|120.
The machine record decodes pauseReason and idlePauseMinutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
@teamleaderleo teamleaderleo changed the title Cloud: pause idle machines after 30 minutes, resume on attach Cloud: resume paused machines on attach, add Pause When Idle (app) Oct 21, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

cmux-next guards / cmux-next localization debt ratchet is red on 16 of 16 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

…sume does not outlive the next pause

- The app reads the machine list on launch and activation only, so a
  machine the idle auto-pause parked while cmux stayed in front still
  read running: the header showed Failed, Retry only woke the daemon, and
  ssh and new terminal refused "not connected". A legacy machine with no
  daemon connection now reads its record again before an attach (open,
  ssh, new terminal, moving a chat) and before Retry decides; the list is
  also read after the Mac wakes and after a legacy link start fails
  (throttled to 30 s).
- resumeFailure clears on a real connect and on every suspend, so a
  failed resume no longer turns a later pause into Failed.
- A legacy link start that a pause overtook changes nothing (noteLink
  checks suspends), so a late socket failure cannot turn a paused machine
  into Failed or Starting.
- Moving a chat stops waiting at once when the resume failed, with its
  reason, instead of waiting out 180 s.
- The cmux-next backend's record maps pauseReason and idlePauseMinutes.
- New terminal's pending answer says resuming only for a paused machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

cmux-next crash-class rules / cmux-next crash-class rules is red on 11 of 11 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

scripts/cmux-next/regenerate-action-exports.sh on the fleet at f67effc
(steps ca18e44f6afe4541045e6c31 and 4556682f68f808ea7aa08207, aws-m4pro;
ActionSurfaceParityTests passed): the new cloudSetIdlePause action in
action-surfaces.json and the counts and Connection folder in actions.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
Keeps base key order in ActionCatalog.xcstrings and appends only
action.cloudSetIdlePause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
Regenerated on the fleet (steps of regenerate-action-exports.sh at
dd18971); differs from base only by cloudSetIdlePause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
@cursor

cursor Bot commented Oct 22, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

CloudMachine.swift: keep both the idle-pause fields (pauseReason,
idlePauseMinutes) and the cost-line fields from #23691.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d3e-acc6-7a03-a574-453e3de27340&host=cosmic-gray
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at ce72f7d

An independent reviewer read the diff at d6ba959. A second independent review covered the base merge to ce72f7d (base ee1c31a). The one conflict, in CloudMachine.swift, keeps both this PR's pause fields and #23691's cost fields. The auto-merged files show no semantic breakage, and the diff against base is still exactly this feature (26 files) with no stray reverts.

What the review confirmed:

  • A paused machine is never shown as Failed, and link failures from before a pause are ignored.
  • Attaching re-reads a stale running record, then resumes the machine and waits for it.
  • resumeFailure is cleared on connect, on attach and on the next pause.
  • CloudMachine(next:) maps pauseReason and idlePauseMinutes.
  • The catalogs only add keys (all 21 locales, base order kept).
  • The generated exports differ from base only by cloudSetIdlePause. They were regenerated on the fleet; base hasn't changed its exports since.

It is safe to land without #23836. Two of the minor notes (the old paused text and the spinner on a paused creation row) are what #23836 fixes.

cmux-ci compile-check --app passed at d6ba959 (job 2270c0dcebe9924e941cea6b) and at ce72f7d (job 327916e27ebc3826be65d1ea).

The reds are all in files this PR doesn't touch, and all are red on base: the convention check on CmuxNextDesign/AttentionHighlightLook.swift:70, the workflow-structure test (this PR changes no workflow files), the crash-class rule on CEFFileDragRelay.swift:162, and release-notary (restored-terminals-come-back.md). At ce72f7d there are also installation rate-limit 403s (intent review, merge tree, review line, fast-gates status post) and release rails and web bundles reds; this PR changes no web/ or rails files. The rest are the wrappers that roll these up.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

This branch had an error being deployed

2 failed and 2 inactive deployments
sentry-dsyms — ce72f7d1 Deployed Oct 22, 2026 by teamleaderleo
artifacts — ce72f7d1 Deployed Oct 22, 2026 by teamleaderleo
cmux-tui-merge-tree — ce72f7d1 Deployed Oct 22, 2026 by teamleaderleo
ci-routing-agent — ce72f7d1 Deployed Oct 22, 2026 by teamleaderleo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant