Skip to content

cmux NEXT Cloud: test that an ACP chat in a Cloud workspace runs its agent on the VM - #25680

Merged
lawrencecchen merged 35 commits into
feat-cmux-nextfrom
25583-acp-cloud-vm-chief
Nov 1, 2026
Merged

lawrencecchen merged 35 commits into
feat-cmux-nextfrom
25583-acp-cloud-vm-chief

Conversation

@austinywang

@austinywang austinywang commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #25583

Adds CloudAgentChatTests to verify that an ACP chat opened in a Cloud workspace runs its agent on that machine. This is a test-only change: the app connects to two real same-tree cmux-tui daemons, and the machine's acpmux runs fake_agent.py through the real AgentTabRemoteHost and RemoteAcpmuxWire path.

The four tests cover five cases:

  • The first send in a project folder obtains folder trust on the machine, starts and binds the remote session, receives the agent's machine-specific HOME, and reattaches with replay without starting another session.
  • A first send without a folder uses the machine workspace's actual work folder.
  • The creator can start a chat in a team machine’s own work folder without a trust question, and the machine binds that session.
  • A link with no owner answer is refused by the app before any remote start, on both plain and team hosts.

The stand-in machine daemon requires the owner word, so dropping the app's vm_kind: "owner" would fail the successful-start cases. Test setup has cleanup on failure and cancellation-aware waits. The suite holds the shared sidebar-document ownership gate, gives both daemons private homes, and disables login import, live catalog fetches and unrelated background probes so the explicit sample harness controls startup.

Starting VMs from a chat (#25597) and starting Cloud workspaces from Chief (#25596) remain outside this change.

Validation

Head: 43267d8e81d5457fad1d2dc0dd7d3adddc10ac74; merged base: 0ffea12ac93f186eebd694bf9c6cc2f17685fa29. Independent correctness-first review returned SHIP on this exact head after reviewing the executed fleet results. The base merge was conflict-free, and the remote head was checked before each normal push.

Run on cmux-ci as class light:

scripts/ci/package-test-lane.sh suite Packages/macOS/CmuxNext CloudAgentChatTests,CloudLinkReconnectingTests

Observed / Not run

  • Exact-head fleet step abc7496dac7a68fb0c8843d7 on aws-m4pro-8 compiled the package and passed both suites: all five Cloud chat cases (four test functions, including both missing-owner arguments) and all five reconnect tests executed with no skips. Cloud chat took 8.014 seconds; reconnect took 0.624 seconds. Exit 0; terminal receipt confirms workspace_reset true.
  • Red before repair: at 23adfaf7eea4b2671e40c30dd9ba337d749f7c69, step 64a6dadbad022f88b748f474 failed three Cloud tests because the fixture expected starts now allowed by base Cloud: a machine's creator can start chats on it from the app, team machines included #29107 and cmux-tui: a creator's first chat on their own Cloud machine starts in its work folder with no question #29556 to be refused. The repair keeps explicit trust coverage in HOME/project, verifies the creator’s owner grant and automatic HOME/work trust on a team machine, and refuses missing-owner starts on both host kinds. No production behavior or timeout was changed.
  • Manual and installed pre-push gates, namespace lint, scoped Swift syntax, localization defaults and feature flags passed. The supplemental workspace package-groups check finds the base’s missing CmuxDeviceHub entry; this PR changes neither the workspace nor package definitions.
  • GitHub checks on this head are still queued/running. The previous-head release-notary guard failed priority-view-drag.md release-note validation, identical to feat-cmux-next at ac5389c4d9333619d4536d78963b728881de5f78. The digest scripts and highlight are unchanged by this PR. Guard status wraps that failure.
  • PlacementDocumentWriteOrderTests failed its unchanged pre-operation ordering assertion in the earlier three-suite run; that unrelated base suite was not rerun for this repair. Earlier Cloud timeout failures were not reproduced; their cause remains unproved.
  • Not run: a live Cloud VM or the full CmuxNext suite. The machine fixture uses a second real same-tree daemon over a stand-in link. No builds/tests ran locally. The older dogfood build is not evidence for this head.

No capture: test-only change, no UI changes.

Changelog

none


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

austinywang and others added 5 commits October 26, 2026 13:25
…the machine (#25583)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…default folder

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@austinywang austinywang added the risk:low Risk tier (docs/ci/risk-tiers.md) label Oct 26, 2026
@austinywang

Copy link
Copy Markdown
Contributor Author

Correctness review (subagent) of the test, and what I did about each finding

No blocking findings. The JSON-RPC shapes match RemoteAcpmuxWire, and the acp --json ls shape matches acpmux's _acpmux/sessions reply.

  1. Medium: the no-folder test never checked which folder was used. Fixed. It now requires the reported folder to equal the machine workspace's folder and differ from this Mac's home. It also checks the trust answer and has no skippable branch.
  2. Low-medium: an echoed prompt could pass for the agent's reply. Fixed. The prompt is now envs:HOME,ACPMUX_HOME, and the test decodes agent_message_chunk updates and requires the machine's HOME and ACPMUX_HOME. That proves the agent process runs with the machine's environment.
  3. Low: this Mac's daemon used the real acpmux home. Fixed. It now gets its own ACPMUX_HOME under the test root.
  4. Low: cleanup leaks a machine daemon when it never connected. Not fixed. CloudMoveTests.end has the same gap; I left it to a shared fix.
  5. Nit: a call with no answer throws a bare cancellation. Fixed. It now records which method got no answer.

@teamleaderleo

Copy link
Copy Markdown
Collaborator

Bead cx-6bnul created for this PR by the cmux coordinator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@austinywang

Copy link
Copy Markdown
Contributor Author

Receipts at head 6cfe386 (owner austinywang, agent Claude Code; base 941ace3)

  • Tests: CloudAgentChatTests 3/3 passed. Fleet step 1bb8ba12e9176b0c38535834 on aws-m4pro-2, cmux-ci light, Xcode 26.6, same-tree cmux-tui.
  • Earlier head 568e1b0: step cabc91f1548a410de5bd1c64 failed on my stricter assertion. acpmux gives its agents no ACPMUX_HOME, so the test now proves the agent runs on the machine by its HOME alone (6cfe386).
  • Dev build: cmux-ci build job ff1a53348d37283fc29c012c, tag 25583-acp-chat-cloud-vm, published. It was built from 568e1b0; the app is the same at 6cfe386, which changes only the test.
  • Review: the subagent's review is above and its findings are resolved. My own check: the diff is one test file, with no change to app code.

@austinywang austinywang added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 43267d8: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr25680 -f ref=25583-acp-cloud-vm-chief -f platform=macos -f backend_mode=local

Not verified: macOS jobs did not run: compile and app tests were skipped on 43267d8e81; ci-status is green without them.

CI passes on 43267d8e81 (run 41262565421 attempt 1).
macOS jobs did not run: compile and app tests were skipped: Fast static checks skipped (the macos job needs both to succeed). Re-run CI once they pass, before merging.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

web and size checks: green

Shipped in:

@blacksmith-sh

blacksmith-sh Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)/
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…eds (#26323)

Since #26323 the app starts a chat on a Cloud machine only on the Cloud API's
vmKind owner answer of the running link. The stand-in machine now runs a
stand-in link whose attach answer says owner; a new case checks that with no
owner answer no chat starts and the page says why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@austinywang

Copy link
Copy Markdown
Contributor Author

Updated to current feat-cmux-next. Head is now 746552f, pushed without force.

  • The rebase replayed 616 upstream commits under shared refs that other worktrees kept moving, so I merged feat-cmux-next tip 93d1c8c into the branch instead. Git reported no conflicts.
  • On the new base, Cloud link: the attested vmKind decides the VM boundary and app-started chats (owner only) #26323 changed the rule: the app now starts a chat on a Cloud machine only when the running link's Cloud API answer says vmKind: "owner". My stand-in machine ran no link, so the app correctly refused with host_unverified, and 2 of the 3 tests failed (step 434a57cbea60d6b84ff25ec7, head 7c003b3). Fixed in the test:
    • the stand-in machine now runs a stand-in link whose attach answer says owner, the same way CloudLinkReconnectingTests does it;
    • a new case noOwnerAnswerOnTheLinkStartsNoChat checks that with no owner answer no chat starts, and the page says why.
  • Tests: CloudAgentChatTests (4) and CloudLinkReconnectingTests (2) all passed. Fleet step 3f43e83b8354efbd7056e4bc on aws-m4pro-4, at 746552f.
  • Still test-only: no app code changed. The dev build 25583-acp-chat-cloud-vm (job ff1a53348d37283fc29c012c, from 568e1b0) predates the new base. A rebuild would only pick up base changes.

@austinywang austinywang added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Oct 27, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@austinywang

austinywang commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor Author

Base red, not from this PR: nightly-guards / workflow-guard-tests / release-notary and its aggregate Guard status

This PR changes only Packages/macOS/CmuxNext/Tests/CmuxNextAppTests/CloudAgentChatTests.swift. The red is two RunsOnFallbackTests cases about nightly.yml runner labels:

  • test_a_picked_signing_label_has_a_rerun_alternative
  • test_no_runs_on_chain_ends_on_the_reserved_side_label (sign-nightly-identity-legacy)

The same two tests fail on feat-cmux-next itself, with no PR involved:

The fix belongs in the base's nightly.yml runs-on chain, so this PR does not patch around it.

The other old red was cmux-next checks "Default cursor", from LaneNoticeBatch.css:17 on the old base. Merging the latest feat-cmux-next (5587de5) fixed it; check-default-cursor.py now prints ok.

PR: #25680 · owner: austinywang · agent: Claude Code (Opus 5.5) · base: 5587de5 · head: 9a4ff7f

@austinywang

Copy link
Copy Markdown
Contributor Author

Review and status at head 9a4ff7f

Review (Claude subagent, correctness first)

No blocking findings in CloudAgentChatTests.swift. Every API, initializer, trait and helper it uses still exists on this head with matching signatures, and the assertions would fail without the behavior they test. That includes the app's vm_kind: "owner" and the owner-link check.

Medium:

  1. Blocking calls on the main actor, no timeout. World.agentSessions() and the cleanup call run, which uses readDataToEndOfFile() and waitUntilExit() synchronously. The target's default isolation is MainActor. A hung cmux acp ls or daemon shutdown would block the main thread, and .timeLimit cannot interrupt that. Fix: an async helper off the main actor that ends the process after a deadline.
  2. Setup waits for a pane, not for the folder. The no-folder test relies on firstTabFolder being set. Waiting for target()?.firstTabFolder == work.path would close that race.

Minor: tests 2–4 do not assert that this Mac's acpmux stayed idle. agentSessions() passes no HOME while the cleanup does. The header slightly overstates the coverage, since frames skip AgentPaneTransport. A timeout reads as CancellationError.

The fleet test is still red, and the red is this PR's own (not a base red)

head step CloudAgentChatTests
699a3c1 590ac4506eec3c3552918132 (aws-m4pro-6) 4/4 pass; test 1 took 22 s
71e8a84 411470d1a2e191624d761bfc 2/4 fail. The first (untrusted-folder) session/new answers agent_session.timeout instead of agent_session.untrusted_folder
7347436 daed724fa3f5bfcf673fbeb1 (aws-m4pro-6) 2/4 fail. That same session/new gets no answer in 60 s

The two refusal tests pass every time. The tests that wait for the machine daemon's agent-session start fail. The 699a3c1 → 71e8a84 diff is only the .ownsSidebarDocStore trait, so the step from slow (22 s) to timing out looks like startup timing on the worker, not a logic change. That is unproven: neither failing run logged the machine's acpmux daemon.log.

PlacementDocumentWriteOrderTests also failed in daed724f. It is a separate base suite that this PR does not touch.

Not done

  • No exact-head fleet run at 9a4ff7f: the build controller timed out on submission twice tonight.
  • The cmux-next app compile (build fleet) red at this head is no verdict, controller unreachable (cmux-ci exit 69), an infrastructure failure. It needs a re-run once its workflow run finishes.
  • ready-to-land is not set.

The base reds are documented in #25680 (comment).

PR: #25680 · owner: austinywang · agent: Claude Code (Opus 5.5) · base: 5587de5 · head: 9a4ff7f

@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent October 31, 2026 21:44 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent October 31, 2026 21:44 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent October 31, 2026 21:50 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 1, 2026 00:20 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 1, 2026 00:20 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 1, 2026 00:27 Inactive
@austinywang

austinywang commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Review (Codex): SHIP at 90b43ea

Fresh independent correctness-first review found no correctness findings in the full diff against 52e78a0. A follow-up review of the runtime evidence found no concrete PR-caused defect remaining to hold ready-to-land. The PR still adds only CloudAgentChatTests.swift.

All four CloudAgentChatTests passed in 8.291 seconds at pushed ancestor 42a65e5 in fleet step b4fe428f875e5bf448d3f3cb on aws-m4pro-2. Exit 0; workspace_reset true. The reviewer verified the test file is identical to this head. No assertion or production timeout was relaxed. The earlier Cloud timeouts did not reproduce; worker/runtime variability is plausible, but its cause is unproved.

Exact-head runtime execution was blocked by the two NativePopupHost.swift:235/239 sending/data-race compiler errors. Exact base 52e78a0 compile job aaf8bb510022be5b3e6c8344 has identical errors (base run); head compile job 2e94a37b45b099096d826549 and exact-head test-build step a2d15b706efd5292e4a55a9a reproduce them. The ancestor pass is not an exact-head runtime pass.

Other evidence:

  • Current-head cmux-next checks and ci-status pass. Manual and installed pre-push gates found no new violations. Syntax, localization defaults and feature flags pass. The supplemental workspace package-group check finds the base's missing CmuxDeviceHub entry; workspace/package definitions are unchanged by this PR.
  • Head release-notary guard fails release-note validation for priority-view-drag.md and running-tray-alignment.md, identical to the base guard at 9d7cc834f65b102c568718278c3a737e94e64f46. Guard status wraps it. Those highlights are unchanged by this PR.
  • Base and concurrent remote updates were merged without conflicts. Remote head was checked before every normal push. No force push, rebase or reset.

Limits: reviewer ran no builds/tests. No successful current-head runtime run, live Cloud VM or full CmuxNext suite. This is test-only, risk:low, with no visual change. Details and commands are in the description.

@austinywang austinywang added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Nov 1, 2026
@austinywang austinywang added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 1, 2026
@austinywang

austinywang commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor Author

Review (Codex): FIX at 23adfaf

The exact-head fleet run found stale fixture expectations, superseding the earlier source-only SHIP review. Step 64a6dadbad022f88b748f474 ran on aws-m4pro-9: CloudAgentChatTests failed 3/4, CloudLinkReconnectingTests passed 5/5, and PlacementDocumentWriteOrderTests failed its pre-operation ordering assertion.

The Cloud failures match two intentional base changes: #29107 (c8af668) permits the creator's owner grant on a team VM; #29556 (d3cf70b) auto-trusts an owner's exact HOME/work absent an explicit refusal. The test expected both to be refused. The fetched daemon source key matches this exact head, and the environment is retained; this is not a wrong-binary or missing-environment failure.

ready-to-land has been removed. A test-only repair keeps explicit trust coverage in HOME/project, verifies owner-on-team first-send/binding at HOME/work, and checks missing-owner refusal on both plain and team hosts. It will receive exact-head fleet verification and a fresh review after push.

@austinywang austinywang removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 1, 2026
Keep explicit folder-trust coverage in HOME/project, since the base now
auto-trusts an owner's HOME/work (#29556). Verify the creator can start
a chat on a team machine (#29107), including the machine session and
store binding. Refuse missing owner answers on both plain and team hosts.

Reproduced at 23adfaf via cmux-ci
step 64a6dadbad022f88b748f474:
scripts/ci/package-test-lane.sh suite Packages/macOS/CmuxNext CloudAgentChatTests,CloudLinkReconnectingTests,PlacementDocumentWriteOrderTests
Cloud chat failed 3/4 due to these stale expectations; reconnect passed
5/5. Placement's pre-operation ordering assertion also fails independently.

Scoped syntax, localization defaults, feature flags and namespace lint
pass. The workspace package-group finding is unchanged base debt.
Exact-head fleet verification follows the normal push; no local builds.
@austinywang

Copy link
Copy Markdown
Contributor Author

Review (Codex): SHIP at 43267d8

Fresh independent correctness-first review found no correctness findings in the full diff against 0ffea12. The test-only repair matches the current owner/trust rules and preserves the remote execution, VM session/store binding, explicit trust, missing-owner refusal and reconnect/replay oracles.

Exact-head cmux-ci step abc7496dac7a68fb0c8843d7 compiled the package and passed on aws-m4pro-8. All five Cloud chat cases and five reconnect tests executed with no skips. Cloud chat: 8.014 seconds; reconnect: 0.624 seconds. Exit 0; terminal receipt confirms workspace_reset true. The reviewer independently inspected those results. The focused command is in the description.

The previous head’s stale expectations failed after intentional base changes #29107 and #29556. The fixture now uses HOME/project for explicit trust, tests the creator’s owner grant at HOME/work on a team VM, and refuses missing-owner starts on plain and team hosts. No production behavior or timeout was changed.

Static pre-push gates and scoped checks pass except the unchanged base workspace package-group finding. GitHub checks on this head are still queued/running. The prior head’s release-notary failure matches the same guard on feat-cmux-next (links in the description); this PR changes neither those digest scripts nor that highlight.

Limits: no live Cloud VM or full CmuxNext suite; the link is a stand-in with two real same-tree daemons. The unrelated base Placement suite was not rerun. Older Cloud timeouts did not reproduce; their cause remains unproved. No local builds/tests ran.

@austinywang

Copy link
Copy Markdown
Contributor Author

CI classification at 43267d8e81d5457fad1d2dc0dd7d3adddc10ac74:

The current-head release-notary guard has three shared failures:

  • test_the_repository_highlights_validate: priority-view-drag.md needs a tryIt action/deeplink or docs link.
  • test_only_the_nightly_pickers_read_the_next_build_runner: the workflow has signer-dmg-image, verify-final-dmg-nightly and verify-signed-nightly beyond the expected list.
  • test_every_listed_hook_is_one_glaeda_released_with_the_check: the live glaeda hook 091eec20d4c4e6873354564b1bd25c96785c210ef8b6dbc16514279f99e1a493 is absent from the signing-hook list.

The first two failures also appear in the feat-cmux-next guard at 6aef70d31712a12e632de200eb4de2c0082ce3c5. All three, including the identical hook hash, appear in the unrelated stall-watchdog PR guard at 3872f74b1e0fbe98929fa5d545525ad65d3d145f. I read all three logs. The full diff against the merged base adds only CloudAgentChatTests.swift; the release, routing, digest and signing files are unchanged.

The exact-head Release compile, ci-status, risk evidence and merge-freeze checks pass. The exact-head fleet test run and independent SHIP review remain valid. ready-to-land is unchanged; awaiting next-autoland's verdict. No additional source change or push was made.

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 43267d8e Deployed Nov 1, 2026 by austinywang via cmux-tui tree preflight #92747
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:low Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants