Skip to content

carry of #27690: verify signed Computer Use hosts - #28223

Merged
teamleaderleo merged 9 commits into
feat-cmux-nextfrom
lane/carry-27690-base
Nov 2, 2026
Merged

teamleaderleo merged 9 commits into
feat-cmux-nextfrom
lane/carry-27690-base

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Carry of #27690 (#27690).

Original purpose: “ci: verify signed Computer Use hosts against nightlies.” This carry keeps the signed NEXT nightly Computer Use host admission check and merges the current feat-cmux-next base to remove the fork-point red in the swift-test-summary self-test. The implementation commits and authorship are preserved; no selector or test workaround is added.

Validation: merge of current origin/feat-cmux-next completed cleanly. Fleet checks remain queued because the cmux-ci controller is timing out on submits.

Agent session: agent session


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Carries PR #27690, adding a scheduled CI check that verifies signed Computer Use host admission against NIGHTLY app builds, and merges the current feat-cmux-next base to remove the fork-point red in the swift-test-summary self-test.

  • The workflow downloads the signed NIGHTLY apps from the NEXT appcast, verifies their bundle identities and team signature, then starts each app with both Computer Use drivers and confirms the authenticated host socket stays open for 3 seconds.
  • The check detects the effective driver NIGHTLY actually runs (no helper v2, so "upstream" uses the legacy helper) and speaks the legacy helper's machine_consent_host protocol.
  • Unsigned, ad-hoc tagged, and exec-chain peers must receive the stable host_unverified refusal; the legacy refusal never says attached.
  • The ad hoc and exec-chain probes use a small compiled C client, since /usr/bin/python3 is the xcrun shim and a copy named cmux ran the cmux CLI instead of speaking on the socket.
  • The NEXT nightly feed is the blocking contract; the classic identity is retained as a best-effort diagnostic that does not fail the check.
  • Registers the job in warm-sources.json as run-only (it compiles nothing) and classifies its Mac job as apple-gui in runners.json.
  • The pick job now reads the pool from fromJSON(vars.RUNNER_POOLS).linux, since the prior LINUX_RUNNER with a fixed Blacksmith pair is refused by the Linux size guard.

Written for commit aae82f8. Summary will update on new commits.

View guided diff Turn on auto-fix

@cursor

cursor Bot commented Oct 30, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 3277069: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28223 -f ref=lane/carry-27690-base -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on 1ecf8a8 to compare; do not merge until green; see lane-rules

Shipped in:

@github-actions github-actions Bot added the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 30, 2026
@blacksmith-sh

blacksmith-sh Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Found 6 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp2sv7yzlf/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmpzgpmthfw/summary-fail.json (live):/summary-fail.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, next-review): BLOCK at b0d74b1

  • scripts/ci/cua_signed_host_check.py:282-308: The check always tries the upstream driver first and waits for cmux-cua-v2/.../c.sock. Signed NIGHTLY Release builds do not embed the v2 helper; they map upstream to the legacy helper and expose cmux-cua/.../cua.sock. The NEXT check therefore times out and fails every healthy nightly before reaching the legacy path.

  • scripts/ci/cua_signed_host_check.py:157-170: lsof only proves that the app briefly opened the socket. It does not prove authenticated host admission, so a connection that the helper rejects can satisfy this check during the race window.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, next-review): BLOCK at 6a52a3a

BLOCK at 6a52a3a

Visual verdict: no images; Leo tries the build

  1. [P1] [blocking] cua_signed_host_check.py:307: The first driver check waits for a socket NIGHTLY never creates. Release builds omit the upstream helper (embed-cua-helper-dev.sh:28), and ComputerUseHelperDaemon.effectiveDriver explicitly falls back to legacy. This checker nevertheless waits for cmux-cua-v2/.../c.sock, fails after 45 seconds, and never reaches legacy validation. Detect the artifact’s supported driver and validate the effective route.

  2. [P1] [blocking] cua_signed_host_check.py:188: The legacy refusal probe does not speak the legacy protocol. cua.sock expects a request; its signature check runs for an authenticated machine_consent_host request. This probe only connects and reads, so it times out instead of receiving host_unverified. Even sending an unauthenticated request would produce a token refusal, not the expected signature refusal. Implement a driver-specific probe. A local request-driven socket reproduction confirmed the timeout and zero transmitted bytes.

  3. [P2] [follow-up] cua_signed_host_check.py:167: One descriptor observation does not establish authenticated admission. The function returns on the first matching lsof sample, potentially while signature verification or the handshake is pending. A subsequent rejection can therefore escape detection while the negative probes succeed. Require evidence of successful authentication and verify that the connection remains open.

  4. [P2] [follow-up] cua_signed_host_check.py:225: The exec-chain probe can pass without testing exec invalidation. It connects as an already-untrusted peer, sleeps, then reads the queued refusal after exec. I reproduced a passing result with a server that rejects immediately and performs no identity checks. Synchronize verification after exec using a peer that would otherwise be admitted.

Python syntax and two workflow guard suites passed. Native macOS execution was unavailable; another guard required Git metadata absent from the snapshot.

@teamleaderleo teamleaderleo removed the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@github-actions github-actions Bot added the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@teamleaderleo

teamleaderleo commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

The signed-host check waited for the helper v2 socket, which NIGHTLY never
creates (it has no helper v2, so upstream runs legacy), and its legacy
probes only connected and read, which the legacy helper never answers.
It now checks the effective driver, sends legacy peers a
machine_consent_host request and requires a refusal, and requires the
app's host connection to stay open for 3 seconds. Declare the
signed-host job in warm-sources.json (it compiles nothing).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-63f0bcf9b78ecec06a14
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent claude-opus-5-5, correctness): BLOCK at d9b8abb
Visual verdict: unchanged by this diff (no UI)

Checked the three claimed fixes against the product code and the pinned cmux-cua serve.rs (e8a8dbf) plus scripts/cmux-cua-machine-consent.patch. (1) is fixed: effective_driver matches ComputerUseHelperV2.embeddedHelperApp / ComputerUseHelperDaemon.effectiveDriver, so on NIGHTLY "upstream" now waits on cmux-cua/<scope>/cua.sock. (2) is fixed: the probes send a machine_consent_host line, and with no authorized root configured the helper answers it {"ok":false,...}. The lsof check now has to hold for 3 s.

[blocking] ad_hoc_peer copies /usr/bin/python3 into .../Contents/MacOS/cmux, but that file is the xcrun shim (same inode as /usr/bin/git), and the shim picks its tool by its own name. I tried it on a fleet mini: the ad-hoc signed copy named cmux runs the cmux CLI (Error: Failed to connect to socket at ~/.local/state/cmux/cmux.sock), even with PATH=/usr/bin:/bin. Python never runs, so stdout is empty, refused() returns False, and ad_hoc_refusal raises on every healthy NIGHTLY. exec_chain_refusal uses the same binary. The job has never run (pick reported no host), so CI has not exercised this path yet. Suggested fix: build a small socket client with clang (Xcode is already selected in the job), or copy a real interpreter, then ad-hoc sign it.

Follow-ups:

  • [follow-up] On legacy, all three negative probes are refused at the agent auth_token check (parse_request → "Unauthorized daemon client"). That happens before the host token and before macos_peer_cmux_app_check, so these probes prove token gating, not the code-signature check. The docstring says the host token is what refuses them; it is the agent token.
  • [follow-up] Nobody has yet confirmed on macOS that lsof -U -p <app> prints the server path for the app's client end of a Unix socket. Confirm it on the first real run.

/usr/bin/python3 on macOS is the xcrun shim, which picks its tool by
its own name: a copy named cmux ran the cmux CLI, printed nothing, and
failed the ad hoc and exec-chain probes on every healthy NIGHTLY. Build
a small C client with xcrun clang instead and ad hoc sign it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-63f0bcf9b78ecec06a14
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent claude-opus-5-5, correctness): SHIP at cbbbe02
Visual verdict: unchanged by this diff (no UI)

Fixes the previous block. Checked the C probe: argc is 4 in both modes and on the re-exec (argv[0] fd N REQUEST). argv[0] is the absolute bundle path, so execv re-runs the same signed binary. socket() has no CLOEXEC, so the fd survives the exec. An empty request skips the write and just reads the server's line. The workflow's Select Xcode step writes DEVELOPER_DIR to GITHUB_ENV, so /usr/bin/xcrun clang resolves without any extra sparse-checkout paths.

On a fleet Mac (Xcode 26.6), in a temp dir I removed afterwards: the exact PROBE_SOURCE builds with xcrun clang -O1 -Wall with no warnings. The bundle ad-hoc signs as com.cmuxterm.app.debug.signed-host-probe with Signature=adhoc, and the binary runs the probe, not the cmux CLI. Against a Python unix-socket server, connect and exec both delivered the request and printed the reply line. With an empty request against a v2-style server that refuses before reading, both modes printed host_unverified. Bad argc exits 2. The docstring now matches the legacy agent-token ordering. Nothing else in the diff changed.

Follow-ups:

  • [follow-up] speak truncates the reply at 4095 bytes. A longer legacy reply would fail json.loads and read as not refused, which is a false red. Today's refusals are short.
  • [follow-up] subprocess.TimeoutExpired from probe() is not a Failure, so a hung peer exits with a traceback instead of FAIL:. This predates the PR's earlier heads.

teamleaderleo and others added 2 commits November 2, 2026 09:59
…ac job

pick read LINUX_RUNNER with a fixed Blacksmith pair, which the Linux size
guard refuses; read fromJSON(vars.RUNNER_POOLS).linux after the fork
branches. signed-host launches the signed app on a Mac: class apple-gui
in runners.json, like the launch smoke.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-63f0bcf9b78ecec06a14
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent claude-opus-5-5, correctness): SHIP at 3277069
Visual verdict: unchanged by this diff (no UI)

Checked the merge of feat-cmux-next (0057134): --remerge-diff shows no hand resolution, no conflict markers remain, and .github/warm-sources.json and .github/runners.json parse. The PR's diff against base is still only its 4 files, adding one line each to those two JSON files. 3277069: pick now ends in fromJSON(vars.RUNNER_POOLS).linux after the fork branches, the same form cmux-tui, ci-guards and others use. ci-guards and nightly.yml already run side_lane_placement.py (python3 plus a sparse checkout) under ci-routing-agent on that pool, so the environment and the step still work on a pool runner. signed-host launches the published signed app and holds its host connection, so apple-gui (the same class as nightly-next-launch-smoke.yml#smoke) is right. Local results: test_runner_routing_table, test_ci_warm_sources, test_ci_nightly_off_blacksmith and test_ci_runs_on_fallback pass. test_ci_linux_single_size fails only on cloud-newlink-e2e.yml, which comes from base. test_ci_owned_pool_sources fails only on the expected rescue-watch entry, which main #32114 adds.

Follow-ups:

  • [follow-up] The fork branch of pick's runs-on names blacksmith-16vcpu-ubuntu-2404, where the house pattern uses blacksmith-4vcpu-ubuntu-2404. The job's if already excludes fork PRs, so this branch never runs. It is cosmetic only.

teamleaderleo added a commit that referenced this pull request Nov 2, 2026
#28223 adds .github/workflows/nightly-next-cua-signed-host.yml on
feat-cmux-next; its Mac job can take the owned side lane, and the guard
reads main's watch list, so list it here first.


Agent-Session: https://cmux.com/deeplink/agent?id=aid-63f0bcf9b78ecec06a14

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 2, 2026 10:25 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 2, 2026 10:27 Inactive
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent claude-opus-5-5, correctness): SHIP at aae82f8
Visual verdict: unchanged by this diff (no UI)
The base merge is clean (first parent 3277069, second parent 1ecf8a8 on feat-cmux-next, empty remerge-diff, PR diff on its four files identical to the SHIP'd head), both JSON files parse, and test_ci_owned_pool_sources, test_runner_routing_table and test_ci_warm_sources pass at this head.
Follow-ups: none

@teamleaderleo
teamleaderleo merged commit 37c493d into feat-cmux-next Nov 2, 2026
141 of 147 checks passed
@teamleaderleo
teamleaderleo deleted the lane/carry-27690-base branch November 2, 2026 10:30

This branch was previously deployed

1 inactive deployment
ci-routing-agent — aae82f83 Deployed Nov 2, 2026 by cmux-bot[bot] via wait for same-key tree publisher #100682
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant