Skip to content

ci(cmux-next): swift-test runs on a pull request only with full-ci - #36468

Merged
teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/swift-test-signal
Nov 5, 2026
Merged

teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/swift-test-signal

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-11-05: "we're kind of pushing a lot of frontiers here. We are at the same time, like doing a lot of goofy stuff and, like, keeping around a lot of baggage."

Problem

cmux-next swift test ran on every same-repository PR push and almost never finished. Of 78 sampled PR check runs (PRs opened Nov 2 to 5): 37 cancelled, 37 skipped, 4 failed, 0 passed. Its estimated cost over 14 days was about 373,000 runner-minutes.

Why the runs never finished:

  • Not a timeout. No cancelled run reached the 135-minute timeout-minutes; the longest lasted 97 minutes.
  • 17 were cancelled within 15 minutes of the PR merging. Merging does not wait for this check (it is not required, and pending checks don't hold a merge), so the PR merged first.
  • 10 were cancelled by a newer push to the PR (cancel-in-progress on pull requests).
  • 10 were cancelled at the same head by a re-run or re-trigger (for example Tab drag: forward index decisions to Rust (migration shard 002) #34545, three times at one sha).
  • The newest runs (Nov 5) are mostly skipped.

Choice

The base already gets its Swift verdict another way. A feat-cmux-next push skips this job, because push CI is the app build only (Lawrence, 2026-10-10; the jobs comment). On main, cmux-next-tip-swift-suite-dispatch.yml was added after a Swift red stayed hidden on the base for a day (2026-10-21; #23687, #26015). Every 30 minutes it dispatches a tip_swift_suite run on the feat-cmux-next tip whenever Swift changed. That run's cmux-next swift test is the base verdict that scripts/ci/base_red_excuse.py reads. Those dispatch runs do finish (for example, "cmux-next tip Swift suite" runs today).

I did not move the job onto the feat-cmux-next push. That would undo the app-build-only rule, and the tip dispatch already batches the same work at the tip with a retry cap. I also did not remove the job: the dispatch and full-ci still need it. So this takes option (b):

  • A pull request runs swift-test only with full-ci (same-repository or fork), the documented opt-in. A dispatch on any ref still runs it, and so does the tip suite dispatch.
  • Generated files keep their verdict on PRs. path_route's generated_host no longer picks swift-test on a PR without full-ci. It falls through to daemon-test (when the tree is ready) or to the generated-files job's own build, so the generated files are still checked exactly once.
  • The Mac placement cascade (CASCADE_RUNS) stops counting swift-test for a PR without full-ci, so placement does not reserve a mini for a job that won't run.
  • fleet/test-compile (advisory) compiles CmuxNext's test targets on PRs again. It had skipped CmuxNext since 2026-10-30 because swift-test already built those targets on every same-repository PR (tests/test_ci_compile_test_plan.py pinned that dependency). With swift-test behind full-ci, the advisory fleet compile is again the one PR-side compile of those targets. It is a compile-only step with a 25-minute run cap, against swift-test's runs of up to 97 minutes.
  • generated-verdict, push-attribution and the push-only job that needs swift-test are unchanged. They already handle a skipped swift-test.

Not changed: required checks, the merge policy, the push concurrency, and the job's steps.

Tests

  • tests/test_cmux_next_generated_fold.py: the exhaustive matrix gains a full-ci label axis. It asserts that a PR without full-ci never runs swift-test or routes generated files to it, and that the generated files are still checked exactly once. The spelled-out host cases now cover PRs with and without the label.
  • tests/test_cmux_next_checks_workflow.py: swift-test joins the heavy compiles behind full-ci. The same-repository PR tier set and the cascade set drop it without the label.
  • tests/test_cmux_next_route.py and tests/test_ci_compile_test_plan.py: their pins of the old swift-test condition and of the fleet-test-compile SKIP now pin the new behavior.
  • Ran on the fleet: test_cmux_next_generated_fold, test_cmux_next_checks_workflow, test_cmux_next_tip_swift_suite, test_ci_next_macos_cascade, test_ci_workflow_guards_are_wired, test_base_red_excuse (579 tests, all pass); then, at the final head, test_ci_compile_test_plan, test_cmux_next_route, test_ci_display_suites, test_ci_linux_single_size, test_ci_overflow_pick_reads_pool and the files above (all pass).

Changelog

none

🤖 Generated with Claude Code

Agent session: agent session

On every PR push the job gave no signal: of 78 PR check runs (Nov 2-5) 37
were cancelled (17 by the PR merging first, 10 by a newer push, 10 by a
re-run at the same head), 37 skipped, 4 failed and none passed. The base
verdict already comes from main's tip suite dispatch, which runs the job on
the feat-cmux-next tip whenever Swift changed. A PR now runs it only with
full-ci (or a dispatch); without it, path routing folds the generated files
check into daemon-test or the generated-files job, and the Mac placement
cascade no longer counts it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3bab30503f0b9a7059e5
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 6dad1da: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr36468 -f ref=lane/swift-test-signal -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 5, 2026

@teamleaderleo teamleaderleo left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review (codex gpt-6-astra high, next-review, since=20261105T113014Z): BLOCK at bbe9b66

BLOCK at bbe9b66

Visual verdict: unchanged by this diff

    • [P1] [blocking] [category:reliability] [finding:F1] [outcome:pending] .github/workflows/cmux-next.yml:1479 removes swift-test from CASCADE_RUNS, but JOBS at line 1520 still includes it. side_lane_placement.decide() assigns idle minis from JOBS without consulting CASCADE_RUNS, so an unlabeled PR can assign an idle runner to the skipped swift-test; that runner remains unused while later daemon-test/generated-files jobs queue on the side lane.
    • [P1] [blocking] [category:tests] [finding:F2] [outcome:pending] The new condition at .github/workflows/cmux-next.yml:3681 removes the same-repository exception, but tests/test_cmux_next_route.py:579 still asserts that exception. python3 tests/test_cmux_next_route.py fails (1 of 127), and this test is run by the workflow’s tier-routing checks.

@blacksmith-sh

This comment has been minimized.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 1de2648

Reviewer: codex gpt-6-astra high, fast tier.

SHIP at 1de2648

Visual verdict: no images; Leo tries the build

Reviewed the workflow expressions, generated-file routing, cascade placement, and focused test suites. All 706 focused tests passed.

Follow-ups: none

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 1de2648: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

The PR check skipped CmuxNext because cmux-next.yml's swift test job built
its test targets on every same-repository PR. That job now runs on a PR
only with full-ci, so without this a PR's CmuxNext test-target compile
break would show only on the tip. The advisory fleet compile is the one
compile of those targets again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3bab30503f0b9a7059e5
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 6dad1da

Reviewer: codex gpt-6-astra high, fast tier.

SHIP at 6dad1da

Visual verdict: no images; Leo tries the build

Targeted workflow, routing, generated-fold, compile-plan, cascade, and base-red tests passed. The guard-wiring test requires Git metadata absent from the review fixture.

Follow-ups: none

cmux-bot[bot]
cmux-bot Bot previously approved these changes Nov 5, 2026

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 6dad1da: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 5, 2026 13:55 Inactive
@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 5, 2026 13:56 Inactive
@cmux-bot
cmux-bot Bot dismissed their stale review November 5, 2026 13:58

A newer verdict blocks 68f81b3 (no independent SHIP at the head).

@teamleaderleo
teamleaderleo merged commit e2e80b3 into feat-cmux-next Nov 5, 2026
202 of 209 checks passed
@teamleaderleo
teamleaderleo deleted the lane/swift-test-signal branch November 5, 2026 15:17

This branch was previously deployed

3 inactive (2 outdated) deployments
ci-routing-agent — 68f81b39 Deployed Nov 5, 2026 by cmux-bot[bot] via wait for same-key tree publisher #116447
artifacts — 1de2648d Deployed Nov 5, 2026 by teamleaderleo via publish tree-addressed cmux-tui #116136
sentry-dsyms — 1de2648d Deployed Nov 5, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #116136
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant