Repository navigation
ci: a CmuxNext suite lane never waits for an active cmux-tui artifacts run - #27454
Conversation
…commit must not wait
package-test-lane.sh sets CMUX_TUI_TREE_WAIT_SECONDS=0 and says its live-daemon
fetch "never waits for an unpublished tree", but on a fleet host (no
GITHUB_ACTIONS) pin-cmux-tui.sh turns an active cmux-tui artifacts run of the
commit into a wait of up to CMUX_TUI_TREE_ACTIVE_WAIT_SECONDS (7200 s), which
the controller does not let a step set. Steps 036f288649bf (3065 s, then
cancelled), 31482fc9e743 (7184 s) and 0b2b6232e4c8 (14 min) held a Mac in that
wait and ran no test.
The lane test runs the real pin-cmux-tui.sh against an unreachable CDN and a
file:// GitHub API that lists one queued run, on the virtual clock
(fake-clock.sh). The pin test asserts that a wait of 0 with an active run
waits 0 s and reads no run state.
Red on this commit (the fix follows):
python3 tests/test_ci_package_lane_suite.py
FAIL: the lane waited 7200s for the active artifacts run
bash scripts/cmux-next/tests/pin-cmux-tui-run-check.test.sh
fleet, no wait, active run: waited 9s
…tive artifacts run The fleet suite lanes (package-test-lane.sh, run-swift-testing-suites.sh) and the daemon integration tests fetch this tree's cmux-tui with CMUX_TUI_TREE_WAIT_SECONDS=0: look once, and on a miss let the live-daemon suites skip (the fleet worker's skip guard then names them and fails the step). The worker gives a suite step its binary first (hq step_tui.go: the published tree, a stored client or a host build within 25 min); the lane's own fetch runs only after that failed. On a fleet host wait_for_tree then extended the 0 to the 7200 s active-run cap whenever the commit's artifacts run was queued or running, so the step held its Mac for up to two hours and ran nothing (036f288649bf: 3065 s until cancelled; 31482fc9e743: 7184 s; 0b2b6232e4c8: 14 min). The active-run extension exists for callers that do wait (fleet app builds, 2700 s base). A wait of 0 now looks once and fails at once: no run-state reads, no extension, no poll. Dispatch (CMUX_TUI_TREE_DISPATCH=1) is unchanged. One path, no timer deciding the result. Green: python3 tests/test_ci_package_lane_suite.py -> ok (waited 0 s) bash scripts/cmux-next/tests/pin-cmux-tui-run-check.test.sh -> ok Real API (VM, commit 44d0c58 while its artifacts run 40889589007 was pending): old pin still waiting at the 420 s timeout; new pin exits 1 with "not waiting" after the one look.
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Passes: CI passes on CI passes on Written by red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on 719c550 to compare; do not merge until green; see lane-rules |
lawrencecchen
added
ready-to-land
|
Bead |
|
Review (independent review subagent on this PR's own head): SHIP at 2362f12. It checked feat-cmux-next's callers of wait_for_tree (the two fleet suite lanes and the daemon tests use 0; the tip wait returns before wait_for_tree at 0; the workflows use 120 and 2700). tests/test_ci_package_lane_suite.py passes at the head and fails on HEAD~1 with "the lane waited 7200s for the active artifacts run". |
lawrencecchen
added
the
ready-to-land
|
Found 6 test failures on Blacksmith runners: Failures
|
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
…of 0 reads no runs pin-cmux-tui.test.sh's public-repository fallback case (an active run whose commit only the public repository has) ran fetch with CMUX_TUI_TREE_WAIT_SECONDS=0 and relied on the run check to fetch that commit. With this PR a wait of 0 looks once and reads no runs, so the case now waits 1 s (poll 1 s): the run check runs once, fetches the commit and the wait ends. Red in CI on 2362f12 (job 123529356192: 'the run commit was not fetched from the public repository'); green here.
lawrencecchen
removed
the
ready-to-land
GitHub Actions
Inactive
|
Review (independent review subagent on this PR's head): SHIP at 31ce1d4. The public-fallback case in pin-cmux-tui.test.sh still reaches the run check on its first loop, with a 1 s wait, and fetches the public-only commit. No other feat-cmux-next test depends on a wait of 0 with a run check. Every pin-cmux-tui and tree test of the cmux-next script-tests step passes at the head. test_ci_package_lane_suite.py and the new run-check case fail on the pre-fix scripts and pass at the head. |
lawrencecchen
added
the
ready-to-land
scripts/merge-main.sh merges only main (BASE_BRANCH = "main" in scripts/ci/merge_main.py), so this PR into feat-cmux-next takes feat-cmux-next with a plain merge; git merge-tree found no conflicts.
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
lawrencecchen
removed
the
ready-to-land
GitHub Actions
Inactive
GitHub Actions
Inactive
|
Review (independent review subagent on this PR's head): SHIP at 8a5d514. This head is a merge of feat-cmux-next 9957fbd, and the PR's diff against the base is still the 4 reviewed files, line for line. The only base commit in those paths (a653ee3) adds an echo in package-test-lane.sh and does not touch the wait. pin-cmux-tui.test.sh, pin-cmux-tui-run-check.test.sh and test_ci_package_lane_suite.py pass at the head. |
lawrencecchen
added
the
ready-to-land
Autoland holds the PR with 'base red since fixed ... needs a base merge', and its update-branch call gets a 403 when feat-cmux-next changed workflow files (cmux#13742), so the author merges the base.
GitHub Actions
Inactive
lawrencecchen
removed
the
ready-to-land
|
Review (independent review subagent): SHIP at e34192c. This head is a plain merge of feat-cmux-next 719c550. The PR's diff is still the same 4 files, line for line, and pin-cmux-tui.sh is the same file that got SHIP at 8a5d514. The only base commits in these paths (#27545, #26558) do not touch the wait. pin-cmux-tui.test.sh, pin-cmux-tui-run-check.test.sh and test_ci_package_lane_suite.py pass at the head. |
lawrencecchen
added
the
ready-to-land
|
Review (codex gpt-6-astra high, next-review): SHIP at e34192c No blocking findings. Syntax checks and isolated zero-wait hit/miss checks passed. Full tests were not run because this read-only snapshot omits required helpers. |
|
Shipped in:
|
|
Shipped in:
|
… main) (#27543) * test(merge-main): --base feat-cmux-next merges feat-cmux-next's tip, and the default base stays main Red on the base: merge_main.Options has no base field and main() has no --base option, so both tests fail (TypeError / unrecognized arguments). On 2026-10-29 #27454 had to be brought current with a raw git merge because merge-main.sh merges only main. * merge-main: --base <branch> merges another base branch (feat-cmux-next PRs); default main scripts/merge-main.sh merged only main (BASE_BRANCH), so a pull request into feat-cmux-next could not use it. --base feat-cmux-next fetches and merges that branch through the same resolver. ci-fast-guards.yml runs on pushes to main only, so only main has a green-commit verdict, and a feat-cmux-next merge never waits on a base red: any base outside GREEN_PICK_BRANCHES merges its tip and reads no verdict. The default stays main, so nothing changes for main PRs. docs/ci/merge-main.md lists the option. * merge-main: a non-main base's tip reads as 'missing', never 'success' (review of #27543) With --guards --base feat-cmux-next the guard report said "main's CI fast guards passed on that commit" about a feat-cmux-next tip, because the tip was put in Selection.chosen. A base outside GREEN_PICK_BRANCHES now has the verdict 'missing'. The new test fails without this line ('success' != 'missing'). Also lists --base in merge-main.sh's usage and corrects the GREEN_PICK_BRANCHES comment.
…mmits stay errors (#27632) * test(prepush-gate): a pull request's sidebar sweep warns on a base commit and errors on its own Red on the base: prepush-gate.py has no --own-range ('unrecognized arguments'). On 2026-10-29 two feat-cmux-next commits without a Sidebar-Look record (#24073, #24295) put an ##[error] on every pull request's cmux-next checks, and next-autoland read it as the PR's own red (#27454 and four more PRs). * ci(cmux-next): a pull request's sidebar sweep names a base commit as a warning; its own commits stay errors The "Pushed commits add no violations" step sweeps every commit since 342462c for a Sidebar-Look record. On a pull request run it printed ::error for base commits too: 4367c16 (#24073) and 6c1a5ab (#24295) put an error on every feat-cmux-next pull request's checks, and next-autoland read that line as the PR's own red, so #27454 and four more PRs stayed held. prepush-gate.py --own-range BASE..HEAD (the sweep only): a finding on a commit inside the range is an ::error and fails the sweep as before; one on a commit outside it is a ::warning that names the commit and plans/cmux-next/sidebar- owner.md. cmux-next.yml passes --own-range HEAD^1..HEAD on pull request runs only. A push sweep of feat-cmux-next is unchanged: every commit without a record is still an error there, so the base's debt stays visible on the branch. risk_tier.py classify: normal. Test: test_cmux_next_prepush_gate's new case fails on the base and passes; the whole file passes (40 tests). * ci(cmux-next): own-range on pull_request runs only; the warning carries the whole finding (review of #27632) A workflow_dispatch run (feat-cmux-next, next-batch/*, a PR branch) checks out a branch tip, not GitHub's merge commit, so HEAD^1..HEAD there is the tip commit only: the != push test turned the branch's own debt into warnings. Now == pull_request, as the app-FFI pin step tests. The warning text is the whole finding, % and newlines escaped for the workflow command.
Summary
Fleet Mac suite steps (
package-test-lane.sh suite Packages/macOS/CmuxNext ...,run-swift-testing-suites.sh) held a Mac for up to two hours and ran no test while the commit'scmux-tui artifactsrun was queued or running.The lane fetches the tree's cmux-tui with
CMUX_TUI_TREE_WAIT_SECONDS=0, which is meant to look once: on a miss, the live-daemon suites skip. But off GitHub Actions,wait_for_treeinpin-cmux-tui.shstretched that 0 to the active-run cap (CMUX_TUI_TREE_ACTIVE_WAIT_SECONDS, 7200 s) whenever an artifacts run for the commit was active. The controller does not let a step set that variable.Seen on the fleet:
31482fc9e743: waited 7184 s.036f288649bf: waited 3065 s, then was cancelled.72dac7f62135: waited 496 s or more, then was cancelled.0b2b6232e4c8: waited 14 min.All of them reached this fetch only after the worker's own cmux-tui recipe (hq
step_tui.go, 25 min bound) had failed or timed out.Now a wait of 0 looks once and exits 1 at once. It reads no run state, does not extend the wait and does not poll. Callers with a real wait (app builds 2700 s, workflows 120 s, release) keep the active-run extension unchanged. Dispatch is unchanged.
Testing
python3 tests/test_ci_package_lane_suite.py: new check. It runs the realpin-cmux-tui.shthrough the lane against an unreachable CDN and afile://GitHub API that lists one queued run, on the fake clock. Base:FAIL: the lane waited 7200s for the active artifacts run. Head:package-test-lane suite: ok.bash scripts/cmux-next/tests/pin-cmux-tui-run-check.test.sh: new case. Base:fleet, no wait, active run: waited 9s. Head: ok.Observed / Not run
pin-cmux-tui.test.sh,tree-key-v2.test.sh,reset-stale-submodules.test.sh,pin-cmux-tui-fallback-key.test.shandpin-cmux-tui-dispatch.test.shpass (run on the perf copy, where thewait_for_treediff is the same).prepush-gate.pyreports no new violations.964c7524a9a2.Performance
No perf numbers: this changes how long a fleet CI test lane waits for a cmux-tui artifact (a wait of 0 now looks once). It changes no app, daemon or UI code path. "perf" in this description names the
perf/chrome-paritybranch copy, not a performance change.Changelog
none
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Fixes fleet CmuxNext suite lanes holding a Mac for up to two hours while waiting on an active
cmux-tuiartifacts run instead of doing a single look.A lane's
CMUX_TUI_TREE_WAIT_SECONDS=0meant to check once and skip on a miss, but on fleet hosts (outside GitHub Actions)wait_for_treeextended the 0 to the 7200 s active-run cap whenever the commit's artifacts run was queued or running. The lane's own fetch only runs after the worker's cmux-tui recipe failed or timed out, so the step ran no tests while a Mac sat idle (observed waits: 7184 s, 3065 s, 14 min).Written for commit e34192c. Summary will update on new commits.