base red: nightly.yml under its 490,000-byte budget (30 run blocks move to scripts/ci/nightly-steps/) by lawrencecchen · Pull Request #30912 · manaflow-ai/cmux

lawrencecchen · 2026-11-01T12:43:33Z

tests/test_workflow_file_size.py (ci-guards preflight) caps nightly.yml at 490,000 bytes. nightly.yml was 505,604 bytes (508,735 before #30824), so this check was red on every PR. Lead decision, 17:37Z: move run blocks, as #25879 did.

Nightly owners (#30284, #30249, #30471, #30278, #30113): nightly.yml now calls 30 more step scripts in scripts/ci/nightly-steps/. To change one of these steps, edit its script. tests/nightly_workflow_text.py shows the workflow with the scripts written back inline.

What changes

30 run: | blocks move verbatim into scripts/ci/nightly-steps/<name>.sh. Each step now runs bash scripts/ci/nightly-steps/<name>.sh, with the same env:, if:, id:, shell and working directory.
nightly.yml goes from 505,604 to 486,014 bytes.
yaml.safe_load of head and base give the same top level, the same jobs and the same steps, except those 30 run strings.
tests/nightly_workflow_text.py expands head to exactly the base's expansion, byte for byte.

How the blocks were chosen (so behavior does not change)

Shell options: each block starts with set -euo pipefail and uses no ${{ }}. bash script.sh does not inherit GitHub's bash -e, so the script sets the same options the inline block did.
Excluded: steps with a working directory, a run block that performs a checkout, and the "Hold the whole Mac before any credential" security gate, whose text test_ci_signing_owned_mac.py pins.
The script exists in the tree the step runs on:
- refresh-compilation-cache, refresh-test-compilation-cache and resolve-nightly-cmux-tui-client check out the workflow commit (decide's head_sha is context.sha).
- build-nightly-app, build-nightly-ghostty-cli-helper and build-nightly-remote-daemon check out build_sha. All three are skipped by an unsigned resume, which is the run that builds an older saved tree.
- The 20+ scripts these jobs already call (chief-*, inputs-web-bundles, visual-gate-*) work the same way.

Remaining exposure

build_sha always has this PR's nightly.yml, so it also has its scripts. The resolver runs with CMUX_TUI_TREE_SAME_PATHS: .github/workflows/nightly.yml ... (nightly.yml:3028). scripts/cmux-next/pin-cmux-tui.sh then skips any candidate whose nightly.yml differs from the tip's. The tree-pending, fast and build-only paths build the tip.

Three edge cases are left:

"Bound the kept Rust target directories" runs under always(). If the warm checkout fails before it reaches build_sha on a builder whose kept tree is older than this PR, the step fails and does not trim. The job has already failed in that case.
The script now comes from the build_sha tree, not from the tip's workflow. This is the same as for the 20+ scripts that were already moved.
Tests that read nightly.yml without expanding it (for example test_ci_self_hosted_guard.sh) no longer see the moved bodies. Nothing violates those checks today.

Tests

The tests that read the raw nightly.yml text of a moved block now read it through nightly_workflow_text.expand, the helper #25879 added for this. Only the loading changed, not the checks. The 8 files: test_ci_fleet_compilation_cache.py, test_ci_nightly_off_blacksmith.py (only the reads of moved blocks; the ensure-gh test still reads raw), test_ci_nightly_push_throttle.py, test_ci_test_compilation_cache_seed.sh, test_nightly_cmux_tui_resolve.py, test_nightly_ghostty_helper_cache.py, test_nightly_next_source_archive.py and test_nightly_trusted_routing.py.

No new CI job, step or gate. Risk tier: high (nightly.yml). This PR needs a SHIP receipt at the exact head and a green compile.

Red and green

python3 tests/test_workflow_file_size.py:
- Red at 623850468f1 (base red: release-notary green (helper admission gate, nightly.yml size, universal build pin, warm sources, promotion admission, highlight) #30824 head) and at the feat tip: "505604 not less than 490000".
- Green at this head.
All 124 tests that read nightly.yml give the same results as at base, except test_workflow_file_size.py, which turns green.
shellcheck -S warning on the 30 new scripts: clean. This is the level the repo's actionlint step gives shellcheck.
actionlint 1.7.12-manaflow.1 (the repo's pin) on nightly.yml: the same 5 findings as at base, all unused anchors that already exist there.

Observed / Not run

Observed: everything above, plus prepush-gate.py.
Not run: no nightly run. That needs the nightly-next push path; the fleet compile on this PR is the compile gate.

Changelog

none

#30249 routed to the dedicated mini #30249 (d8f537d) let ten nightly.yml control jobs take the dedicated nightly mini (vars.CI_NIGHTLY_ONLY_RUNNER), so tests/test_ci_warm_sources.py counts them as Mac jobs, and they had no entry. Each takes mac-sign, the class of the other nightly.yml Mac jobs that compile nothing, with an exemption that says what the job does. cmux-tui-spec.yml:clippy (also unlisted; it wires Swatinem/rust-cache) takes linux-rust with actions-cache, as its cmux-tui-spec peers do. Red at 9288e32: python3 tests/test_ci_warm_sources.py FAILED (11 unlisted jobs) Green here: same command, OK.

…rged helper route and admission gate The test pins the full runs-on and if: of the nightly helper jobs. Two merged changes moved them: - #30249 (d8f537d) put the dedicated nightly mini (vars.CI_NIGHTLY_ONLY_RUNNER) first in build-nightly-ghostty-cli-helper's runs-on. The pinned expression now has that route; the merged workflow is the source of truth. - #30284 (4e81bfe) added the early admission conjunct to the resolver and the helper lookup. The test now names it (EARLY_ADMISSION) and checks that the resolver and the helper carry it as one conjunct after their gate, and that build-nightly-app keeps its own gate (it is skipped with the resolver it needs). The build_only exclusion is checked the same way as before. Red at 9288e32: bash ./tests/test_nightly_universal_build.sh FAIL: nightly must build and hand off the macOS 26 helper ... Green with the nightly.yml commit after this one: same command, PASS.

… admission gate; nightly.yml back under its headroom tests/test_nightly_ghostty_helper_cache.py requires build-nightly-ghostty-cli-helper and resolve-nightly-ghostty-cli-helper to share one if:. #30284 (4e81bfe) added the early admission conjunct to the lookup on purpose; the helper build now has it too, so it never builds while its lookup waits for admission. tests/test_nightly_workflow_size.py: nightly.yml was 508,735 bytes, over the 508,000 headroom. The cache save steps repeated the same three R2 env values six times each (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, CI_CACHE_R2_ENDPOINT); each is now an anchor on its first use and an alias on the others, as &pub-feeds, &pub-dmg and &pub-tooling already are. yaml.safe_load gives the same workflow as before except the helper's if:. nightly.yml is 507,061 bytes. No step is removed and no limit changes. (Job-level anchors are refused by test_nightly_signer_jobs.py and runner_routing_table.py, and step aliases hide text that test_nightly_notices_submodule.py and test_nightly_promotion_admission.py count, so the anchors are on step env.) Red at 9288e32: python3 tests/test_nightly_ghostty_helper_cache.py FAILED (failures=1) python3 tests/test_nightly_workflow_size.py 508735 not less than 508000 Green here: both OK. All 123 tests that read nightly.yml: the same results as at 9288e32 except these and test_nightly_universal_build.sh, now green.

…d-nightly-notary

…ed promote-guard checkout per job #30471 (e27f116) shares promote-nightly-next's "Checkout the promote guard from main" with promote-nightly-next-acked through the YAML anchor &legacy-route-shared-4, so `scripts/ci/nightly_next_train.py` appears once in the text and test_both_promote_decisions_pass_the_train_verifier (it counted the text) failed. It now finds the two jobs that call guard.trainVerifier in the parsed workflow and requires each to check out the verifier; it fails when either job's checkout drops it (checked by removing it from the anchor). Red at origin/feat-cmux-next 265c68a: python3 tests/test_nightly_promotion_admission.py AssertionError Green here: PASS (75 cases).

…s its pull request #30357 (7d5f3ae) added release-notes/next/highlights/cloud-environment-editor.md with no try-it action, docs or pr link, so scripts/whats-new/test_digest.py failed. It links #30357, as #30624 did for three others. Red at 265c68a: python3 scripts/whats-new/test_digest.py FAILED (failures=1) Green here: OK.

…d-nightly-notary

…30 run blocks move to scripts/ci/nightly-steps/ tests/test_workflow_file_size.py (ci-guards preflight) caps nightly.yml at 490,000 bytes; it was 505,604 after #30824 (508,735 before it), red on every pull request. Lead decision 17:37Z: move run blocks, as #25879 did. 30 `run: |` blocks move verbatim into scripts/ci/nightly-steps/<name>.sh; each step now runs `bash scripts/ci/nightly-steps/<name>.sh` with the same env:, if:, id:, shell and working directory. yaml.safe_load gives the same jobs and steps as before except those 30 run strings, and tests/nightly_workflow_text.py puts the scripts back inline byte-identical to the old workflow. nightly.yml goes from 505,604 to 486,014 bytes. Which blocks: - Every block uses no ${{ }} expression and starts with `set -euo pipefail`, so the script sets the same shell options the inline block did (bash scripts/x.sh does not inherit GitHub's `bash -e`). - No block has a working-directory, runs inside a checkout it performs, or is a security gate whose text a test pins ("Hold the whole Mac before any credential"). - Each step's script is in the tree the step runs on: steps in refresh-compilation-cache, refresh-test-compilation-cache and resolve-nightly-cmux-tui-client check out the workflow commit (decide's head_sha is context.sha); steps in build-nightly-app, build-nightly-ghostty-cli-helper and build-nightly-remote-daemon check out build_sha and are skipped by an unsigned resume, the run that builds an older saved tree. Tests that read the raw nightly.yml text for a moved block now read it through tests/nightly_workflow_text.py (expand), as #25879 set up; what they check is unchanged. The new scripts pass shellcheck -S warning (the level actionlint gives shellcheck), and actionlint reports the same 5 findings on nightly.yml as before (unused anchors). Red at 6238504 (#30824 head): python3 tests/test_workflow_file_size.py 505604 not less than 490000 Green here: OK. All 124 tests that read nightly.yml: the same results as at 6238504 except test_workflow_file_size.py, now green.

…d-nightly-size

github-actions · 2026-11-01T12:43:46Z

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review 17bf155: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr30912 -f ref=fix/base-red-nightly-size -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); base red: the same failure on the base fe91eba; does not block

Shipped in:

cmux NEXT nightly 1.0.0-nightly.4131523971401 (fa5830f6d5)
cmux NEXT nightly 1.0.0-nightly.4131541733001 (49aa2917ea)
cmux NEXT nightly 1.0.0-nightly.4131674790301 (331563226d)
cmux NEXT nightly 1.0.0-nightly.4131746817401 (af670df60f)
cmux NEXT nightly 1.0.0-nightly.4131829275601 (829d1467c6)
cmux NEXT nightly 1.0.0-nightly.4131862054001 (593b25f9f6)
cmux NEXT nightly 1.0.0-nightly.4131908227901 (ab8c3a2196)

cubic-dev-ai

1 issue found across 39 files

_{You’re at about 93% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/ci/nightly-steps/helper-build.sh">

<violation number="1" location="scripts/ci/nightly-steps/helper-build.sh:1">
P2: This newly extracted script is unavailable when the nightly builds a pre-PR commit: the helper job checks out that build SHA before invoking the relative path, so older nightly builds fail immediately with “No such file or directory.” Keep the helper script available from the workflow revision or add a compatibility fallback for older build refs.</violation>
</file>

_{Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic}

cubic-dev-ai · 2026-11-01T12:48:42Z

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash


P2: This newly extracted script is unavailable when the nightly builds a pre-PR commit: the helper job checks out that build SHA before invoking the relative path, so older nightly builds fail immediately with “No such file or directory.” Keep the helper script available from the workflow revision or add a compatibility fallback for older build refs.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At scripts/ci/nightly-steps/helper-build.sh, line 1: <comment>This newly extracted script is unavailable when the nightly builds a pre-PR commit: the helper job checks out that build SHA before invoking the relative path, so older nightly builds fail immediately with “No such file or directory.” Keep the helper script available from the workflow revision or add a compatibility fallback for older build refs.</comment> <file context> @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# nightly.yml build-nightly-ghostty-cli-helper: "Build Ghostty CLI helper". Moved out of the workflow to keep nightly.yml under +# its size budget (tests/test_workflow_file_size.py); the step's env: still sets every input. </file context>

lawrencecchen · 2026-11-01T13:13:28Z

The cmux-next checks red on this PR is a base finding, not caused by this PR: "Default cursor" (check-default-cursor.py) finds 2 pointer cursors in webviews/src/agent-session/acpmux/cloudStart.css (from #30386). It is also red at origin/feat-cmux-next 12d83e7f433. This PR does not touch that file. #30890 fixes it.

lawrencecchen · 2026-11-01T13:16:28Z

Review (subagent, correctness first): SHIP at 17bf155

What the reviewer confirmed at this head:

yaml.safe_load of head and merge base: the same top level, jobs, job keys and step counts. Exactly 30 steps differ, only in run, and each script body equals its old inline block. expand(head) == expand(base) byte for byte.
No moved step has a shell: or a working directory. Each block starts with set -euo pipefail, and none uses ${{, $0, BASH_SOURCE, return or stdin.
Every moved step comes after a full checkout, of head_sha (= context.sha) or of build_sha. Every build_sha step is skipped by an unsigned resume. The resolver's CMUX_TUI_TREE_SAME_PATHS check means build_sha always has this nightly.yml and its scripts.
No secret literal is in any script.
Mutation map: emptying each of the 30 bodies gives the same test failures at head as the same change gives at base, so no check became vacuous.
The size test, the nightly size test, the universal build test and all 8 edited tests pass. shellcheck -S warning on the 30 scripts is clean.

Non-blocking findings:

The "Remaining exposure" paragraph overstated the risk. It is corrected in the description.
The always() Rust-target step after a failed warm checkout.
The scripts follow build_sha.
Unexpanded guards no longer see the moved bodies.
Only 3,986 bytes of room are left under the cap.

blacksmith-sh · 2026-11-01T13:37:55Z

Found 6 test failures on Blacksmith runners:

Failures

Test	View Logs
`/tmp/tmp6fpvc6t3/summary-fail.json (live):/summary-fail.json (live):`	View Logs
`/tmp/tmpw_al6l2u/summary-missing.json (live):/summary-missing.json (live):`	View Logs
`[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/` `[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows`	View Logs
`[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames`	View Logs
`[tabs] debug.tab_switch timed 0 of 40 switches/` `[tabs] debug.tab_switch timed 0 of 40 switches`	View Logs
`[tabs] the window draws no frames/[tabs] the window draws no frames`	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

lawrencecchen · 2026-11-01T13:38:34Z

Review (subagent, correctness first): SHIP at 2036240

Merge commit has no changes of its own: git show --remerge-diff is empty.
PR diff unchanged: patch-id of merge-base(feat, head) vs head is b75c8b374a4a1c1f1680e9c64b576840f5127abd for both 17bf155 and the new head.
feat's only change since the prior head on the PR's paths (fix(nightly-next): bind cmux: on the feed root and read feeds that lack it (publish red) #30932) edits scripts/ci/nightly-steps/publish-next-feed-keep-nightly.sh, which feat already had and this PR does not touch. nightly.yml is byte-identical to 17bf155.
nightly.yml at the new head is 486,014 bytes, under the 490,000 cap in tests/test_workflow_file_size.py.

… notes into scripts, expressions through env) (#31013) * base red: release-notary, warm-sources entries for the nightly controls #30249 routed to the dedicated mini #30249 (d8f537d) let ten nightly.yml control jobs take the dedicated nightly mini (vars.CI_NIGHTLY_ONLY_RUNNER), so tests/test_ci_warm_sources.py counts them as Mac jobs, and they had no entry. Each takes mac-sign, the class of the other nightly.yml Mac jobs that compile nothing, with an exemption that says what the job does. cmux-tui-spec.yml:clippy (also unlisted; it wires Swatinem/rust-cache) takes linux-rust with actions-cache, as its cmux-tui-spec peers do. Red at 9288e32: python3 tests/test_ci_warm_sources.py FAILED (11 unlisted jobs) Green here: same command, OK. * base red: release-notary, test_nightly_universal_build follows the merged helper route and admission gate The test pins the full runs-on and if: of the nightly helper jobs. Two merged changes moved them: - #30249 (d8f537d) put the dedicated nightly mini (vars.CI_NIGHTLY_ONLY_RUNNER) first in build-nightly-ghostty-cli-helper's runs-on. The pinned expression now has that route; the merged workflow is the source of truth. - #30284 (4e81bfe) added the early admission conjunct to the resolver and the helper lookup. The test now names it (EARLY_ADMISSION) and checks that the resolver and the helper carry it as one conjunct after their gate, and that build-nightly-app keeps its own gate (it is skipped with the resolver it needs). The build_only exclusion is checked the same way as before. Red at 9288e32: bash ./tests/test_nightly_universal_build.sh FAIL: nightly must build and hand off the macOS 26 helper ... Green with the nightly.yml commit after this one: same command, PASS. * base red: release-notary, the Ghostty helper build takes its lookup's admission gate; nightly.yml back under its headroom tests/test_nightly_ghostty_helper_cache.py requires build-nightly-ghostty-cli-helper and resolve-nightly-ghostty-cli-helper to share one if:. #30284 (4e81bfe) added the early admission conjunct to the lookup on purpose; the helper build now has it too, so it never builds while its lookup waits for admission. tests/test_nightly_workflow_size.py: nightly.yml was 508,735 bytes, over the 508,000 headroom. The cache save steps repeated the same three R2 env values six times each (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, CI_CACHE_R2_ENDPOINT); each is now an anchor on its first use and an alias on the others, as &pub-feeds, &pub-dmg and &pub-tooling already are. yaml.safe_load gives the same workflow as before except the helper's if:. nightly.yml is 507,061 bytes. No step is removed and no limit changes. (Job-level anchors are refused by test_nightly_signer_jobs.py and runner_routing_table.py, and step aliases hide text that test_nightly_notices_submodule.py and test_nightly_promotion_admission.py count, so the anchors are on step env.) Red at 9288e32: python3 tests/test_nightly_ghostty_helper_cache.py FAILED (failures=1) python3 tests/test_nightly_workflow_size.py 508735 not less than 508000 Green here: both OK. All 123 tests that read nightly.yml: the same results as at 9288e32 except these and test_nightly_universal_build.sh, now green. * base red: release-notary, the promotion admission test reads the shared promote-guard checkout per job #30471 (e27f116) shares promote-nightly-next's "Checkout the promote guard from main" with promote-nightly-next-acked through the YAML anchor &legacy-route-shared-4, so `scripts/ci/nightly_next_train.py` appears once in the text and test_both_promote_decisions_pass_the_train_verifier (it counted the text) failed. It now finds the two jobs that call guard.trainVerifier in the parsed workflow and requires each to check out the verifier; it fails when either job's checkout drops it (checked by removing it from the anchor). Red at origin/feat-cmux-next 265c68a: python3 tests/test_nightly_promotion_admission.py AssertionError Green here: PASS (75 cases). * base red: release-notary, the Cloud environment editor highlight links its pull request #30357 (7d5f3ae) added release-notes/next/highlights/cloud-environment-editor.md with no try-it action, docs or pr link, so scripts/whats-new/test_digest.py failed. It links #30357, as #30624 did for three others. Red at 265c68a: python3 scripts/whats-new/test_digest.py FAILED (failures=1) Green here: OK. * base red: preflight, nightly.yml back under its 490,000-byte budget: 30 run blocks move to scripts/ci/nightly-steps/ tests/test_workflow_file_size.py (ci-guards preflight) caps nightly.yml at 490,000 bytes; it was 505,604 after #30824 (508,735 before it), red on every pull request. Lead decision 17:37Z: move run blocks, as #25879 did. 30 `run: |` blocks move verbatim into scripts/ci/nightly-steps/<name>.sh; each step now runs `bash scripts/ci/nightly-steps/<name>.sh` with the same env:, if:, id:, shell and working directory. yaml.safe_load gives the same jobs and steps as before except those 30 run strings, and tests/nightly_workflow_text.py puts the scripts back inline byte-identical to the old workflow. nightly.yml goes from 505,604 to 486,014 bytes. Which blocks: - Every block uses no ${{ }} expression and starts with `set -euo pipefail`, so the script sets the same shell options the inline block did (bash scripts/x.sh does not inherit GitHub's `bash -e`). - No block has a working-directory, runs inside a checkout it performs, or is a security gate whose text a test pins ("Hold the whole Mac before any credential"). - Each step's script is in the tree the step runs on: steps in refresh-compilation-cache, refresh-test-compilation-cache and resolve-nightly-cmux-tui-client check out the workflow commit (decide's head_sha is context.sha); steps in build-nightly-app, build-nightly-ghostty-cli-helper and build-nightly-remote-daemon check out build_sha and are skipped by an unsigned resume, the run that builds an older saved tree. Tests that read the raw nightly.yml text for a moved block now read it through tests/nightly_workflow_text.py (expand), as #25879 set up; what they check is unchanged. The new scripts pass shellcheck -S warning (the level actionlint gives shellcheck), and actionlint reports the same 5 findings on nightly.yml as before (unused anchors). Red at 6238504 (#30824 head): python3 tests/test_workflow_file_size.py 505604 not less than 490000 Green here: OK. All 124 tests that read nightly.yml: the same results as at 6238504 except test_workflow_file_size.py, now green. * ci(nightly): 14 more run blocks and the step notes move to scripts/ci/nightly-steps/; expressions reach scripts through env (headroom under the size budget) After #30912 nightly.yml had 3,986 bytes left under tests/test_workflow_file_size.py's 490,000-byte cap, so the next nightly.yml change would turn preflight red on every pull request again. Lead decision 13:2xZ: (a) the remaining safe blocks, (b) step notes into their script's header, (c) the `${{ }}` blocks with each expression passed through env. nightly.yml goes from 486,014 to 457,877 bytes (32,123 bytes of room). (a) 7 blocks with no ${{ }} move verbatim as in #30912. Four of them (Fetch the Chromium engine, Upload dSYMs to Sentry, Finish the dSYM upload to Sentry, Report a missing daemon attestation) do not start with `set -euo pipefail`; they ran under GitHub's default `bash -e {0}`, so they are called with `bash -e`, and tests/nightly_workflow_text.py accepts that call form. (c) 7 blocks used ${{ }} expressions. Each expression now sits in a new `env:` entry of the same step and the script reads that variable in the same place, GitHub's pattern against script injection. Each name is new to the workflow, read only by its script, and never assigned there. `--allow-non-descendant`, which was spliced in unquoted when non-empty and vanished when empty, is `${ALLOW_NON_DESCENDANT_FLAG:+"$ALLOW_NON_DESCENDANT_FLAG"}`: one word or none, as before. Republish deltas and revised appcasts stays inline: tests/test_nightly_republish_delta_guard.py runs its exact text, and its job is a resume job. (b) The comment lines above and inside 46 script-backed steps (only those whose script is called once) move into the header of their script, under "The step's notes, moved from nightly.yml". Comments of steps that keep inline code stay. yaml.safe_load of nightly.yml is unchanged by this part. Not moved: the run blocks of the 7 jobs an unsigned resume runs over an older build_sha (build-sign-notarize-nightly, prepare-nightly-variant, prepare-nightly-variant-primary, verify-signed-nightly, sign-release-notes, published-in, publish-nightly-reel). A resume of a source compiled before this change would not find a new script. They move 8 days after #30912 merged, when no unsigned copy of a pre-merge commit is left. Tests: 8 tests that read the inline text of a moved block read the expanded view (tests/nightly_workflow_text.py) or copy the script into their scratch tree; test_nightly_trusted_routing and test_ci_fleet_compilation_cache check the expression in the step's env and the variable in its script. What they check is unchanged. Red at the size budget: none at #30912's head (486,014 < 490,000); the gain is headroom. All 124 tests that read nightly.yml: the same results as at 17bf155 (all green). shellcheck -S warning: clean on scripts/ci/nightly-steps/*.sh. actionlint 1.7.12-manaflow.1: the same 5 findings on nightly.yml as before (unused anchors). * ci(nightly): Generate deltas and revised appcast stays inline (its .publish-tooling checkout lists single files) Review of #31013 at 8a2494b: generate-nightly-deltas checks out .publish-tooling with a non-cone sparse list of five named files (anchor &checkout-the-gh-token-wrapper), so scripts/ci/nightly-steps/deltas-generate-appcast.sh would not be in it and the step would fail on every run, hidden by the job's continue-on-error. My selection matched the directory name as a substring of the list. The step goes back exactly as on the feat tip (block and its notes), the script is removed, and test_nightly_sparkle_key.sh is back to the feat tip's text. An exact check of all 140 step-script calls (the last checkout before each call that provides its path, root or .publish-tooling, must contain the file: no sparse set, a cone-mode directory or a non-cone pattern) finds no gap at this head; it found exactly this one before the fix, and none in the feat tip's 127 calls. nightly.yml: 458,313 bytes (31,687 under 490,000). All 124 tests that read nightly.yml: the same results as at the feat tip 49aa291. shellcheck -S warning clean; actionlint: the same 5 findings as before. --------- Co-authored-by: cmux-bot[bot] <348531757+cmux-bot[bot]@users.noreply.github.com>

…nightly-steps (#31121) * test(nightly): the resolver refuses a published commit whose nightly step scripts differ from the tip's (red) Since #30912 the nightly runs step scripts from scripts/ci/nightly-steps/ in the resolved commit's checkout, under the tip's workflow. A published commit behind the tip whose step scripts differ would run older copies of steps the tip's workflow names. The case runs nightly.yml's own CMUX_TUI_TREE_SAME_PATHS policy. Red here: bash scripts/cmux-next/tests/pin-cmux-tui-resolve-newest.test.sh FAIL: nightly must not resolve a published revision with an older nightly step script * fix(nightly): the resolved build commit carries the tip's scripts/ci/nightly-steps resolve-nightly-cmux-tui-client's CMUX_TUI_TREE_SAME_PATHS gains scripts/ci/nightly-steps. pin-cmux-tui.sh compares `git rev-parse <rev>:<path>`, which for a directory is its tree, so a candidate whose step scripts differ in any file from the tip's is skipped with "scripts/ci/nightly-steps differs from the tip", as for nightly.yml and the Chief lockfile. Follow-up from the #31013 review. Cost: a nightly no longer builds a published commit behind the tip when the tip changed a step script after it; it takes a newer published commit, or builds the tip while its tree publishes (resolve-tree-pending.sh), as for a nightly.yml change today. Green here: bash scripts/cmux-next/tests/pin-cmux-tui-resolve-newest.test.sh PASS

lawrencecchen added 9 commits November 1, 2026 09:39

Merge remote-tracking branch 'origin/feat-cmux-next' into fix/base-re…

3f10db8

…d-nightly-notary

Merge remote-tracking branch 'origin/feat-cmux-next' into fix/base-re…

6238504

…d-nightly-notary

Merge remote-tracking branch 'origin/feat-cmux-next' into fix/base-re…

17bf155

…d-nightly-size

lawrencecchen added the risk:high Risk tier (docs/ci/risk-tiers.md) label Nov 1, 2026

lawrencecchen temporarily deployed to ci-routing-agent November 1, 2026 12:43 — with GitHub Actions Inactive

cubic-dev-ai Bot reviewed Nov 1, 2026

View reviewed changes

lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 1, 2026

Merge branch 'feat-cmux-next' into fix/base-red-nightly-size

2036240

cmux-bot Bot temporarily deployed to ci-routing-agent November 1, 2026 13:30 Inactive

lawrencecchen added ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds and removed ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds labels Nov 1, 2026

lawrencecchen merged commit fa65549 into feat-cmux-next Nov 1, 2026
193 of 203 checks passed

lawrencecchen deleted the fix/base-red-nightly-size branch November 1, 2026 13:56

lawrencecchen mentioned this pull request Nov 1, 2026

ci(nightly): headroom under the size budget (14 more run blocks, step notes into scripts, expressions through env) #31013

Merged

This was referenced Nov 1, 2026

fix(nightly): the resolved build commit carries the tip's scripts/ci/nightly-steps #31121

Merged

nightly runs scripts from the build commit's checkout, so older copies of scripts/ci/*.py and scripts/cmux-next/*.sh can run #31127

Open

Navigation Menu

Repository navigation

base red: nightly.yml under its 490,000-byte budget (30 run blocks move to scripts/ci/nightly-steps/) - #30912

base red: nightly.yml under its 490,000-byte budget (30 run blocks move to scripts/ci/nightly-steps/)#30912
lawrencecchen merged 10 commits into
feat-cmux-nextfrom
fix/base-red-nightly-size

lawrencecchen commented Nov 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 1, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

cubic-dev-ai Bot Nov 1, 2026

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

blacksmith-sh Bot commented Nov 1, 2026

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

lawrencecchen commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes

How the blocks were chosen (so behavior does not change)

Remaining exposure

Tests

Red and green

Observed / Not run

Changelog

Uh oh!

github-actions Bot commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot Nov 1, 2026

Choose a reason for hiding this comment

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

blacksmith-sh Bot commented Nov 1, 2026

Failures

Uh oh!

lawrencecchen commented Nov 1, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lawrencecchen commented Nov 1, 2026 •

edited

Loading

github-actions Bot commented Nov 1, 2026 •

edited

Loading