Repository navigation
ci(nightly): take the cmux-tui download and the light-job placement wait off build-nightly-app's tail - #32501
Merged
Conversation
…ait off build-nightly-app's tail build-nightly-app ended with two waits on the feed's critical path: - "Bundle the cmux-tui client" downloaded the resolved client and its companions after the compile: 40-121 s, median 50 s, of which 31-108 s was the download (16 nightly-next runs, 2026-11-02). The tree companions step now starts install-cmux-tui-client.sh in the background on a scratch bundle, beside the compile, into the cache the bundle step reads; the bundle step joins it and then installs from the cache, which it reuses only for slices whose sha256 matches the attested manifest. A failed prefetch is stopped and its cache dropped, so the install downloads as before. - "Place the light jobs on a free mini" waited up to 180 s for idle nightly-only runners as the job's last step: 3-188 s, mean 66 s. It now starts in the background before the archive and is collected at the end, so the wait overlaps the archive, the uploads and the browser-data proof. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7
Contributor
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Shipped in:
|
|
Found 6 test failures on Blacksmith runners: Failures
|
Collaborator
Author
This was referenced Nov 2, 2026
teamleaderleo
added a commit
that referenced
this pull request
Nov 2, 2026
…ghtly.yml back under its size cap (#32513) #32490, #32501 and #32498 together took nightly.yml to 493,218 bytes, past the 490,000-byte cap in tests/test_workflow_file_size.py. Move "Build and sign the release notes" and "Embed channel provisioning profile" into scripts/ci/nightly-steps/, the same way earlier steps were moved; the second runs under bash -e like its inline block did. The workflow expanded by tests/nightly_workflow_text.py is byte-identical to base. nightly.yml is now 485,819 bytes. Base-red-job: test_workflow_file_size (nightly.yml over its 490,000-byte cap) Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
teamleaderleo
added a commit
that referenced
this pull request
Nov 2, 2026
…ness PRs (#32526) - Validate pull request macOS pool choice: #32501 moved the light-job placement's portable_job_placement.py call into scripts/ci/nightly-steps/light-placement.sh, so the step that carries CI_RUNNER_STATE no longer names the picker in its run text. The leak check now also reads the nightly-steps script a step calls; a step whose script runs no picker still fails. - Validate supply-chain security boundaries: #32455's sentry-dsyms-nightly job takes SENTRY_AUTH_TOKEN with no environment. It now has the same environment expression build-nightly-app gave the in-job Sentry upload it replaced; no environment or its protection changes. - Validate test determinism gate: #32498's test asserted on elapsed wall time. It now asserts the failure names the stalled range, which shows the fetch ended on its own bound. Base-red-job: Validate pull request macOS pool choice Base-red-job: Validate supply-chain security boundaries Base-red-job: Validate test determinism gate Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
build-nightly-apptakes about 17 min, and the path from landing to the NEXT feed needs to fit in 30 min. After the Release compile, the job ends with two waits on the feed's critical path. This PR moves both off it. The compile itself is unchanged.What changes
install-cmux-tui-client.shin the background (background_build.py) on a scratch bundle while the app compiles. It writes into the sameCMUX_TUI_CLIENT_CACHEand uses the same manifest URL, expected commit and capability checks. The bundle step joins it and then runs its own install, which reuses a cached slice only if its sha256 matches the attested manifest. A failed or unfinished prefetch is stopped and its cache is deleted, so the install then downloads everything as it does today. The prefetch runs only when the bundle step would also run and the tree is already published. Its token goes into the prefetch command only, not into thepin-cmux-tui.shfetch in the same step.light-placement.sh start), so its wait runs during the archive, the uploads and the browser-data proof (about 100-230 s). "Collect the light-job placement" (id: place-light, still the last step before the always() cleanup) joins it and copies its outputs. The light jobs readsteps.place-lightexactly as before. If the placement fails, the light jobs take Blacksmith as before, and "Stop the helper builds still running" ends the placement if the job fails first.browser-data-proof-start.shto keep nightly.yml under its 490,000-byte cap.Performance
The before numbers come from run logs. The after numbers are estimates, not measurements.
Blacksmith minutes per day: unchanged. The light jobs use the same placement logic, and the placement now observes capacity about 2-3 min earlier.
Warm sources: none change. Both cuts remove waits after the compile. "Build nightly app (Release)" took 383-1304 s over the same runs, and its kept DerivedData and compilation cache work as before.
Tests
tests/test_nightly_trusted_routing.py: new tests runlight-placement.sh startandfinishagainst a stand-in placement. They check that the outputs and summary reach the collecting step and that a failed placement fails it. They also check the step order: the start step comes before the archive and the collect step is last before cleanup. A structural test checks that the prefetch uses the bundle step's cache, commit, manifest and capability checks, that it runs only where the bundle step runs, and that the join comes before the install and drops the cache on failure. 47 tests pass.test_ci_portable_job_placement.py(updated to the start step's id),test_nightly_step_coverage.py(new step registered),test_workflow_file_size.py(489,915 bytes),test_nightly_background_steps.py,test_ci_nightly_off_blacksmith.py,test_nightly_build_state.py,test_ci_resolve_cmux_tui_client_commit.sh,test_ci_self_hosted_guard.sh,test_nightly_delivery_concurrency.py,test_nightly_signer_jobs.py.test_nightly_universal_build.shfails on the current base with the same message ("the short nightly decide gate must use the pooled Linux route with a Blacksmith fallback"), so that failure is not from this PR.No started nightly run is cancelled by this change or while landing it.
Agent session: agent session
🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Takes two waits off
build-nightly-app's critical path so the path from landing to the NEXT feed fits its 30-minute window. The Release compile is unchanged; the estimated job median drops from about 1118 s to about 1020 s.Cmux-tui client prefetch
install-cmux-tui-client.shin the background on a scratch bundle while the app compiles, using the same cache, manifest URL, expected commit, and capability checks as the install.Light-job placement
Written for commit cbf707c. Summary will update on new commits.