Repository navigation
ci(nightly): the Mac signer jobs read the signed app and final DMG in parallel ranges - #32498
Conversation
… parallel ranges download-artifact reads an artifact's blob in one stream with no stall timeout. On the nightly's Macs the 355 MB signed app took a p50 of 133 s in signer-dmg-image (hosted Linux reads it in 9 s) and ran out its 5-minute limit in 4 of the last 26 nightly-next runs; in run 41478567483 both tries ran out and that tip never published. verify-final-dmg-nightly's final-DMG download took a p50 of 89 s and up to 446 s (run 41481821056). scripts/ci/fetch_run_artifact.py reads the blob in 16 parallel ranges, asks again for a range that sends nothing for 20 s from where it stopped with a fresh signed URL, checks the zip against its upload digest and unpacks it like download-artifact. Both jobs run it first and keep download-artifact as the fallback when it fails. The two jobs gain actions: read to list this run's artifacts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Shipped in:
|
…ce, with no new permission Drop the actions: read grants. fetch_run_artifact.py now asks the artifact service download-artifact itself uses (twirp ListArtifacts and GetSignedArtifactURL at ACTIONS_RESULTS_URL) with the job's runtime token, reading the run and job backend ids from the token's Actions.Results scope. A run step does not see that token, so a pinned actions/github-script step spawns the script and hands it its own environment; nothing goes to $GITHUB_ENV except the SIGNER_APP_FETCHED flag. The signed URL is the same Azure blob SAS URL, which takes Range requests. download-artifact stays the fallback in both jobs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7
|
Found 8 test failures on Blacksmith runners: Failures
|
|
Review (subagent claude-opus-5-5 medium): SHIP at 8d67fd0 The runtime token is sent only to ACTIONS_RESULTS_URL, and nothing is written to $GITHUB_ENV except SIGNER_APP_FETCHED. The signed URL must be https, zip-slip is blocked, artifact names reach the script only through env, download-artifact runs after every failure, and every publish gate is unchanged. 51 tests pass. Follow-ups:
|
…ghtly.yml back under its size cap (#32513) #32490, #32501 and #32498 together took nightly.yml to 493,218 bytes, past the 490,000-byte cap in tests/test_workflow_file_size.py. Move "Build and sign the release notes" and "Embed channel provisioning profile" into scripts/ci/nightly-steps/, the same way earlier steps were moved; the second runs under bash -e like its inline block did. The workflow expanded by tests/nightly_workflow_text.py is byte-identical to base. nightly.yml is now 485,819 bytes. Base-red-job: test_workflow_file_size (nightly.yml over its 490,000-byte cap) Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ness PRs (#32526) - Validate pull request macOS pool choice: #32501 moved the light-job placement's portable_job_placement.py call into scripts/ci/nightly-steps/light-placement.sh, so the step that carries CI_RUNNER_STATE no longer names the picker in its run text. The leak check now also reads the nightly-steps script a step calls; a step whose script runs no picker still fails. - Validate supply-chain security boundaries: #32455's sentry-dsyms-nightly job takes SENTRY_AUTH_TOKEN with no environment. It now has the same environment expression build-nightly-app gave the in-job Sentry upload it replaced; no environment or its protection changes. - Validate test determinism gate: #32498's test asserted on elapsed wall time. It now asserts the failure names the stalled range, which shows the fetch ended on its own bound. Base-red-job: Validate pull request macOS pool choice Base-red-job: Validate supply-chain security boundaries Base-red-job: Validate test determinism gate Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Why
The NEXT nightly's path from landing to feed should fit in 30 minutes and takes about 45. Two of the losses are
actions/download-artifacton the nightly's Mac runners. It reads the artifact's blob in one stream with no stall timeout.publish-nightlyskipped it as superseded. That tip never reached the feed. The same artifact downloads in a p50 of 9 s (max 98 s, n=51) insigner-app-notaryandsigner-dmgon hosted Linux.From a Linux VM, the same 257 MB blob (its signed URL taken through the REST redirect, which points at the same blob host) took 30.0 s in one stream, 4.6 s in 8 ranges and 2.9 s in 16 ranges, with the digest matching every time. The limit is per connection.
What
scripts/ci/fetch_run_artifact.py(stdlib only):ListArtifactsandGetSignedArtifactURLatACTIONS_RESULTS_URL) with the job's runtime token. It reads the run and job backend ids from the token'sActions.Resultsscope. NoGITHUB_TOKENpermission is needed, and no job's permissions change.actions/github-scriptstep (v9.0.0, the pin used elsewhere in the repo) starts the script and hands it its own environment. Nothing is written to$GITHUB_ENVexcept theSIGNER_APP_FETCHEDflag. The token goes to the artifact service only, never to the blob host.*.blob.core.windows.net), and that host answers Range requests with 206.continue-on-error, 5 min). The existing download-artifact step and its retry run only when the ranged step did not setSIGNER_APP_FETCHED. It is an env flag rather than a step outcome becausesigner-dmgreuses those two anchored steps, has no ranged step, and so keeps its current route.steps.fetch-final-dmgdid not succeed.signer-dmg, and verify has to read those exact bytes on a Mac. The image is already smoked in parallel (launch-smoke-nightly,smoked_in_job), so the transfer is the only part left to cut.Every gate stays: both notarizations, the static verify, Apple's checks, and the launch smoke that publish needs. The digest checks downstream (
APP_SHA256onsigned-app.tar, the tested DMG digest) are unchanged.Performance
The "after" column is an estimate: about 6 times the single-stream rate, from the Linux measurement. The first nightly-next runs after this lands will give the real numbers, which the ranged step prints (MB, seconds, MB/s, range retries). Blacksmith minutes are unchanged: both jobs run on owned Macs.
Tests
python3 tests/test_fetch_run_artifact.py(new, added to ci-guards' "Validate the release signer scripts"). Against a fake API and blob host it covers: a ranged read and unpack; a stalled range resuming with a fresh URL; a digest mismatch leaving nothing behind; a range that never finishes failing within the budget; and a zip member that escapes the destination being refused.python3 tests/test_nightly_signer_jobs.py(new wiring test: ranged step first, fallback conditions, permissions, signer-dmg unchanged),tests/test_nightly_step_coverage.py(two new coverage entries),tests/test_nightly_sign_runner.py,tests/test_nightly_next_launch_smoke.py,tests/test_nightly_trusted_routing.py,tests/test_nightly_dedicated_mini.py,tests/test_nightly_next_post_publish.py,tests/test_ci_self_hosted_guard.sh,tests/test_ci_upload_artifact_retention.py: all pass.nightly.ymlandci-guards.yml.tests/test_supplychain_security.pyfails the same way on base (sentry-dsyms-nightly), so it is not this PR's red.Agent session: agent session
🤖 Generated with Claude Code