Repository navigation
ci(nightly): the build job's tail uploads send 32 artifact blocks at a time, not 5 - #32343
Conversation
…a time, not 5 build-nightly-app ends with the unsigned-app upload (413 MB, 92-211 s) and the dSYM handoff (221 MB, 34-108 s); the signers and prepare wait on the job's end. upload-artifact sends 8 MB blocks 5 at a time by default, and these uploads ran at 2.1-6.4 MB/s, a block every 3-17 s. Raise ACTIONS_ARTIFACT_UPLOAD_CONCURRENCY to 32 on both uploads and the app retry, as ci-macos.yml's product upload already does. Same artifacts, names, compression, retention and step order; a test pins the setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Shipped in:
|
|
Review (subagent claude-opus-5-5 medium): SHIP at 10d74bb Checked: the pinned upload-artifact v7.0.1 reads ACTIONS_ARTIFACT_UPLOAD_CONCURRENCY (getConcurrency, default 5). 32 is in range, and ci-macos.yml uses the same value. The anchor gives exactly that env on all three steps, and no keys are lost. Nothing else changes. These tests pass: test_nightly_stop_if_publish_doomed (10), step_coverage, fork_runner_routing, universal_build, upload_artifact_retention. Follow-ups:
|
…me, not 5 (#32490) Signing waits on prepare-nightly-variant-primary's end, and its last step uploads the 346 MB prepared app. On a dedicated nightly Mac that upload took 65-156 s (median 85 s over 21 runs, 41455121554-41488515134) at upload-artifact's default 5 blocks at a time; the same Macs move the 413 MB unsigned app in a median 53 s at 32 since #32343. Reuse that step's ACTIONS_ARTIFACT_UPLOAD_CONCURRENCY anchor on the prepared-app upload, which the fallback leg (prepare-nightly-variant) shares by alias. Same artifact, name, compression and retention; a test pins it on both legs. Agent-Session: https://cmux.com/deeplink/agent?id=aid-f724bb1ff9295c18fee7 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The NEXT nightly's
build-nightly-appends with two serial uploads: the unsigned app (413 MB) and the dSYM handoff (221 MB).prepare-nightly-variant-primaryand the signers wait for that job to end.actions/upload-artifactsends 8 MB blocks 5 at a time by default. Across five recent runs these two uploads ran at only 2.1-6.4 MB/s, with a block finishing every 3-17 s.This PR sets
ACTIONS_ARTIFACT_UPLOAD_CONCURRENCY: "32"(one YAML anchor) on:Upload unsigned nightly appKeep the nightly-next dSYMs for publicationci-macos.yml's product upload already uses the same setting (#14375). The action caps the value at 16 per CPU.Nothing else changes: artifacts, names, paths, compression levels (app 0, dSYMs 6), retention (1 and 21 days), overwrite, step order, timeouts and the retry are all the same. No step was added or renamed, so
.github/nightly-step-coverage.jsonneeds no change. No secret, permission, environment or concurrency-group change. nightly.yml grows to 480,200 bytes, still under the 490,000 budget.Why not move the dSYMs off the job
The dSYMs exist only on the build Mac's disk. A separate job would have to download an artifact this job uploads first, so it saves nothing.
Running the dSYM upload in the background would mean handing the job's Actions runtime token to a run step. That changes how a secret is handled, so it is out of scope here.
The app upload already runs first, at
compression-level: 0, and nothing slow comes before it. That leaves the upload's block concurrency as the safe cut.Performance
Metric: from the start of
Upload unsigned nightly appto the start ofsigner-app(the build job's tail, plus prepare and the signer pick).How "after" is projected: the best throughput these builders reached at 5 blocks was 6.4 MB/s (the dSYM upload in 41444184160), so the uplink carries at least that. Each upload is taken at min(measured, size / 6.4 MB/s), which is about 65 s for the app and 35 s for the dSYMs. Everything else in the window is kept as measured.
This is a projection, and it is meant to be conservative. More blocks in flight can go faster than 6.4 MB/s if the uplink has room, and no slower than today if it does not. The first nightly after landing measures it from the steps' own timings.
Blacksmith minutes per day: unchanged. These steps run on the owned build Mac, and no job moved.
Risk
Low. This only changes how many 8 MB blocks the action holds in flight (up to about 256 MB of buffers on the build Mac); the uploaded bytes are the same. If 32 connections stalled, the existing 10-minute timeout and one retry still bound the app upload, and the dSYM step stays
continue-on-error. A revert is one commit.Tests
test_tail_uploads_send_blocks_in_parallelintests/test_nightly_stop_if_publish_doomed.py: fails 3/3 on base, passes at head.test_nightly_stop_if_publish_doomed,test_nightly_background_steps,test_ci_upload_artifact_retention,test_nightly_step_coverage,test_ci_fork_runner_routing,test_nightly_build_state,test_nightly_signer_jobs,test_nightly_delivery_concurrency(its functions called directly),test_nightly_workflow_size,test_nightly_unsigned_resume,test_nightly_helpers_beside_release_build,test_nightly_trusted_routing,test_nightly_build_runner.bash tests/test_nightly_universal_build.sh,bash tests/test_ci_self_hosted_guard.sh.SHELLCHECK_OPTS="-S warning"reports nothing on nightly.yml, the same as base.Agent session: agent session
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Speeds up the nightly build job's final artifact uploads by raising the artifact block concurrency from 5 to 32, matching
ci-macos.yml's product upload.The
build-nightly-appjob's unsigned app (413 MB) and dSYM (221 MB) uploads kept the signers waiting; at the default 5 concurrent 8 MB blocks they ran at 2.1-6.4 MB/s. SettingACTIONS_ARTIFACT_UPLOAD_CONCURRENCY: "32"on the app upload, its retry, and the dSYM handoff shortens the tail without changing what is uploaded: same bytes, artifact names, compression, retention, and step order. A new test pins the setting on all three steps.Written for commit 10d74bb. Summary will update on new commits.