[teamleaderleo]

The release publisher no longer loses a built nightly to a 10-minute burst of refused uploads. It backs off GitHub's secondary rate limit within a 30-minute budget, uploads one asset at a time, and logs exactly which limit refused it.

Why. Run 40298250989 attempt 3 (nightly-next 52d22f3) had already built, signed and verified the nightly when 18 upload POSTs were refused with "API rate limit exceeded for installation", between 12:04:51Z and 12:14Z. The refused responses had no retry-after and no x-ratelimit-* headers. At the same moment, the janitor's /rate_limit read with github.token showed core at 4899/5000 (12:05:11Z). So this is GitHub's secondary limit on write bursts, not the hourly budget: every workflow's GITHUB_TOKEN counts as one actor. The publisher waited a flat 60 s per refusal, hit its 600 s cap, and failed.

What changes:

• publish-release-assets.py:
  • It honors retry-after and x-ratelimit-reset when present. Otherwise it backs off 60, 120 and 240 s, then 300 s per wait, inside its budget.
  • The budget comes from CMUX_RELEASE_RATE_LIMIT_SECONDS, default 600. The nightly asset publish sets 1800, and publish-nightly's timeout goes from 90 to 120 min.
  • Uploads run one at a time instead of 2 in parallel, with a 1 s pause between writes.
  • On every refusal it logs all response headers plus a /rate_limit snapshot (that read is free), so the next red names the resource. Snapshot calls can't recurse.
  • A rate-limited refusal no longer uses up one of the 3 transport attempts; the budget bounds it instead.
• gh_budget_wait.sh (ci(nightly): publish-path gh calls wait through a rate limit instead of failing #25434): when no budget reads exhausted, it backs off the same way (60, 120, 240, 300 s) instead of a flat 60 s. It now makes up to 6 tries, still within the job's one-hour cap.

Ordering, verification, immutability and feed-last guarantees are unchanged. Tests:

• tests/test_github_release_publication.py: 40 pass. New ones cover the backoff sequence, header and snapshot logging, and the env budget. The parallel-peak test now expects 1.
• tests/test_gh_budget_wait.py: 16 pass, including the backoff sequence with a fake sleep.
• Also passing: the nightly resolve, coverage, concurrency, recovery and ring tests, and the universal-build script.

Risk: normal. Serial uploads make a publish a little slower (about a dozen assets, each a few seconds to a minute), in exchange for not tripping the secondary limit ourselves.

Agent session: agent session

🤖 Generated with Claude Code

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review 71bd413: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr25470 -f ref=gt-quota/publish-rate-limit-wait -f platform=macos -f backend_mode=local

[teamleaderleo]

Owned by an agent session · session

[teamleaderleo]

Bead cx-3vqxy created for this PR by the cmux coordinator.

[teamleaderleo]

cmux-next fast gates / post cmux-next/fast-gates is red on 12 of 16 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

[teamleaderleo]

high-signal review link / link is red on 10 of 16 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

[github-actions]

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4035126911001 (61ea34bde1)
• cmux NEXT nightly 1.0.0-nightly.4035798236101 (3660113fa5)