Repository navigation
ci(cmux-next): the DEV REF capture retries a rate-limited lookup and compiles instead of reusing - #25866
Merged
Conversation
…nd a reused product Red at cdc41df: the Have tests and the two new Workflow tests fail (no have subcommand, the lookup runs after the product lookup and a failed lookup sets no capture). The job-timeout assertion was already red on base (the job's timeout-minutes is an expression); it now reads its mini branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
…compiles instead of reusing DEV run 40428066457 (c28f2f1) took no capture: the decide step's gh api lookup hit "API rate limit exceeded for installation", exited 1 before it wrote capture, and continue-on-error kept the run green. A rerun would not have helped: the product was a reuse hit, so it would have compiled nothing. - hybrid_ref_capture.py have: the artifact lookup in Python, waiting out a rate limit (Retry-After or the reset) and backing off on server errors, for 600 s at most; then it fails with the last error. - The decide step runs before the product lookup, and a key with no capture skips reuse (SKIP_REASON), so that build really compiles and captures. - A lookup that still fails sets capture=true, prints an ::error:: and fails the step (continue-on-error), so the run shows it and the capture is taken. tests/test_cmux_next_hybrid_ref_capture.py: 18 of 18 pass (9 red at base). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
Collaborator
Author
|
Owned by an agent session · session |
Contributor
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
web and size checks: green |
Collaborator
Author
|
Bead |
…nce a day, not on every landing An incomplete capture was uploaded as hybrid-ref-capture-KEY-incomplete-RUNID, which no exact-name lookup finds, so with reuse skipped for a missing capture a key that never captures whole would have compiled on every landing. Its manifest is now hybrid-ref-capture-incomplete-KEY; have also counts those of the last day, and the decide step's new compile output (what SKIP_REASON reads) is true only when there is neither. A failed lookup still captures a build that compiles, without forcing one. tests: the decide step's shell runs on four have answers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 02:44 — with
GitHub Actions
Inactive
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 02:44 — with
GitHub Actions
Inactive
|
Found 1 test failure on Blacksmith runners: Failure
|
…t lost to a reused product Codex review of 012949b: with compile=false a failed lookup could still reuse the product, and a reuse builds nothing to capture (40428066457 again). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 03:11 — with
GitHub Actions
Inactive
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 03:12 — with
GitHub Actions
Inactive
teamleaderleo
temporarily deployed
to
artifacts
October 27, 2026 03:24 — with
GitHub Actions
Inactive
teamleaderleo
temporarily deployed
to
sentry-dsyms
October 27, 2026 03:24 — with
GitHub Actions
Inactive
…n guards The polling guard scans hybrid_ref_capture.py whole, so have's bounded rate-limit retry flags both steps that call it; the retention allowlist is keyed on the manifest's artifact name, which this PR changed. The manifest is small JSON kept 14 days as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
…ture-fix # Conflicts: # .github/workflows/cmux-next.yml
…lookup, and a truncated error body Codex review round 2 of e3297c9: - The capture step's own jobs read (for the 40-minute rule) has no retry; a rate limit still active after the lookup's 10 minutes made it skip the capture. It now falls back to the decide step's start, written before the lookup (that step runs early in the job). - An HTTPError whose body read fails (IncompleteRead) escaped the retry; it now counts as its status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a1351026211d556f5a03
…ture-fix # Conflicts: # tests/test_cmux_next_hybrid_ref_capture.py
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 03:47 — with
GitHub Actions
Inactive
teamleaderleo
temporarily deployed
to
ci-routing-agent
October 27, 2026 03:48 — with
GitHub Actions
Inactive
teamleaderleo
added a commit
that referenced
this pull request
Oct 27, 2026
…e.py have Takes #25866's lookup: have counts complete and recent incomplete captures, waits out a rate limit (30 s here, before the nightly compile) and exits 1 on a lookup it cannot answer. The nightly decide step compiles and captures anyway and fails red then, as the DEV one does. The artifact names follow the profile (hybrid-ref-capture-nightly-next-[incomplete-]KEY; DEV names unchanged), the upload writes the same names have reads, and the product lookup misses on compile=true. An unknown job start moves the capture's deadline earlier instead of skipping. The cleanup tests reap their own orphans (subreaper) and check the helper ended by SIGKILL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-f6b9335f650995bf635b
Contributor
|
Shipped in:
|
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
DEV run 40428066457 at c28f2f1 took no REF capture, so the fleet whole-app proof has no hybrid-ref-12 to build from. The capture-decision step's
gh apilookup hit "API rate limit exceeded for installation" at 02:11:03Z and exited 1 before it wrotecapture. Because the step is continue-on-error, the run stayed green. A rerun would not have fixed it: the product was a reuse hit, so the rerun would have compiled nothing.hybrid_ref_capture.py have: the artifact lookup, moved to Python. It waits out a rate limit (Retry-After orx-ratelimit-reset) and backs off on 5xx and network errors, for 600 s at most, then fails with the last error. A 403 that is not a rate limit fails at once.compileoutput, whichSKIP_REASONreads, so that build really compiles and captures. Most landings share a key that already has a capture and still reuse.hybrid-ref-capture-incomplete-KEY(it washybrid-ref-capture-KEY-incomplete-RUNID, which no exact-name lookup finds).havecounts the ones from the last day, and while one exists the key does not skip reuse. A key whose capture always comes out incomplete therefore forces at most one extra compile a day, not one on every landing. hq'shybrid-ref-run.shreads only complete manifests, so the rename doesn't affect it.capture=true compile=true, prints an::error::annotation and fails the step. The job is still not failed, so DEV publication is never held, but the run shows the failure and the build compiles and captures.github.token.Not touched: the nightly's capture in #25812, which has the same
gh apilookup and can switch tohaveafter this lands.Tests
python3 tests/test_cmux_next_hybrid_ref_capture.py: 9 red at base cdc41df (first commit); 20 of 20 pass at the head, eight runs in a row.test_the_decisionruns the decide step's real shell on fourhaveanswers.tests/test_dev_release_product_reuse.pypasses. One of the 9,test_off_by_default_and_never_in_the_way, was already red on base: the job'stimeout-minutesbecame an expression. It now reads the mini branch, 60. actionlint gives the same 4 findings as base, all pre-existing. prepush-gate is clean.Observed / Not run
Ran: the test file above (base and head), actionlint and prepush-gate. Read the 40428066457 DEV compile job log (job 121939218995) for the failure.
Not run: a real DEV daily with capture on. That is the next step: a DEV daily dispatched at the tip after this lands.
Changelog
none
🤖 Generated with Claude Code
Agent session: agent session