Skip to content

ci(nightly-next): the build picker may offer glaeda's nightly slot first - #26251

Merged
teamleaderleo merged 3 commits into
feat-cmux-nextfrom
fanout-infra/picker-nightly-slot
Oct 27, 2026
Merged

teamleaderleo merged 3 commits into
feat-cmux-nextfrom
fanout-infra/picker-nightly-slot

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-10-27: "BUT WTF COULD POSSIBLY BE MORE IMPORTANT ON OUR FUCKING MINIS". The standing order: a waiting nightly job is admitted on a mini ahead of every PR, DEV and side job.

Why

GitHub hands a queued job to the next idle runner in arrival order. So nightly-next's build-nightly-app, queued on the shared side label, waits behind every PR job queued there first. Run 40518198747 shows it: queued on glaeda-side-light-xcode-26.6 at 14:08Z, never acquired by a runner, moved to Blacksmith at 14:12Z, and still queued there at 14:28Z. At that time the std side label had 18 of 18 runners busy and the light side label 19 of 19.

glaeda#1684 adds a nightly slot: one runner per mini, labelled glaeda-nightly-std-xcode-26.6, that takes only nightly jobs. Its gate listens however full the mini is, and its job waits first in line on the mini, with every other runner there held.

What

owned_label() in nightly_build_runner.py also accepts glaeda-nightly-<class>-xcode-<version>. With vars.CI_NIGHTLY_NEXT_BUILD_RUNNER set to that label, the picker's order becomes:

  1. the dedicated builder;
  2. an idle nightly slot;
  3. the side label;
  4. Blacksmith.

Main's rescue watches the label through the main PR beside this one, which lands first.

Tests

tests/test_nightly_build_runner.py: 24 OK, including the new test_the_nightly_slot_comes_first_while_idle (an idle slot wins, a busy slot falls back to the side label, and near-miss labels are rejected).

Risk

Low. Nothing changes until the variable names the label. The variable is set only after the slots are registered and main's rescue watches the label.

This touches the same owned_label lines as #26237, the dispatch and re-run routing. The new test works with or without owned_route's attempt argument, so either PR can land first.

Agent session: agent session

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review b1e90af: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr26251 -f ref=fanout-infra/picker-nightly-slot -f platform=macos -f backend_mode=local

web and size checks: green

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-edqdlc created for this PR by the cmux coordinator.

@github-actions github-actions Bot added the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 27, 2026
@blacksmith-sh

blacksmith-sh Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
nightly.yml must derive the embedded key and sign every appcast through the helper/
nightly.yml must derive the embedded key and sign every appcast through the helper
View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)/
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo
teamleaderleo merged commit 3f8b07b into feat-cmux-next Oct 27, 2026
131 of 134 checks passed
@teamleaderleo
teamleaderleo deleted the fanout-infra/picker-nightly-slot branch October 27, 2026 16:13
@github-actions

Copy link
Copy Markdown
Contributor

Shipped in:

This branch was previously deployed

1 inactive deployment
ci-routing-agent — b1e90afa Deployed Oct 27, 2026 by teamleaderleo via wait for same-key tree publisher #65682
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant