Repository navigation
ci(nightly): prefer dedicated mini for variant preparation - #30287
teamleaderleo wants to merge 6 commits into
Conversation
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
There was a problem hiding this comment.
2 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/test_nightly_dedicated_variant.py">
<violation number="1" location="tests/test_nightly_dedicated_variant.py:1">
P2: This new test file is not registered in `tests/test-execution.toml`, so the registry validation fails for this PR. No workflow runs it either, so these assertions would never execute in CI. Add a `[[test]]` entry for `tests/test_nightly_dedicated_variant.py` and wire it into a job that runs, like `test_nightly_trusted_routing.py` in `ci-guards.yml`.</violation>
</file>
<file name=".github/workflows/nightly.yml">
<violation number="1" location=".github/workflows/nightly.yml:3847">
P2: With the reservation enabled, the fallback `prepare-nightly-variant` also routes to the single mini after a successful primary. There it is a no-op that still queues behind other variants' primary legs, which reintroduces the queued hop the job comment warns about. Gate the reservation on the primary having failed, so a successful primary keeps the existing fallbacks.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| @@ -0,0 +1,76 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
P2: This new test file is not registered in tests/test-execution.toml, so the registry validation fails for this PR. No workflow runs it either, so these assertions would never execute in CI. Add a [[test]] entry for tests/test_nightly_dedicated_variant.py and wire it into a job that runs, like test_nightly_trusted_routing.py in ci-guards.yml.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At tests/test_nightly_dedicated_variant.py, line 1:
<comment>This new test file is not registered in `tests/test-execution.toml`, so the registry validation fails for this PR. No workflow runs it either, so these assertions would never execute in CI. Add a `[[test]]` entry for `tests/test_nightly_dedicated_variant.py` and wire it into a job that runs, like `test_nightly_trusted_routing.py` in `ci-guards.yml`.</comment>
<file context>
@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""The dedicated nightly-only mini is opt-in for the two preparation legs."""
+
</file context>
| runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-32vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }} | ||
| # Keep the exact reservation contract from the primary leg. It is unset | ||
| # until the dedicated mini's rollout and hook registration complete. | ||
| runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || github.repository == 'manaflow-ai/cmux' && needs.decide.outputs.track == 'nightly-next' && github.run_attempt == 1 && vars.CI_NIGHTLY_ONLY_RUNNER == 'glaeda-nightly-only-m4pro' && vars.CI_NIGHTLY_ONLY_RUNNER || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-32vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }} |
There was a problem hiding this comment.
P2: With the reservation enabled, the fallback prepare-nightly-variant also routes to the single mini after a successful primary. There it is a no-op that still queues behind other variants' primary legs, which reintroduces the queued hop the job comment warns about. Gate the reservation on the primary having failed, so a successful primary keeps the existing fallbacks.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/nightly.yml, line 3847:
<comment>With the reservation enabled, the fallback `prepare-nightly-variant` also routes to the single mini after a successful primary. There it is a no-op that still queues behind other variants' primary legs, which reintroduces the queued hop the job comment warns about. Gate the reservation on the primary having failed, so a successful primary keeps the existing fallbacks.</comment>
<file context>
@@ -3838,7 +3842,9 @@ jobs:
- runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-32vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }}
+ # Keep the exact reservation contract from the primary leg. It is unset
+ # until the dedicated mini's rollout and hook registration complete.
+ runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || github.repository == 'manaflow-ai/cmux' && needs.decide.outputs.track == 'nightly-next' && github.run_attempt == 1 && vars.CI_NIGHTLY_ONLY_RUNNER == 'glaeda-nightly-only-m4pro' && vars.CI_NIGHTLY_ONLY_RUNNER || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-32vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }}
timeout-minutes: 100
strategy:
</file context>
| runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || github.repository == 'manaflow-ai/cmux' && needs.decide.outputs.track == 'nightly-next' && github.run_attempt == 1 && vars.CI_NIGHTLY_ONLY_RUNNER == 'glaeda-nightly-only-m4pro' && vars.CI_NIGHTLY_ONLY_RUNNER || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-32vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }} | |
| runs-on: ${{ github.repository_owner != 'manaflow-ai' && 'ubuntu-24.04' || needs.prepare-nightly-variant-primary.result != 'success' && github.repository == 'manaflow-ai/cmux' && needs.decide.outputs.track == 'nightly-next' && github.run_attempt == 1 && vars.CI_NIGHTLY_ONLY_RUNNER == 'glaeda-nightly-only-m4pro' && vars.CI_NIGHTLY_ONLY_RUNNER || needs.build-nightly-app.outputs.light_attempt == format('{0}', github.run_attempt) && contains(needs.build-nightly-app.outputs.light_mini_jobs, ' prepare-nightly-variant ') && needs.build-nightly-app.outputs.light_mini_runner || needs.prepare-nightly-variant-primary.result == 'success' && (vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-32vcpu-ubuntu-2404') || vars.LINUX_RUNNER == 'blacksmith-16vcpu-ubuntu-2404' && 'blacksmith-16vcpu-ubuntu-2404' || 'blacksmith-16vcpu-ubuntu-2404' }} |
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
Review (codex gpt-6-astra high, review-box): SHIP at 28d33c8 No findings. CI-only diff; 161 targeted tests passed. Broader guard failures match the merge base or stem from omitted certificate fixtures. |
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="scripts/ci/runner_label_policy.py">
<violation number="1" location="scripts/ci/runner_label_policy.py:419">
P3: This trims the value before enforcing the exact selector, so a whitespace-padded label passes the variable policy while both jobs’ raw equality checks fall through to existing routes. Pass the untrimmed value here so non-exact configuration is reported.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| elif name == NIGHTLY_NEXT_BUILD_VARIABLE: | ||
| reason = nightly_next_build_reason(value.strip()) | ||
| elif name == NIGHTLY_ONLY_VARIABLE: | ||
| reason = nightly_only_reason(value.strip()) |
There was a problem hiding this comment.
P3: This trims the value before enforcing the exact selector, so a whitespace-padded label passes the variable policy while both jobs’ raw equality checks fall through to existing routes. Pass the untrimmed value here so non-exact configuration is reported.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At scripts/ci/runner_label_policy.py, line 419:
<comment>This trims the value before enforcing the exact selector, so a whitespace-padded label passes the variable policy while both jobs’ raw equality checks fall through to existing routes. Pass the untrimmed value here so non-exact configuration is reported.</comment>
<file context>
@@ -399,6 +415,8 @@ def drifted_runner_variables(
elif name == NIGHTLY_NEXT_BUILD_VARIABLE:
reason = nightly_next_build_reason(value.strip())
+ elif name == NIGHTLY_ONLY_VARIABLE:
+ reason = nightly_only_reason(value.strip())
elif name == FAST_LANE_VARIABLE:
reason = fast_lane_reason(value.strip())
</file context>
| reason = nightly_only_reason(value.strip()) | |
| reason = nightly_only_reason(value) |
|
Owned by an agent session · session |
|
Merge conflicts with feat-cmux-next, not resolved: base already landed most of this plumbing (#31266, #31291 and earlier). CI_NIGHTLY_ONLY_RUNNER is already in ci-health-report.yml and ci-repo-variables.yml, runner_label_policy.py already validates it (so the new nightly_only_reason branch would be dead code), and test_ci_self_hosted_guard.sh already defines nightly_only_selector (redefining it would change the guard). Only the prepare-nightly-variant(-primary) routing to the dedicated mini is new, so this needs a small rework on current base rather than a merge. |
|
Orphaned: no activity for 13 h, and no lane holds a lease on it. The orphan sweep could not merge |
|
Closing as superseded; nothing left to rework onto feat-cmux-next.
|
Leo said: “BUT WE HAVE A DEDICATED F***ING MAC FOR THAT.”
This stacked follow-up gives only
prepare-nightly-variant-primaryandprepare-nightly-variantan opt-in preference for the dedicated M4 Pro mini. The selector is deliberately exact: repositorymanaflow-ai/cmux, NEXT track, attempt 1, andCI_NIGHTLY_ONLY_RUNNER == glaeda-nightly-only-m4pro. When the variable is unset (the current state until rollout and hook registration), existing light-placement and Linux overflow routes are unchanged. Forks, retries, other tracks, and non-matching values preserve their existing fallbacks.This PR is stacked on #30113 at
f70b3ddbddcefb359e6484276b977c3c47e51517; it does not modify #30113. After #30113 lands, rebase this single dedicated-routing commit onto the newfeat-cmux-nexttip before the pinned merge.Validation:
python3 tests/test_nightly_dedicated_variant.py(4 passed)python3 tests/test_nightly_trusted_routing.py(34 passed)python3 tests/test_ci_portable_job_placement.py(30 passed)python3 tests/test_ci_nightly_off_blacksmith.pypassedgit diff --checkpassedAgent session: https://cmux.com/deeplink/agent?id=aid-cd7a5a0ee7f7c79a3232
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Makes
prepare-nightly-variant-primaryandprepare-nightly-variantprefer a dedicated M4 Pro mini runner during NEXT-track nightly preparation. The routing is opt-in: it only applies on attempt 1 formanaflow-ai/cmuxwhenCI_NIGHTLY_ONLY_RUNNERis set toglaeda-nightly-only-m4pro; all other forks, retries, tracks, and unset values keep the existing light-placement and Linux overflow fallbacks.Adds tests covering the reservation selector, including assertion that only the two preparation jobs read it and that unset or non-matching contexts keep existing fallbacks.
CI_NIGHTLY_ONLY_RUNNERin the runner label policy: only the exactglaeda-nightly-only-m4prolabel is accepted, and the variable counts as a no-hosted-Mac variable.Written for commit 28d33c8. Summary will update on new commits.