Skip to content

ci: the routing table fails a macOS job pinned to one fixed label - #28600

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
dep-fleet-3/ios-kit-picker
Oct 31, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
dep-fleet-3/ios-kit-picker

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-10-28: "Bro, I thought we already had logic to automatically route stuff. How can we still have pins like this? I hate pins. Like, just pin the set of the stuff that isn't hilariously slow."

cmux-next-ios-kit.yml's kit-tests runs on
... || vars.CI_PAID_MACOS_OVERFLOW == '1' && vars.MACOS_RUNNER_26 || 'blacksmith-6vcpu-macos-26': one fixed Blacksmith macOS label, with no owned Mac first and no stall-aware failover. None of the runs-on checks caught it:

  • tests/test_ci_runs_on_fallback.py refuses only single-host labels (a glaeda-runner-* label or a mini's name) and the 12 vCPU Blacksmith Mac as a last resort.
  • scripts/ci/runs_on_values.py refuses only runner-name variables that are read bare.
  • scripts/ci/runner_routing_table.py counted only a bare vars.CI_SIDE_LANE_RUNNER / CI_LIGHT_LANE_RUNNER as bypassing placement.

A Blacksmith 6 vCPU macOS literal is none of those, so a job that moved off the owned pool onto a fixed Blacksmith label passed every check.

Change. runner_routing_table.check() now also fails a macOS job whose runs-on (plus its matrix, when it reads one) reads no picker output. A picker output is a needs.<job>.outputs.<x> other than the static capability map, or a caller's inputs.*. Each failure names the pickers to use: ios_runner_pool.py, simple_pool_picker.py or side_lane_placement.py.

The 42 macOS jobs pinned this way today each get a picker_followup in .github/runners.json. Like linux_followup and placement_followup, a followup that no longer applies fails, so the list only shrinks. The release-signing jobs are marked as needing a signing picker (per-job keychain, exclusive claim). A placement bypass already has its placement_followup and is not counted twice.

Red/green.

New fixture tests cover:

  • the exact kit-tests expression;
  • a bare literal, a variable plus a literal, and the capability map;
  • a literal matrix against a picked matrix;
  • a picker output and a caller's input;
  • a stale followup;
  • no double count with a placement bypass.

Validation: tests/test_runner_routing_table.py, test_ci_runner_capability_resolver.py, test_runs_on_labels.py, test_ci_no_github_hosted_macos.py and test_feat_ci_guard_group.py were run locally. test_ci_change_areas.py has one failure (test_repository_helpers_route_by_where_they_run), which is the same on base. No workflow file changes.

No capture: CI routing check only.

Agent session: agent session

🤖 Generated with Claude Code

cmux-next-ios-kit.yml's kit-tests ran on
`... || vars.CI_PAID_MACOS_OVERFLOW == '1' && vars.MACOS_RUNNER_26 || 'blacksmith-6vcpu-macos-26'`
and no runs-on check caught it. test_ci_runs_on_fallback.py only refuses
single-host labels (a glaeda-runner-*, a mini's name) and the 12 vCPU Blacksmith
Mac. runs_on_values.py only refuses runner-name variables. runner_routing_table.py
counted only a bare vars.CI_SIDE_LANE_RUNNER / CI_LIGHT_LANE_RUNNER as bypassing
placement. A Blacksmith 6 vCPU macOS literal is none of those, so moving a job
off the owned pool onto a fixed Blacksmith label passed every check.

runner_routing_table.check() now also fails a macOS job whose runs-on (with
its matrix, when it reads one) reads no picker output: no needs.<job>.outputs.<x>
other than the static capability map, and no caller's inputs.*. The 42 macOS
jobs pinned this way today get a picker_followup entry in .github/runners.json.
Like linux_followup and placement_followup, a followup that no longer applies
fails, so the list only shrinks. Release-signing jobs are marked as needing a
signing picker (per-job keychain, exclusive claim). kit-tests gets no entry:
the test stays red until #28145 routes it through ios_runner_pool.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review e7fde0e: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28600 -f ref=dep-fleet-3/ios-kit-picker -f platform=macos -f backend_mode=local

Shipped in:

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 30, 2026
@blacksmith-sh

blacksmith-sh Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Found 5 test failures on Blacksmith runners:

Failures

Test View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/
test_download_prints_the_active_profile_only
View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/
Security has no stub
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/
CoreFoundation, missing 1: ['_CFRetain']
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/
CoreFoundation, missing 1: ['_log2']
View Logs
test_the_ten_first_party_gpl_crates_are_named_exactly (main.AppCrateLicenseGate.tes
t_the_ten_first_party_gpl_crates_are_named_exactly)/
test_the_ten_first_party_gpl_crates_are_named_exactly (main.AppCrateLicenseGate.tes
t_the_ten_first_party_gpl_crates_are_named_exactly)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

The check catches kit-tests' fixed Blacksmith label, which is the point of this
change. Until #28145 moves the job onto ios_runner_pool.py, it carries a
followup like the other 42 pinned jobs, so the guard can land now and refuse
new pins. When #28145 lands, the stale-followup rule makes it drop this line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
Review round 2: matching a picker output anywhere in runs-on or the whole
strategy let a fixed label pass. An unrelated matrix field that read an output
was enough, and so was an output used only as a condition. Now only value
positions count: the last && operand of each || alternative, read through
groups and fromJSON(...). For matrix.<key>, only that key's axis and include
values count. The tighter read catches one more pin,
cmux-next-perf-budgets.yml#blacksmith-mac. It compares inputs.runner_class and
then always names a Blacksmith label, so it gets a picker_followup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
Review round 3: an expression-valued include could vouch for a fixed runner
axis. GitHub keeps the axis's own legs whatever an include adds, so each leg's
value now has to come from a picker: the axis items, and the include rows that
set the key. An opaque include counts only when nothing else gives the key a
value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
Review round 4: the every-leg rule also asked a picker of the Linux and Windows
legs in a mixed matrix. A literal leg that names a Linux or Windows runner and
no macOS hint is left out. The macOS legs must still all be picked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
Review round 5: `fromJSON(needs.pick.outputs.runners)['label']` was taken for a
literal. A value term now counts as picked when it reads a picker output
anywhere outside a quoted string, unless it is a boolean: a negation, a
comparison, or contains/startsWith/endsWith.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cabecb079fe356666905
@teamleaderleo
teamleaderleo merged commit fe23e80 into feat-cmux-next Oct 31, 2026
146 of 150 checks passed
@teamleaderleo
teamleaderleo deleted the dep-fleet-3/ios-kit-picker branch October 31, 2026 01:31
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Follow-ups from the independent review (Codex gpt-6-astra high, SHIP at eda4401). Neither one blocks:

  1. A condition inside an indexed call is still read as a picker. In fromJSON(needs.p.outputs.ok == 'true' && '["macos-26"]' || '["macos-15"]')[0] the output appears only in a condition, so the value is a literal label, yet the check counts it as picked. Fix: apply the value-position analysis beneath indexing too.
  2. Bracket property access gives false failures. needs.pick.outputs['runner'] and matrix['runner'] read the same values as the dot forms, but the check reports them as fixed labels.

No workflow in the tree uses either shape today.

This branch was previously deployed

1 inactive deployment
ci-routing-agent — eda44019 Deployed Oct 31, 2026 by teamleaderleo via cmux-tui tree preflight #86107
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant