Skip to content

Route light CI status jobs through portable placement - #34615

Merged
teamleaderleo merged 5 commits into
feat-cmux-nextfrom
codex/light-routing
Nov 4, 2026
Merged

teamleaderleo merged 5 commits into
feat-cmux-nextfrom
codex/light-routing

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Short CI status and resolver jobs waited 42–43 minutes behind hosted/paid overflow while owned minis were idle. These three affected workflows now ask portable placement for fleet Linux or a free owned mini before their existing overflow route. The bootstrap reads the trusted runner-state publication without minting a routing-App token or making a runner-list request; stale or missing state leaves the fallback intact.

Leo's request: “how's ci? we still have bigger wins we can get? what's the state of it all? more things we can skip/do better”.

The central runner table declares all affected portable jobs and the secretless bootstraps. It validates their read-only route and current-attempt outputs. Live Linux placement is bounded by its idle slots; nightly priority and queued work still reduce mini capacity. Optional placement failure cannot fail the web result rollup.

The placement bootstraps use hosted Ubuntu on attempt 1 and the reconciled Linux pool on retry; they never wait on a fixed owned label. Merge groups skip placement and retain the web status rollup. Set CI_PORTABLE_LIGHT_JOBS_OFF=1 to skip bootstrap scheduling and keep all six consumers on their previous fallback routes; consumers also fall back if placement fails.

Performance:

metric before after delta harness or run
Affected light jobs with owned-capacity placement before overflow 0/6 6/6 +6 same YAML scan on merge base and head: PR size/UI/split, fast-gates resolve/post, web-validation status
Runner/API reads by a secretless fresh-state placement 0 0 0 test_runner_state_bootstrap_does_not_fall_through_to_github_api

The live queue p95 and idle-mini-minute acceptance window is tracked by the fleet queue work; this PR does not claim that routing declarations alone prove the two-hour queue outcome. No compile is added, so existing incremental/shared compile caches are unaffected.

Validation: 127 placement/routing/web-rollup/fork-routing tests, the runner routing policy check, and the self-hosted runner guard passed in fleet verification: https://github.com/manaflow-ai/cmux/actions/runs/41768629618. After updating the workflow-shape assertions, all 97 fast-gate and PR base-only tests passed at head 56a898a: https://github.com/manaflow-ai/cmux/actions/runs/41772247878. The latter commit changes only those assertions; production code is identical to the first successful verification. The original CI merge also failed WorkflowShape.test_the_type_check_covers_every_webviews_test_file because bench/replayLogs.test.ts is outside the base tsconfig; the identical failure was reproduced on CI base 3c3b97d in https://github.com/manaflow-ai/cmux/actions/runs/41772333335. Actionlint and the prepush gate passed. Regression coverage includes skipped/failed placement, merge-group rollup, trusted checkout, and rejection of fixed bootstrap labels.

🤖 Generated with Codex

Agent session: agent session

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 07f94f3: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr34615 -f ref=codex/light-routing -f platform=macos -f backend_mode=local

Shipped in:

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 4, 2026
@cursor

cursor Bot commented Nov 4, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@blacksmith-sh

blacksmith-sh Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test_only_advisory_jobs_and_every_one_waits_on_it (main.WorkflowShapeTests.test_onl
y_advisory_jobs_and_every_one_waits_on_it)/
test_only_advisory_jobs_and_every_one_waits_on_it (main.WorkflowShapeTests.test_onl
y_advisory_jobs_and_every_one_waits_on_it)
View Logs
test_the_size_check_never_skips_on_a_base_only_update (main.WorkflowShapeTests.test
_the_size_check_never_skips_on_a_base_only_update)/
test_the_size_check_never_skips_on_a_base_only_update (main.WorkflowShapeTests.test
_the_size_check_never_skips_on_a_base_only_update)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent gpt-6.1-sol high): SHIP at 56a898a

Checked the full routing change and final workflow-shape test repair: placement attempts, failed or skipped picker fallback, fork routing, rollups, and the kill switch. No scoped correctness blocker found.

Visual verdict: unchanged; CI-only diff.

Follow-ups: none.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 56a898a

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. Each lands as an issue in this repo that links this receipt; the lane that opened the PR fixes it forward.

SHIP at 56a898a

Visual verdict: no images; Leo tries the build

Focused placement, routing, and web-validation tests passed (124 tests). No correctness regressions found.

  1. [P2] [follow-up] [category:ux] [finding:F1] [outcome:pending] No after screenshots were supplied (review/images/); visual verification is deferred.

Follow-ups:

  • [P2] [follow-up] [category:ux] [finding:F1] [outcome:pending] Provide after screenshots if visual validation is needed.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Finding outcome: comment-6622738114/F1 wrong This CI-only change alters no product UI; screenshots do not verify runner routing.

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 56a898a: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

teamleaderleo and others added 5 commits November 4, 2026 05:01
Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-11af0abc73e245862c06
Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-11af0abc73e245862c06
The fleet fork-routing check found the new bootstrap environment inputs lacked its explicit fork branch. Match the existing safe runner routes so fork bootstrap attempts place nothing on owned capacity.

Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-11af0abc73e245862c06
Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-11af0abc73e245862c06
Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-11af0abc73e245862c06
@cursor

cursor Bot commented Nov 4, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high): SHIP at 07f94f3

Visual verdict: unchanged by this CI-only diff; no product UI is affected.

Focused CI unit tests passed. Checked the rebased portable-placement workflow wiring, runner declarations, bootstrap permissions, current-attempt guards, fork precedence, Linux pool fallback and fail-open behavior. No correctness or regression findings.

Follow-ups: none

@teamleaderleo
teamleaderleo merged commit f1598ac into feat-cmux-next Nov 4, 2026
159 checks passed
@teamleaderleo
teamleaderleo deleted the codex/light-routing branch November 4, 2026 05:52

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 07f94f3d Deployed Nov 4, 2026 by teamleaderleo via cmux-tui tree preflight #109676
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant