Skip to content

ci: harden fast-checks admission credentials and deadline - #30762

Merged
teamleaderleo merged 1 commit into
feat-cmux-nextfrom
lane/fast-checks-runtime-followups
Nov 1, 2026
Merged

teamleaderleo merged 1 commit into
feat-cmux-nextfrom
lane/fast-checks-runtime-followups

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

The fast-checks admission picker now keeps routing authority and snapshot authority separate and enforces one 180-second deadline across the complete runner census. Missing ROUTE_TOKEN immediately uses measured Blacksmith overflow; the repository GITHUB_TOKEN is never accepted as routing authority. When a snapshot is requested, its own SNAPSHOT_TOKEN client reads it. Every API and artifact request receives the remaining monotonic deadline, including multi-page census reads and retries.

This is a fix-forward for the three P2 follow-ups in #30648: #30743, #30744 and #30745. The optional GitHub(deadline=...) argument is backward-compatible for existing clients. #30724 also edits pr_runner_pool.py for cmux-tui selector guards; this branch changes only the shared client timeout plumbing and fast-checks caller.

Validation: python3 -m unittest tests.test_ci_fast_checks_runner -q (11 passed), python3 -m py_compile scripts/ci/fast_checks_runner.py scripts/ci/pr_runner_pool.py, and git diff --check passed. The existing full test_ci_pr_runner_pool suite was dispatched but exceeded the lane tool window; no full-suite pass is claimed here.

Review receipt: #30648 (comment)

Changelog

none

Agent session: agent session


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Hardens the fast-checks admission picker so routing authority and snapshot authority stay separate, and one 180-second deadline bounds the entire runner census.

GITHUB_TOKEN is no longer accepted for fleet census reads; a missing ROUTE_TOKEN immediately falls back to the measured Blacksmith overflow. Snapshot reads use their own SNAPSHOT_TOKEN client. Every API request, retry wait, and artifact download is capped by the remaining monotonic deadline. The new optional GitHub(deadline=...) argument is backward-compatible for existing callers.

Written for commit cc95bff. Summary will update on new commits.

View guided diff Turn on auto-fix

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review cc95bff: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr30762 -f ref=lane/fast-checks-runtime-followups -f platform=macos -f backend_mode=local

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (claude opus-5-5 medium, subagent): SHIP at cc95bff

No blocking findings. All three P2 follow-ups from #30648 are fixed, and existing callers of the GitHub client in pr_runner_pool.py behave as before when they pass no deadline.

  1. Snapshot token (scripts/ci/fast_checks_runner.py:82-85): the snapshot is read through its own GitHub(snapshot_token, repo, deadline=...) client, so the routing token no longer authenticates it, and the snapshot token goes only to the snapshot request. Covered by test_read_live_uses_snapshot_token_for_snapshot_client.
  2. Routing credential (fast_checks_runner.py:72, :100): read_live and the guard in choose read only ROUTE_TOKEN. With it missing, choose goes straight to overflow without reading the census. Covered by test_github_token_alone_does_not_admit_fleet.
  3. Deadline (fast_checks_runner.py:102-104; scripts/ci/pr_runner_pool.py:2157-2182, 2193, 2203-2206, 2227, 2234): one deadline goes into each read. GitHub._timeout caps each API and artifact request at the time remaining and raises TimeoutError once it has passed; the retry path also refuses a wait that would run past it. read_live's existing except catches the error and falls back. The worst case after the deadline is about one sleep of at most 15s.
  4. Existing callers: deadline is keyword-only and defaults to None. With None, _timeout(d) returns d unchanged, and the retry-path check sits under if self.deadline is not None. I checked every GitHub(...) construction under scripts/ (about 25 sites); all are positional or pass only retry_cap/retries, so none changes behaviour. base_red_excuse.py:63 subclasses a different GitHub class (_evidence.GitHub), and the mocks in test_ci_next_macos_cascade replace the whole class.

Findings:

  • [follow-up] scripts/ci/fast_checks_runner.py: no workflow under .github/ calls this picker yet. Whichever workflow adopts it must pass ROUTE_TOKEN (and SNAPSHOT_TOKEN if a fresh snapshot is wanted); with only GITHUB_TOKEN it always gets the Blacksmith overflow, which is intended.
  • [follow-up] scripts/ci/pr_runner_pool.py:2176-2182: _timeout and the retry-path check each work out the time remaining in their own code; a shared helper would keep them from drifting apart. Cosmetic.

Follow-ups:

  • When fast_checks_runner.py is wired into a workflow, have it pass ROUTE_TOKEN (and SNAPSHOT_TOKEN if wanted).
  • Optional: merge the two remaining-time calculations into one helper.

Tested (extracted copies of head and base, no checkout):

  • tests/test_ci_fast_checks_runner.py at head: 11 OK, including the 4 new tests.
  • tests/test_ci_pr_runner_pool.py at head with .github extracted: 242 OK. Without .github, head and base fail the same 31 tests by name (missing workflow files), so no regression.
  • tests/test_ci_side_lane_placement.py at head: OK (callers that pass retry_cap/retries).
  • tests/test_ci_portable_job_placement.py at head ran without a traceback; its summary was not captured.

@teamleaderleo
teamleaderleo merged commit 265c68a into feat-cmux-next Nov 1, 2026
86 of 101 checks passed
@teamleaderleo
teamleaderleo deleted the lane/fast-checks-runtime-followups branch November 1, 2026 10:39
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 1, 2026
@blacksmith-sh

blacksmith-sh Bot commented Nov 1, 2026

Copy link
Copy Markdown

Found 4 test failures on Blacksmith runners:

Failures

Test View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/
test_download_prints_the_active_profile_only
View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/
Security has no stub
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/
CoreFoundation, missing 1: ['_CFRetain']
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/
CoreFoundation, missing 1: ['_log2']
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

teamleaderleo added a commit that referenced this pull request Nov 4, 2026
…#34428)

Run 41718897738's admit step spent 182 s polling for an idle fleet runner (helper ecc5d60, #30648). 159a701
(#34270) decides on one census read: an online fleet runner keeps the fleet label (the lane-gh broker bounds a
checks job left queued there at 180 s), none overflows at once. The pin also carries #30762, #30822, #31896 and
#33652, each already reviewed and merged on feat-cmux-next.


Agent-Session: https://cmux.com/deeplink/agent?id=aid-342ae944382c55b76f4c

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
ci-routing-agent — cc95bffc Deployed Nov 1, 2026 by teamleaderleo via cmux-tui tree preflight #95726
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant