[teamleaderleo]

Summary

The queue janitor lists in-flight runs one status at a time (status=queued, in_progress, pending) and stopped at the first page with fewer than 100 runs. GitHub's status-filtered run listing can return a short page before the last one. At 06:10Z on 2026-10-26, status=queued returned 99 runs on page 1 and 85 on page 2, against a total_count of 182. So each sweep dropped every queued run older than page 1:

• The pool snapshot undercounted the oldest waits. The 05:57Z macos-pool-load put blacksmith-12vcpu-macos-26's oldest wait at 24 minutes. Run 40259152940's fast tier had been queued there for 205 minutes by then. next_macos_cascade.py and pr_runner_pool.py route by those numbers, and the cascade's stall check reads that oldest wait.
• Old runs were never acted on. Category b never cancelled run 40259152940, even though ci(queue janitor): see cmux-next runs; move a cmux-next leg stuck on Blacksmith macOS #25328 (merged at 6f07da4) made the janitor list cmux-next runs and PR acpmux: a new chat starts at the default, not the last mode picked in a running chat #24021 had landed at 04:03Z. The 05:57Z sweep cancelled eight newer cmux-next stale runs and never saw this one. Its job finally got a Blacksmith runner at 06:08Z.

The fix: paging continues until a page is empty or the runs read reach that status's total_count. When there is no total_count, it falls back to the short-page rule. MAX_RUN_PAGES (3) still caps it, so a sweep costs at most one more request per status than before. owned_pool_rescue.GitHub.failed_runs() (status=failure) gets the same rule.

Risk: low. More runs are visible to the janitor, so each category may act on older runs it used to miss. Those runs were always in scope; the janitor just never saw them.

Testing

pytest -q tests/test_ci_queue_janitor.py tests/test_ci_owned_pool_rescue.py -k "InFlightRunListing or short_page" on cmux15 (Python 3.13):

• base: 2 failed (test_a_short_first_page_does_not_hide_the_older_queued_runs, test_failed_runs_reads_past_a_short_page_while_the_total_says_more).
• head: all pass. Across the janitor, rescue, starved-label and picker suites: 659 passed, 121 subtests passed.

Observed / Not run

• Observed: the page sizes and total_count above, from the live API. The 05:57Z snapshot. That sweep's log (eight stale-pr cancels, no mention of 40259152940). Both test runs.
• Not run: a live sweep with this code.

Changelog

none

Agent session: cosmic-gray

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

Summary by CodeRabbit

• Bug Fixes
  • CI run listings now continue through short pages when more results may be available. Duplicate runs are listed only once, and listings stop when a page is empty or the page limit is reached.
• Tests
  • Added coverage for pagination across short and empty pages, duplicate runs, page limits, and listings without a reported result count.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[teamleaderleo]

Owned by an agent session · session

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: manaflow-ai/cmux
• Branch: deputy-2/janitor-pages-past-short-page
• File: .coderabbit.yaml
• Commit: 2af326a
• Layers: Repository YAML
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: fc326c8a-cf02-46bb-979d-0da1ce81f288

Commits

Reviewing files that changed from the base of the PR and between 6f07da4 and 2af326a.

Files selected for processing: 4

• scripts/ci/owned_pool_rescue.py
• scripts/ci/queue_janitor.py
• tests/test_ci_owned_pool_rescue.py
• tests/test_ci_queue_janitor.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

Walkthrough

Two CI run listings now use the API’s integer total_count to continue pagination after short pages. Both stop when they reach the reported total or receive an empty page. Tests cover these conditions and the fallback behavior when total_count is unavailable.

Changes: CI run pagination

Layer / File(s)	Summary
Pagination and regression tests scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, tests/test_ci_owned_pool_rescue.py, tests/test_ci_queue_janitor.py	Both listings track results across pages and use an integer total_count to determine when to stop. The tests check short-page continuation, complete-page stopping, and fallback behavior when total_count is absent.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related issues

• manaflow-ai/cmux issue 14196: The issue concerns CI queue latency and the queue janitor. This change updates the janitor’s pagination so it can continue past a short page when more runs are reported.

Suggested reviewers: lawrencecchen

Merge Risk: Minimal · up to 2af32

The pagination change is mergeable after normal checks; no specific blocking failure is identified.

Pre-merge checks: 24 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 24 passed

Check name	Status	Explanation
Title check		The title clearly and concisely describes the primary change: continuing queue-janitor pagination to the reported total count instead of stopping at the first short page.
Description check		The description provides a detailed problem statement, resulting behavior, risk, testing results, and observed/not-run evidence. It omits the template's Proof and Checklist sections, but the core revi…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input		PASS: The pull request changes only CI run-list pagination in scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and related tests. The diff introduces no Cloud terminal creation, cmux-…
Cmux Swift Actor Isolation		The pull request changes only Python production files and Python test files. The reviewed diff contains no Swift or Swift interface files, so it introduces no Swift 6 actor isolation issue.
Cmux Swift Blocking Runtime		PASS: The pull request changes only Python files and Python tests. The diff contains no production Swift files and introduces no Swift semaphores, blocking waits, sleeps, delayed dispatch, polling, ma…
Cmux Browser Automation Off-Main		PASS. The pull request changes only Python CI listing logic and its tests: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and two Python test files. The diff contains no browser.*…
Cmux Expensive Synchronous Load		PASS: The pull request changes only Python CI scripts and Python tests. The diff contains no Swift changes and no agent-history load, main-actor, or interactive-path code. The custom check does not ap…
Cmux Cache Substitution Correctness		PASS: The pull request changes only Python files under scripts/ci and tests. The diff contains no production Swift, TypeScript, or JavaScript changes, so the cache-substitution correctness condition d…
Cmux No Hacky Sleeps		PASS: The pull request changes pagination only in the two production Python CI scripts. It adds seen/total_count handling and no fixed sleeps, timers, polling waits, delayed dispatch, or wall-cloc…
Cmux Algorithmic Complexity		PASS. The production changes add bounded pagination, not a nested collection scan. queue_janitor.py processes each returned run once across three fixed statuses and at most MAX_RUN_PAGES = 3 pages…
Cmux Swift Concurrency		PASS: The pull request changes only Python CI scripts and Python tests. It introduces no Swift code and therefore no Swift concurrency patterns covered by this check.
Cmux Swift @Concurrent		The reviewed diff changes only four Python files: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and two Python test files. It contains no Swift changes, so the @concurrent Swift …
Cmux Swift Package Boundaries		PASS: The pull request changes only Python files under scripts/ci and tests. It introduces no production Swift code or SwiftPM target-boundary changes, so the Swift package boundary check does not…
Cmux Swiftpm Lockfiles		PASS: The pull request changes only Python CI scripts and Python tests. It does not change a SwiftPM package, Xcode project, .gitignore, workflow, or dependency pins. The SwiftPM lockfile rules ther…
Cmux Swift Logging		The pull request changes only Python files and tests. The authoritative diff contains no Swift files and no changed Swift logging statements, so the Swift logging rule does not apply.
Cmux User-Facing Error Privacy		PASS. The diff changes pagination logic and adds regression tests. It does not add or alter a cmux app UI, product CLI, or product API error, alert, command output, or recovery message. The changed fu…
Cmux Full Internationalization		PASS: The PR changes only CI pagination logic, comments/docstrings, and tests. It adds no Swift UI text, app catalog or Info.plist entries, web UI/API copy, rendered markdown, changelog text, or local…
Cmux Swiftui State Layout		The pull request changes only four Python files: two CI scripts and two Python test files. The reviewed diff contains no Swift or SwiftUI changes, so the SwiftUI state-layout criteria do not apply.
Cmux Architecture Rethink		The pull request changes only Python CI scripts and Python tests. No Swift architecture change is present, so the Swift architectural-rethink failure conditions do not apply.
Cmux Swift Auxiliary Window Close Shortcuts		PASS: The pull request changes only Python scripts and Python tests. It adds no Swift code or standalone cmux-owned window, so the auxiliary-window close-shortcut rule does not apply.
Cmux Source Artifacts		All four changed paths are intentional Python source and test files: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, tests/test_ci_owned_pool_rescue.py, and `tests/test_ci_queue_ja…
Cmux No Test Or Debug Seam In Production Source		The pull request changes only Python files under scripts/ci and tests/. It changes no Swift file under a production Sources/ path, so the specified production Swift test/debug seam condition does not …

Full details: Description check

Explanation

The description provides a detailed problem statement, resulting behavior, risk, testing results, and observed/not-run evidence. It omits the template's Proof and Checklist sections, but the core review information is complete and the change is CI-only.

Full details: Cmux Cloud Persistent Session And Early Input

Explanation

PASS: The pull request changes only CI run-list pagination in scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and related tests. The diff introduces no Cloud terminal creation, cmux-tui transport, manual renderer, input-routing, snapshot, auth, revision, or lease behavior covered by the rule.

Full details: Cmux Swift Blocking Runtime

Explanation

PASS: The pull request changes only Python files and Python tests. The diff contains no production Swift files and introduces no Swift semaphores, blocking waits, sleeps, delayed dispatch, polling, main-queue sync, or manual locks.

Full details: Cmux Browser Automation Off-Main

Explanation

PASS. The pull request changes only Python CI listing logic and its tests: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and two Python test files. The diff contains no browser.* socket commands, WebKit/AppKit access, socket-worker routing, or browser policy tests. The browser automation rule is therefore not applicable.

Full details: Cmux Expensive Synchronous Load

Explanation

PASS: The pull request changes only Python CI scripts and Python tests. The diff contains no Swift changes and no agent-history load, main-actor, or interactive-path code. The custom check does not apply.

Full details: Cmux Cache Substitution Correctness

Explanation

PASS: The pull request changes only Python files under scripts/ci and tests. The diff contains no production Swift, TypeScript, or JavaScript changes, so the cache-substitution correctness condition does not apply.

Full details: Cmux No Hacky Sleeps

Explanation

PASS: The pull request changes pagination only in the two production Python CI scripts. It adds seen/total_count handling and no fixed sleeps, timers, polling waits, delayed dispatch, or wall-clock synchronization. The added tests also contain no timing scaffolding.

Full details: Cmux Algorithmic Complexity

Explanation

PASS. The production changes add bounded pagination, not a nested collection scan. queue_janitor.py processes each returned run once across three fixed statuses and at most MAX_RUN_PAGES = 3 pages per status. owned_pool_rescue.py similarly processes each page once with the caller’s documented SWEEP_LISTING_PAGES = 5 bound. The diff adds no repeated sorting, filtering, rescanning, or in-memory join over the run collection.

Full details: Cmux Swift `@Concurrent`

Explanation

The reviewed diff changes only four Python files: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, and two Python test files. It contains no Swift changes, so the @concurrent Swift check is not applicable.

Full details: Cmux Swift Package Boundaries

Explanation

PASS: The pull request changes only Python files under scripts/ci and tests. It introduces no production Swift code or SwiftPM target-boundary changes, so the Swift package boundary check does not apply.

Full details: Cmux Swiftpm Lockfiles

Explanation

PASS: The pull request changes only Python CI scripts and Python tests. It does not change a SwiftPM package, Xcode project, .gitignore, workflow, or dependency pins. The SwiftPM lockfile rules therefore do not apply.

Full details: Cmux User-Facing Error Privacy

Explanation

PASS. The diff changes pagination logic and adds regression tests. It does not add or alter a cmux app UI, product CLI, or product API error, alert, command output, or recovery message. The changed functions run only in scheduled/manual GitHub Actions workflows and feed internal queue snapshots and operator step summaries. The added provider names, run IDs, and diagnostic details are in an internal CI docstring/comment or operational path, which the rule allows.

Full details: Cmux Full Internationalization

Explanation

PASS: The PR changes only CI pagination logic, comments/docstrings, and tests. It adds no Swift UI text, app catalog or Info.plist entries, web UI/API copy, rendered markdown, changelog text, or locale-dependent data. The added production prose is developer-only documentation, which the rule allows.

Full details: Cmux Source Artifacts

Explanation

All four changed paths are intentional Python source and test files: scripts/ci/owned_pool_rescue.py, scripts/ci/queue_janitor.py, tests/test_ci_owned_pool_rescue.py, and tests/test_ci_queue_janitor.py. The diff adds paging logic and regression tests. It adds no logs, screenshots, caches, build output, scratch directories, dependency checkouts, or other source-control artifacts.

Full details: Cmux No Test Or Debug Seam In Production Source

Explanation

The pull request changes only Python files under scripts/ci and tests/. It changes no Swift file under a production Sources/ path, so the specified production Swift test/debug seam condition does not apply.

Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Merge receipt for 98b7a502cf: every check was green at merge (10 verified; 16 skipped by policy). Full suite runs on main after merge.