Skip to content

ci(e2e): refuse a focused batch whose concurrency group exceeds 400 characters - #14104

Merged
teamleaderleo merged 3 commits into
mainfrom
ci/e2e-concurrency-group-length
Sep 24, 2026
Merged

teamleaderleo merged 3 commits into
mainfrom
ci/e2e-concurrency-group-length

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Why

test-e2e.yml builds its concurrency group from the runner, the ref and the whole test_filter:

e2e-<runner>-<40-hex ref>-<test_filter>

A batch of four long Swift Testing selectors produced a 520-character group, and run 35939770225 was created with no jobs. GitHub reported only "This run likely failed because of a workflow file issue". run-e2e.sh printed the run URL as if the dispatch had worked.

GitHub does not document the limit. Recent dispatches show where it falls:

group length (approx.) run jobs
372 35930869627 4
393 35935925123 3
437 35938283242 0
462 35930690423 0
520 35939770225 0

So the cutoff is between 393 and 437. This PR uses 400.

Change

dispatch-focused-test.py computes the group the workflow will build and refuses the dispatch before any runner is spent. The error names the length and says to split the selectors or select the whole suite. When the pool can't be resolved, it measures against the longest known runner label.

Verification

  • A new launcher test (902047d is the test and 11c1768 the fix): four long selectors are refused and nothing is dispatched; two of the same selectors still dispatch as one batch.
  • tests/test_run_e2e.py: 52 pass.
  • All 123 guard commands listed in ci-guards.yml pass locally.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Refuses focused test dispatches whose concurrency group would exceed 400 characters, preventing the silent broken runs GitHub creates for oversized groups.

  • Previously such a batch dispatched and started with zero jobs and no explanation; now dispatch-focused-test.py measures the group length first and errors with a hint to split selectors or run the whole suite.
  • When the runner pool can't be resolved, the length check uses the longest known runner label.
  • The launcher test uses three selectors whose filter alone is under 400 but whose whole group exceeds it, and pins the workflow's group expression so the dispatcher's copy cannot drift silently.

Written for commit 5985754. Summary will update on new commits.

Review in cubic

teamleaderleo and others added 2 commits September 23, 2026 19:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…haracters

test-e2e.yml groups on runner, ref and the whole test_filter. GitHub
rejects a group over 400 characters as a workflow file issue, so a batch
of four long Swift Testing selectors started with no jobs and no reason
(run 35939770225, 520 characters). Refuse it in the dispatcher, before a
runner is spent, and say to split it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 45e77995-2c18-4117-bac9-2c3cffc43a13

📥 Commits

Reviewing files that changed from the base of the PR and between 229edc5 and 5985754.

📒 Files selected for processing (2)
  • scripts/ci/dispatch-focused-test.py
  • tests/test_run_e2e.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Three selectors keep the filter under 400 while the group exceeds it, so
a check on the filter alone would fail the test. Assert the workflow's
group expression so the copy in the dispatcher cannot drift silently.
Drop the repeated default_runner() call from the fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

An independent review subagent found nothing blocking at 11c1768. It confirmed:

  • the computed group matches test-e2e.yml on every path the dispatcher sends (explicit, auto, routed large runner, unknown default);
  • the check runs before any dispatch or history lookup, and --force does not skip it;
  • the test fails without the fix.

It also found two more zero-job runs that narrow the limit: 35834658722 (group 407) and 35830735643 (group 426). The cutoff is therefore between 394 and 407.

Pushed 5985754 for its low-severity points:

  • The refused case now uses three selectors. The filter is 377 characters and the group is 448, so a check on the filter alone would now fail the test. Before, anything between 324 and 575 passed.
  • The test asserts the workflow's concurrency.group expression verbatim, so the dispatcher's copy of it cannot drift silently.
  • The unknown-runner fallback no longer calls default_runner() a second time, which could add a 30 s timeout.

tests/test_run_e2e.py: 52 pass.

— Ibex g1 🌿

@teamleaderleo
teamleaderleo enabled auto-merge (squash) September 24, 2026 02:21
@teamleaderleo
teamleaderleo merged commit 623019c into main Sep 24, 2026
40 checks passed
teamleaderleo added a commit that referenced this pull request Sep 24, 2026
Keeps both sides of dispatch-focused-test.py: main's runner-pool overflow and
400-character concurrency-group check (#14067, #14104, #14132), and this
branch's per-definition dispatch history and selector normalization.
test-depot.yml is now test-macos-suite.yml (#14075); the UI filter guard
followed the rename and its test reads the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant