Repository navigation
ci: refresh cmuxTests shard timings - #13068
Conversation
The manifest covered 522 suites. The test target now has 1142, so over a quarter of selectors were packed at the 200 ms default. Regenerated with scripts/ci/generate_test_timings.py from six green shard logs: 931 suites and 1447 methods, 2458 of 2790 selectors measured. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 15 minutes. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
All contributors have signed the CLA ✍️ ✅ |
 |
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
286b523 docs(agents): run the compile-only check in the tag's derived data (manaflow-ai#13033) bd980f6 ci: refresh cmuxTests shard timings from run 35427062807 (manaflow-ai#13068)
2 participants
Summary
Regenerates
scripts/ci/cmux-unit-test-timings.jsonwithscripts/ci/generate_test_timings.pyfrom the six app-host shard logs of https://github.com/manaflow-ai/cmux/actions/runs/35427062807. Data only, no planner change.The old manifest covered 522 suites, the target now has 1142. Measured selectors go from 2070 to 2458 of 2790, so 388 fewer selectors are packed at the 200 ms default.
Expected effect
Small. Replaying the old plan against the new measurements puts the heaviest shard at 1.19x the mean, the new plan at 1.0x. That is about 3 minutes off the slowest shard's test step.
The large shard 6 outlier (47 min median) is mostly not planner weight. In the source run shard 6 spent 1250 s in
Run unit testswith under 200 s of measured test time: four app-host crash restarts (#9348) and threeRemoteTmuxMirrorPaneInputMappingTestscases that each sat about 5 minutes. Shard 6 also carries about 400 s of pinned regression steps the planner does not see. Those are separate follow-ups.Testing
python tests/test_ci_cmux_unit_test_shard.pypassespython3 scripts/ci/cmux_unit_test_shard.py --validate: 2458/2790 measured🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Refreshes the CI shard timings manifest with measurements from the latest green shard logs so sharding follows the current test suite and the slowest shard runs less.
The regenerated manifest now has measured timings for 2458 of 2790 selectors, up from 2070, so fewer tests fall back to the 200 ms default. Replaying the new plan puts the heaviest shard at 1.0x the mean instead of 1.19x, saving about 3 minutes on the slowest shard's test step. This is data only; the planner logic is unchanged. Both
tests/test_ci_cmux_unit_test_shard.pyandscripts/ci/cmux_unit_test_shard.py --validatepass. The known shard 6 outlier is separate, caused by crash restarts and pinned regression steps rather than planner weight.Written for commit 443f1c6. Summary will update on new commits.