[teamleaderleo]

Follow-up to #27202. The four react-apps-check shards took webview test files by index mod 4, so adding one test file moved every file after it, and the heavy dev-server files fell together by chance (the #27045 case #27194 describes). On feat-cmux-next push runs the shards were consistently 2x apart. The slowest took about 20 min and the fastest about 9.

What changes

• Split by recorded duration. New scripts/ci/webviews_test_shards.py. Files with a recorded time go, longest first, to the shard whose estimated wall time grows least. A shard is modelled the way run-webviews-tests.sh runs it: plain files on 32 lanes, one warming file per Vite config in series, the other dev-server files on 8 lanes, then // cmux-web-test-alone files one after another. Files with no recorded time go by a stable hash of their path, after the others, so a new file moves only itself. The split never reads the runner's own size, so the four shard jobs agree even when one lands on the 16 vCPU fallback.
• Order inside a shard. With a manifest, each Vite config warms on its shortest file, and both pools start their longest files first. accent-glass-invariants (549 s) came first in the index and warmed its config alone for 9 of shard 3's 18.6 min (run 40815704587).
• The manifest.
  • web-subarea-scope restores the newest manifest from the Actions cache once and hands that copy to all four shards as an artifact, so they split by the same copy. A shard that read the cache itself could get a newer copy than its siblings and skip or repeat files.
  • Each shard records each file's first-run seconds (CMUX_WEB_TEST_DURATIONS_OUT; a retry runs alone and says nothing about the file among the others).
  • A new job, webviews-test-durations, averages them with the prior manifest and saves the next one. Nothing requires it, so a stalled download or cache save can't time out react-apps-check; a missing manifest only means the next split is by path hash.
  • A pull request's cache entry stays in its own ref's scope, and fork pull requests save none. feat-cmux-next push runs keep the shared manifest fresh.

Performance

metric	before	after	delta	harness or run link
slowest shard, test step (min)	18.6	9.8	-8.8 (-47%)	run 40815704587's per-file times through the runner's phases
fastest shard, test step (min)	7.7	8.9	+1.2	same
slowest / fastest shard	2.41	1.10	-1.31	same

Before: the replay of the measured per-file times reproduces today's shards (18.6, 11.2, 9.5, 7.7 min against 18.6, 12.1, 9.9, 7.7 measured), so it models the runner's phases closely. After: the same times through the new split and order. This PR's first run starts from an empty manifest (path-hash split). The manifest it saves feeds the next run, so a measured "after" comes from a run that reads a manifest. The measured slowest shard job before is 19.6 min (run 40815704587, 32 vCPU). I'll add the measured after from the first run that reads a manifest, on the same 32 vCPU class.

Red / green

• python3 tests/test_webviews_tests_per_file.py: the 5 new runner tests fail on the base runner and pass on this head: the load split, a new file moving only itself, the hash split for files with no recorded time, the warming pick and pool order, the recorded seconds. There are also new helper tests (merge, record, the lane weighting) and a workflow test (one manifest from web-subarea-scope, merged and saved by react-apps-check, never saved by forks). 36/36 pass.
• tests/test_webviews_tests_new_reds.py: the test of the base replay re-running every red file now counts the files its shard ran, since the shard is no longer index mod 2. 23/23 pass.

Observed / Not run

• Ran on a Linux host, on this head: tests/test_webviews_tests_per_file.py (35 OK), tests/test_webviews_tests_new_reds.py (23 OK), tests/test_web_validation.py, tests/test_ci_change_areas.py, tests/test_test_reds.py, scripts/ci/tests/run-webviews-tests-retry.test.sh, actionlint on ci-web.yml, scripts/cmux-next/prepush-gate.py (no new violations).
• Not run yet: a real shard run with a manifest. This PR's CI is the first run without one; its successor is the first with one.

Risk: CI only (risk:low). Every shard reads the same manifest artifact or fails its download, so shards cannot disagree on the split. A stale or missing manifest only makes the split less even, never incomplete: every file in the git index lands on exactly one shard.

Changelog

none

🤖 Generated with Claude Code

Agent session: agent session

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review cc63e5a: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr27230 -f ref=ci/webviews-shards-by-duration -f platform=macos -f backend_mode=local

Yours to fix: next-docs.test.ts (this PR changes the web test runner (.github/workflows/ci-web.yml)), docs-shortcuts.test.ts (this PR changes the web test runner (.github/workflows/ci-web.yml)). Also failing here, not on main: rowSummary.test.tsx, rowSummary.test.tsx.

CI failed on cc63e5ab5f (run 40824110742 attempt 1): 3 code, 1 unknown.

Job	Verdict	Why
web / Web tests (4/4)	code	a web test failed
web / Web tests (3/4)	code	a web test failed
web / react-apps-base-check (1)	code	a web test failed
web / react-apps-check	unknown	no known signature; failed step: Check webview test shards

Failures:

• yours next-docs.test.ts cmux-next docs > the settings reference is fresh against the settings schema: failed (job)
• yours docs-shortcuts.test.ts keyboard shortcuts docs > the web/data copy is fresh against the catalog export and string catalogs: failed (job)
• new in this PR rowSummary.test.tsx no description line on any Settings page is over 60 characters, in any language: failed (job)
• new in this PR rowSummary.test.tsx every schema summary fits 60 characters in every language: failed (job)

Matched log lines

web / Web tests (4/4): (fail) cmux-next docs > the settings reference is fresh against the settings schema [3.82ms]
web / Web tests (3/4): (fail) keyboard shortcuts docs > the web/data copy is fresh against the catalog export and string catalogs [16.71ms]
web / react-apps-base-check (1): (fail) no description line on any Settings page is over 60 characters, in any language [13185.38ms]

Not re-run automatically: web / Web tests (4/4), web / Web tests (3/4), web / react-apps-base-check (1), web / react-apps-check are not machine failures.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on 9af05f0 to compare; do not merge until green; see lane-rules
red: web / Web status; base red: the same failure on the base 9af05f0; does not block
red: web / Web tests (3/4); no finished base run on 9af05f0 to compare; do not merge until green; see lane-rules
red: web / Web tests (4/4); no finished base run on 9af05f0 to compare; do not merge until green; see lane-rules
red: web / react-apps-base-check (1); no finished base run on 9af05f0 to compare; do not merge until green; see lane-rules
red: web / react-apps-check; base red: the same failure on the base 9af05f0; does not block

[blacksmith-sh]

Found 8 test failures on Blacksmith runners:

Failures

Test	View Logs
/tmp/tmp9a42z2v8/summary-missing.json (live):/summary-missing.json (live):	View Logs
/tmp/tmpug80fk01/summary-fail.json (live):/summary-fail.json (live):	View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/ [new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows	View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames	View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/ [tabs] debug.tab_switch timed 0 of 40 switches	View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames	View Logs
src/pages/settings/rowSummary.test.tsx/rowSummary.test.tsx	View Logs
src/pages/settings/rowSummary.test.tsx/rowSummary.test.tsx	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[teamleaderleo]

Bead cx-6y9x0c created for this PR by the cmux coordinator.

[github-actions]

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4083321330101 (ad7f29e3da)
• cmux NEXT nightly 1.0.0-nightly.4083436704201 (ba6bbf43e2)
• cmux NEXT nightly 1.0.0-nightly.4083583746201 (ba3b38d3b6)

[teamleaderleo]

Measured after landing: react-apps-check shard job wall times on feat-cmux-next push runs, all on the 32 vCPU class. The "after" runs all read the 1,196-file duration manifest.

Performance

metric	before	after	delta	harness or run link
slowest shard job, median (min)	19.7	17.0	-2.7 (-14%)	before: 8 push runs ending 40815704587; after: 14 push runs 40844104395..40846198563
fastest shard job, median (min)	9.0	13.2	+4.2	same
slowest / fastest shard, median	2.24	1.31	-0.93	same
runs with all shards within 20%	0 of 8	4 of 14	+4	best: 40844350052 at 1.10 (16.0, 15.1, 14.5, 15.1 min)

The split is more even, but the slowest shard fell far less than the replay predicted (9.8 min). There are two causes, and both read the same way in 40846198563 (max/min 1.93):

• Heavy plain files contend when packed together. One shard got the 41 heaviest plain files. They took 5.3 min, against the ~2 min their recorded times predicted, because each file's time was recorded beside light neighbours. The lane model treats the 32 plain lanes as free of CPU contention, and they aren't.
• Base-red retries are not modelled. The base was red during these runs, so shards spent up to 6 min retrying failed files (14 retries on shard 2). Those minutes land on whichever shard holds the red files.

Follow-up, not in this PR: weigh plain files by contention (balance each shard's total recorded seconds as well as its lane time), and count the expected retry time of files that are red on the base.