Skip to content

perf(test): parallelize unit test files to halve validate wall time - #849

Merged
kody-bot merged 3 commits into
mainfrom
cursor/faster-validate-911e
Jul 22, 2026
Merged

kody-bot merged 3 commits into
mainfrom
cursor/faster-validate-911e

Conversation

@kentcdodds

@kentcdodds kentcdodds commented Jul 22, 2026 •

Copy link
Copy Markdown
Owner

Problem

npm run validate in CI takes ~5m20s, and the whole job ~6.5min. Per-check completion times inside a recent run (29892110417) show the unit test suite is the entire critical path:

Check Completion
primitives, migrations ~2s
format, lint ~9s
typecheck 40s
MCP e2e 1m36s
Playwright e2e 3m15s
unit tests 5m20s

The vitest summary explains why: 327 files, 310s total, of which 180s is module import time and only 62s is actual test time — all serialized by fileParallelism: false in vitest-shared.ts (present since the original projects migration with no stated rationale).

Change

  • Remove fileParallelism: false from the shared vitest project config, so node-unit (289 files) and workers-unit (37 files) run files across all cores. In CI, cap vitest at 3 workers so the concurrent Wrangler/Playwright startups are not starved on 4-core runners.
  • Keep mcp-e2e explicitly serial with a comment: each file boots a real Wrangler dev server against the shared seeded e2e database.
  • Raise the Playwright webServer startup budget to 180s in CI: with unit tests using all cores, the e2e server (client build + D1 migrations + Wrangler boot) exceeded the default 60s on the first CI attempt.
  • Fix an nx cache-correctness gap found while measuring: worker:test / worker:test-mcp only hashed project files, so edits to the root vitest*.ts configs or tools/ (vite plugins + root-level test files the runner picks up) returned stale cache hits. Both targets now include those paths in their inputs.

Measurements

  • Unit suite alone (4-core VM, cold nx cache): 196s → 82s.
  • Full npm run validate locally with CI=true, cold cache: 1m54s, all 8 checks green.
  • Real CI (run 29894981813, green): validate step 5m20s → 4m00s, unit tests 5m20s → 3m41s, total job ~6m35s → 5m27s. Playwright e2e is now the critical path.
  • Stability: 6+ green runs of the parallel config across local, pre-push hooks, and CI.

Follow-up candidates (not in this PR)

  • Playwright is the new critical path (workers: 1 due to documented SQLITE_BUSY contention on the shared local D1 file). Per-worker databases would unlock parallelism there.
  • isolate: false for node-unit could cut the remaining per-file import cost further, but risks cross-file state leakage; not worth the flake risk without more investigation.
System recap — composes existing primitives (low risk)

Mode: recap · Base: main @ 36d84e2e · Head: e52169df

Classification: composes — no primitives added or changed. The diff touches only test-runner configuration (vitest-shared.ts, vitest.mcp-e2e.config.ts, playwright.config.ts) and nx cache inputs (nx.json); the primitives classifier matches zero primitives, the lowest-risk outcome. No runtime, storage, or surface code changes.

Primitives touched

None. All four changed files are repo tooling outside the primitives map.

Open in Web Open in Cursor 

Summary by CodeRabbit

  • Tests
    • Improved test execution consistency by refining inputs used for cached test runs.
    • Limited test worker concurrency on CI for more predictable performance.
    • Updated end-to-end test execution to avoid parallel file startup when sharing dev servers and seeded data.
    • Increased the web server startup timeout (with CI-aware values) to reduce flaky startup failures.

Drop the shared fileParallelism: false so node-unit and workers-unit
files run across all cores. The serialized run spent 180s of its 310s
CI duration on module imports alone; parallel workers cut the suite
from ~196s to ~82s locally. The mcp-e2e project keeps fileParallelism
disabled explicitly because each file boots a real Wrangler dev server
against the shared seeded e2e database.
The worker:test and worker:test-mcp targets only hashed project files,
so edits to vitest*.ts configs or tools/ (which hold vite plugins and
root-level test files picked up by the runner) returned stale cache
hits.
@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Nx test targets now track Vitest and tooling inputs. CI worker limits, MCP e2e file sequencing, and Playwright web-server startup timeouts are updated to coordinate test execution and startup.

Changes

Test configuration

Layer / File(s) Summary
Nx test input tracking
nx.json
The test and test-mcp targets now include default, workspace Vitest configuration, and tooling inputs while retaining caching.
Test runtime coordination
vitest-shared.ts, vitest.mcp-e2e.config.ts, playwright.config.ts
CI limits shared Vitest workers to three, MCP e2e tests disable file parallelism, and Playwright uses CI-aware web-server startup timeouts.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: parallelizing unit test files to reduce validation time.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cursor/faster-validate-911e

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kody-bot
kody-bot marked this pull request as ready for review July 22, 2026 05:42
@github-actions

github-actions Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

🔎 Preview deployed: https://kody-pr-849.kody-a99.workers.dev

Worker: kody-pr-849
D1: kody-pr-849-db
KV: kody-pr-849-oauth-kv

Mocks:

…t tests

The parallel unit test workers saturate all 4 CI cores at the start of
validate, and the Playwright webServer (client build + D1 migrations +
Wrangler boot) timed out at the default 60s. Raise the webServer budget
to 180s in CI and cap vitest at 3 workers there so Wrangler startup is
not starved.
@kody-bot
kody-bot merged commit 4b43eff into main Jul 22, 2026
5 checks passed
@kody-bot
kody-bot deleted the cursor/faster-validate-911e branch July 22, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants