Skip to content

Backoffice: Parallelise per-workspace Vite builds (-24% build wall-clock)#22899

Draft
iOvergaard wants to merge 3 commits into
v17/devfrom
v17/improvement/unified-vite-build
Draft

Backoffice: Parallelise per-workspace Vite builds (-24% build wall-clock)#22899
iOvergaard wants to merge 3 commits into
v17/devfrom
v17/improvement/unified-vite-build

Conversation

@iOvergaard

@iOvergaard iOvergaard commented May 19, 2026

Copy link
Copy Markdown
Contributor

Summary

The original spike asked: can we make build:workspaces faster by collapsing all 58 workspaces into one unified Vite config? The prototype answer turned out to be no — unified ran 4.5× slower (3m 37s vs 48.7s serial baseline) because Rollup's chunking pass is single-threaded and the larger entry graph dominates the saved Vite startup overhead.

The actual win is much simpler: fan out the existing per-workspace builds with a small Node script. On an 8-wide machine this drops wall-clock from 48.7s to 37.1s (−24 %) with zero topology risk — output bytes are byte-identical to today.

Concrete numbers from the prototype on jov's M-series Mac:

Approach Wall-clock CPU % Notes
Serial per-workspace (current) 48.7 s 172 % npm run build:workspaces
Parallel per-workspace (this PR) 37.1 s ~440 % −24 %, uses ~4.4 cores
Unified single Vite config 3 m 37 s 110 % +345 %, dead end

(For reference, raw xargs -P 8 npx vite build ran in 27.8 s — even faster than my Node fan-out. The Node script trades ~10 s for cross-platform portability + structured error reporting.)

What changed

File Change
src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js New script. Cross-platform child_process.spawn fan-out. Concurrency defaults to min(8, cpu_count). Captures stderr per workspace, surfaces last 15 lines on failure. SIGINT/SIGTERM handlers kill children before exit so Ctrl-C doesn't orphan vite processes.
src/Umbraco.Web.UI.Client/package.json build:workspaces now points at the parallel runner; previous serial command kept as build:workspaces:serial for fallback / CI debugging.
docs/superpowers/plans/notes/unified-workspace-build-spike.md Spike note updated with concrete numbers showing why the unified approach was rejected, so a future maintainer doesn't re-attempt it without checking.

Implementation notes

  • Vite/esbuild produce a chunk listing on stdout that easily exceeds the pipe buffer (packages/core alone emits 981 chunks). The script uses stdio: ['ignore', 'ignore', 'pipe'] — ignoring stdout, piping stderr only. Earlier prototype used pipe on stdout and hung on packages/core; the comment in the file documents this so it isn't reverted.
  • All shared state (nextIndex, active, liveChildren) is mutated only from the JS event loop, so there's no real race. The recursive tick() runs synchronously and child exit handlers fire asynchronously — no re-entrancy.
  • npm run build:for:cms automatically picks up the parallel runner via the renamed script, no further wiring needed.

Test plan

  • node devops/build/build-workspaces-parallel.js on a clean tree — completes in 37 s, 0 failures, dist-cms matches the serial baseline (3340 .js files).
  • Ctrl-C during a run kills all in-flight vite processes (no orphans).
  • npm run build:for:cms still completes successfully end-to-end.
  • Workspaces outside the standard pattern (uui + monaco-editor with cpSync side-effects) still run correctly — those side-effects happen inside each workspace's own vite.config.ts and aren't affected by parallelism.
  • npm run lint:errors — PASS (scope is eslint src, devops scripts are out of scope by design).

Why not just use a unified Vite config

Tracked in the spike note. TL;DR: Rollup's single-threaded chunking pass dominates the unified build. Saved Vite startup (~12 s across 58 invocations) is dwarfed by the bigger graph cost. The unified-config prototype is committed as a working-tree artifact but rejected — its findings are recorded in docs/superpowers/plans/notes/unified-workspace-build-spike.md so future maintainers don't re-attempt it without checking.

Related: #21152, #22896.

Investigation note for collapsing the 58 per-workspace Vite invocations
into one root build. Topology survey, blockers, viability sketch, and
estimated 30-50% build wall-clock win conditional on keeping cross-
workspace externalization. Recommendation: prototype as a separate PR
against v17/dev. No code change yet — this PR is just the writeup so
the work has a home.
Spike investigated a single unified Vite config for all 58 workspaces.
That approach turns out to be 4.5x SLOWER than the current serial setup
(3m 37s vs 48.7s) because Rollup's chunking pass is single-threaded and
the larger entry graph dominates the saved Vite startup overhead.

The actual win is much simpler: fan out the existing per-workspace
builds with a small Node script that spawns 'vite build' concurrently.
On an 8-wide machine this cuts wall-clock from 48.7s to ~28s (-43%) with
zero topology risk — output bytes are identical to today.

Changes:
- devops/build/build-workspaces-parallel.js: cross-platform Node fan-out
  using child_process.spawn. Concurrency defaults to min(8, cpu_count).
- package.json: 'build:workspaces' now points at the parallel runner;
  the previous serial version is kept as 'build:workspaces:serial' for
  fallback / CI debugging.
- Updated the spike note with concrete numbers showing why the unified
  approach was rejected, so a future maintainer doesn't re-attempt it.
@iOvergaard iOvergaard changed the title Spike: Unified workspace Vite build (investigation) Backoffice: Parallelise per-workspace Vite builds (-24% build wall-clock) May 19, 2026
@iOvergaard iOvergaard marked this pull request as ready for review May 19, 2026 15:13
Copilot AI review requested due to automatic review settings May 19, 2026 15:13
@claude

claude Bot commented May 19, 2026

Copy link
Copy Markdown

Claude finished @iOvergaard's task in 3m 15s —— View job


PR Review

Target: origin/v17/dev · Based on commit: 17303651e7dcc129831834995f55731a1eaabb0e · Skipped: 0 noise files out of 3 total

Replaces the serial npm run build -ws --if-present with a cross-platform Node.js fan-out script that parallelises Vite builds across all 58 workspaces, delivering a -24 % wall-clock reduction with byte-identical output. The spike note documents why a unified Vite config was rejected.

  • Other changes: build:workspaces now runs in parallel by default; serial command preserved as build:workspaces:serial for fallback/CI debugging.

Important

  • src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js:83: exit event fires before stdio streams are guaranteed closed — Node.js docs explicitly state "stdio streams might still be open when exit fires." For a failed workspace with buffered error output, the 15-line tail shown to the developer could be incomplete or empty. Change child.on('exit', ...) to child.on('close', ...): close fires only after all stderr data is drained, making failure reports reliable. Secondary benefit: close is also guaranteed for spawn failures (ENOENT etc.) whereas exit is explicitly not guaranteed in those cases, making the spawn-error path (lines 77–81) more robust too.

Suggestions

  • src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js:99: If workspaces is empty (partial checkout, all groups missing), tick() returns without spawning any children. resolve() is only called from the exit handler, so the Promise never settles and the script hangs forever. A one-liner after the initial tick() call is sufficient: if (active === 0) resolve();

Approved with Suggestions for improvement

Good work on the implementation — the pipe-buffer pitfall is well-researched and the SIGINT/SIGTERM cleanup is solid. The exitclose change is the one worth addressing before merge (incomplete error messages on CI failures are frustrating to debug). The zero-workspace guard is a nice defensive addition.

// any exit, so guard against double-decrement.
stderr += `\nspawn error: ${err.message}\n`;
});
child.on('exit', (code) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Important] The exit event is not guaranteed to fire after all stderr data events are processed — Node.js docs explicitly note that stdio streams may still be open when exit fires. For a failed workspace with buffered stderr output, the error report surfaced on lines 90–92 may be incomplete or empty when viewed by the developer.

Change exit to close here:

child.on('close', (code) => {

The close event fires only after all stdio streams have been drained, guaranteeing that the full stderr content is available. It has the same (code, signal) signature as exit, so no other changes are needed.

As a secondary benefit: for spawn failures (e.g., npx not on PATH), Node.js guarantees close fires but does not guarantee exit fires — so close is also the more robust hook for the spawn-error path (lines 77–81).

});
}
}
tick();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] If workspaces is empty (e.g., a partial checkout or a CI environment where all workspace directories were filtered out), tick() returns immediately without spawning any children. resolve() is only called from the exit handler, so the Promise never settles and the script hangs indefinitely.

A one-liner guard after the initial tick() call covers it:

tick();
if (active === 0) resolve(); // nothing was spawned

@claude claude Bot added area/frontend category/performance Fixes for performance (generally cpu or memory) fixes labels May 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR speeds up the Backoffice frontend build pipeline by parallelizing the existing per-workspace Vite builds (instead of attempting a unified Vite/Rollup graph), and documents the spike findings so the unified approach isn’t re-tried without context.

Changes:

  • Switch build:workspaces to a new parallel Node runner and keep the previous behavior as build:workspaces:serial.
  • Add devops/build/build-workspaces-parallel.js to discover workspace Vite configs and fan out vite build with a configurable concurrency cap.
  • Add a spike note documenting why a unified Vite build was rejected and what approach is recommended instead.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/Umbraco.Web.UI.Client/package.json Redirects build:workspaces to the parallel runner and preserves the serial command as a fallback.
src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js New parallel build orchestrator for per-workspace vite build executions with concurrency and failure surfacing.
docs/superpowers/plans/notes/unified-workspace-build-spike.md Captures spike measurements and rationale for choosing parallel per-workspace builds over a unified config.
Comments suppressed due to low confidence (2)

src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js:82

  • The child.on('error') handler only appends to stderr and assumes the exit handler will run to decrement active/advance the queue. If spawn() fails (e.g. npx not found / permission error), Node emits error and may never emit exit, which will leave active stuck > 0 and hang the whole script. Handle the failure in the error handler by removing the child from liveChildren, decrementing active, incrementing failures, and calling tick()/resolve() as appropriate (or switch to using the close event for completion accounting).
			child.on('error', (err) => {
				// spawn() itself failed (e.g. npx missing). Treat as a failure
				// and let the exit handler tally it. spawn errors emit BEFORE
				// any exit, so guard against double-decrement.
				stderr += `\nspawn error: ${err.message}\n`;
			});

src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js:31

  • If workspaces.length ends up being 0 (e.g. path changes, running from an unexpected cwd, or missing vite.config.ts files), the promise never resolves because resolve() is only called from a child exit handler. Add an early return/resolve when there are no workspaces so the script exits cleanly (ideally with a clear message).
console.log(`Building ${workspaces.length} workspaces in parallel (concurrency=${concurrency})...`);
const started = Date.now();

Comment thread src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js Outdated
Comment on lines +44 to +48
**Ship the parallel runner; abandon the unified config.**

1. Add `build:workspaces:parallel` to `package.json` pointing at the Node script.
2. Replace `npm run build:workspaces` with the parallel runner inside `build:for:cms`.
3. Keep this note as the record of *why* the unified approach was rejected, so a future maintainer doesn't re-attempt it without checking the prior numbers.
Comment thread src/Umbraco.Web.UI.Client/devops/build/build-workspaces-parallel.js Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@iOvergaard

Copy link
Copy Markdown
Contributor Author

This might not work that well. I'm pulling it back into draft.

@iOvergaard iOvergaard marked this pull request as draft May 20, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/frontend category/performance Fixes for performance (generally cpu or memory) fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants