Repository navigation
fix(ci): bundle-size ratchet no longer passes when it cannot measure (#15159 G-04) - #15282
Merged
diegosouzapw merged 1 commit intoOct 2, 2026
Conversation
…iegosouzapw#15159 G-04) `check-bundle-size.mjs --ratchet` printed `bundleSize=SKIP reason=…` and exited **0** on five separate "could not measure" paths, each with a comment saying so on purpose — e.g. :252 "SKIP sai 0 mesmo com --ratchet (erro de medição nunca bloqueia)". The ci.yml step is labelled "Bundle size (ratchet, blocking)". A ratchet that cannot take a measurement and then reports OK converts "unknown" into "verified". The five paths: size-limit missing, size-limit without plugins, unexpected size-limit error, no baseline in quality-baseline.json, and a fallback-stat measurement (raw bytes) not comparable to the gzip baseline. **The rule is now one sentence:** under `--ratchet`, "not measured" is exit 1 with the reason and the remedy; advisory mode still exits 0 (a developer running the gate locally without a build should not be blocked). A CI `::error::` annotation is emitted so the reason is visible in the run summary. ## The fix surfaced a second, larger bug Once an unmeasurable run stopped exiting 0, the real gate started failing — and the cause was not the ratchet at all: node_modules\.bin\size-limit:2 basedir=$(dirname "$(echo "$0" | sed -e 's,\,/,g')") SyntaxError: missing ) after argument list `runSizeLimit` invoked `node node_modules/.bin/size-limit --json`. On Windows that path is a **shell shim, not JavaScript**, so `node <shim>` died on every invocation. The preferred measurement mode was unreachable on any Windows machine — always falling through to the raw-byte fallback, which is exactly the path that could never produce a ratchet verdict. It was invisible only because the failure exited 0. Fixed by resolving the package's own bin entry via the repo's existing, already tested `scripts/build/buildToolRunner.mjs` (the helper written for the esbuild Windows postbuild ENOENT incident) rather than inventing a second resolver. It prefers the JS entry, detects native binaries, and falls back to the `.cmd` shim with a shell on win32. The preferred measurement now works on Windows: **9394 bytes** measured against the 10384 baseline, exit 0. Also fixed while threading the root through `runSizeLimit(cwd)`: resolution previously used a module-level `ROOT` and silently ignored its `cwd` argument. ## Verification (release/v3.8.52 @ dbe703a) | Check | Result | | --- | --- | | `check-bundle-size-unmeasurable.test.ts` (new) | 12/12 pass | | `check-bundle-size.test.ts` (existing) | 21/21 pass | | `build-tool-runner-win-shim.test.ts` | pass (reused helper unaffected) | | `npm run check:bundle-size -- --ratchet` | exit 0, real measurement 9394 | | `npm run check:bundle-size` (advisory) | exit 0 | | `tests/unit/build/*.test.ts` | 534 tests, 12 fail — the same 12 that fail at base | The new suite drives the real CLI as a subprocess against fixture repos with a stubbed size-limit package, and keeps four guard cases so the fix cannot over-correct: advisory still exits 0, a measured regression still blocks, and a clean measurement still passes.⚠️ base-red inherited: diegosouzapw#15246. Refs diegosouzapw#15159 (G-04)
Contributor
Author
|
Ordering note (not a defect in this PR): the |
diegosouzapw
merged commit Oct 2, 2026
dd66b15
into
diegosouzapw:release/v3.8.52
15 of 16 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
check-bundle-size.mjs --ratchetprintedbundleSize=SKIP reason=…and exited0 on five separate "could not measure" paths. The ci.yml step is labelled
"Bundle size (ratchet, blocking)".
A ratchet that cannot take a measurement and then reports OK converts unknown
into verified — worse than no gate at all. The five paths, each with a comment
stating the behaviour was intentional:
size-limitnot installedSL_NO_BIN→ fallback-stat → exit 0size-limitwithout pluginsSL_NO_PLUGINS→ fallback-stat → exit 0size-limiterrorconsole.erroran Aviso, print SKIP, exit 0metrics.bundleSizebaselineSKIP, sai 0SKIP (medição não-comparável…)→ exit 0The catalog named
:250-256. The other four had the same defect and are fixedhere too — a partial fix would leave the gate able to pass without measuring.
The fix surfaced a second, larger bug
Once an unmeasurable run stopped exiting 0, the real gate started failing, and
the cause was not the ratchet:
runSizeLimitinvokednode node_modules/.bin/size-limit --json. On Windows thatpath is a shell shim, not JavaScript, so
node <shim>died on everyinvocation. The preferred measurement mode was unreachable on any Windows
machine — every run fell through to the raw-byte fallback, which is precisely the
path that could never produce a ratchet verdict. It went unnoticed only because
the failure exited 0.
Fixed by resolving the package's own
binentry through the repo's existing,already-tested
scripts/build/buildToolRunner.mjs— the helper written for theesbuild Windows postbuild ENOENT incident — rather than inventing a second
resolver. It prefers the JS entry, detects native binaries, and falls back to the
.cmdshim with a shell on win32.The preferred measurement now works on Windows: 9394 bytes vs the 10384
baseline, exit 0.
Also fixed while threading the root through
runSizeLimit(cwd): resolution used amodule-level
ROOTand silently ignored itscwdargument.The rule, now
A CI
::error::annotation is emitted so the reason is visible in the runsummary. The
bundleSize=SKIP reason=<token>stdout token is unchanged, so logscrapers keep working.
Verification (release/v3.8.52 @ dbe703a)
check-bundle-size-unmeasurable.test.ts(new)check-bundle-size.test.ts(existing)build-tool-runner-win-shim.test.tsnpm run check:bundle-size -- --ratchetnpm run check:bundle-size(advisory)tests/unit/build/*.test.tsThe new suite drives the real CLI as a subprocess against fixture repos with a
stubbed
size-limitpackage. It keeps four guard cases so the fix cannotover-correct into a gate that blocks on legitimate conditions: advisory still
exits 0, a measured regression still blocks, and a clean measurement still
passes.
Two fixture bugs were caught and fixed during the cycle rather than papered over:
writeFileSyncdoes not create missing parent dirs, and stubbing the.binshimstops working once the gate correctly bypasses it — the stub has to be the
package entry.
release/v3.8.52is not green (6 hard failures inunit, vitest, integration, package-artifact and tarball jobs). None originate here.
Scope
scripts/check/check-bundle-size.mjs·tests/unit/build/check-bundle-size-unmeasurable.test.ts(new) ·tests/unit/build/check-bundle-size.test.ts(one test updated for the newrunSizeLimitsignature) ·.github/workflows/ci.yml(comment only — the step wasalready correct, its description was the lie).
Refs #15159 (G-04)