Skip to content

Test: Update Vitest to version 5 - #82687

Merged
ciampo merged 4 commits into
trunkfrom
codex/update-vitest-5
Sep 9, 2026
Merged

ciampo merged 4 commits into
trunkfrom
codex/update-vitest-5

Conversation

@ciampo

@ciampo ciampo commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

What?

Follow up to #80855.

Updates Gutenberg's test infrastructure to Vitest 5, including the related Browser, Testing Library, jest-dom, and flakiness reporter dependencies. This is a standalone change based on trunk.

Why?

Vitest 5 requires Node 22.12 or later. Gutenberg now requires Node 24, so the repository can adopt the current major release.

This PR applies the migration changes needed by current consumers and leaves optional configuration and performance experiments for follow-ups.

How?

  • Aligns vitest and @vitest/browser-playwright at 5.0.0. It also updates Testing Library React to 16.3.3, jest-dom to 7.0.1, and @flakiness/vitest to 1.9.2.
  • Migrates custom matcher types, runner-specific jest-dom setup, the removed describe.sequential API, and changed snapshot title formatting.
  • Keeps clearMocks: false because mockReset: true already performs the same call-history cleanup and also resets implementations.
  • Ignores Vitest's .vitest artifact directory.
  • Disconnects useResizeObserver on unmount. Vitest 5's stricter error handling exposed a queued callback running during jsdom teardown.
Complete release span and dependency versions
  • vitest 4.1.11 to 5.0.0: 5.0.0
  • @vitest/browser-playwright 4.1.10 to 5.0.0: 4.1.11, 5.0.0
  • @testing-library/react 16.3.2 to 16.3.3: 16.3.3
  • @testing-library/jest-dom 6.9.1 to 7.0.1: 6.10.0, 7.0.0, 7.0.1
  • @flakiness/vitest 1.9.0 to 1.9.2: 1.9.1, 1.9.2

Vitest's transitive Browser, mocker, UI, and runtime packages use the same 5.0.0 monorepo release. @vitest/ui is new to the lockfile, so it has no earlier resolved version.

Notable upstream changes

  • Runtime requirements. Vitest 5 requires Node 22.12+ and Vite 6.4+. Gutenberg uses Node 24 and Vite 8.2.2. This affects development tests only.
  • Matcher types and removed APIs. Gutenberg's shared matcher environment and Interactivity tests used the changed Matchers<R, T> type. One infrastructure test used describe.sequential. Both required migration. TypeScript and the full suite pass.
  • Stricter assertions and errors. No assertion or hoisted-mock rewrites were needed. The stricter runner did expose useResizeObserver work continuing after unmount, which this PR fixes and covers with a regression test.
  • clearMocks by default. Gutenberg already enables the stronger mockReset. The explicit clearMocks: false avoids a redundant cleanup pass, and an infrastructure test documents the choice.
  • Browser Mode. Gutenberg does not use the changed custom-command or browser.api contracts. The Gutenberg Browser project and all 833 Storybook Browser tests pass. .vitest is ignored for new failure artifacts.
  • Optional features. Nested projects, shared Vite servers, the filesystem module cache, Temporal mocking, and new reports are not compatibility requirements. Gutenberg has no Temporal consumer or Vitest bench import. The useful experiments are listed below.
  • Related dependencies. Testing Library React 16.3.3 fixes re-entrant act() handling. jest-dom 7's Node and @testing-library/dom requirements are satisfied, and 7.0.1 declares Vitest as an optional peer. The flakiness patches correct Vitest stdio and .fails reporting. These changes affect tests and reporting only.

The review also covered the complete Vitest migration guide and Vitest 5 release highlights.

Known compromise

@storybook/addon-vitest 10.5.9 still declares Vitest 3/4 peers, so npm ls reports the Vitest 5 packages as invalid. The clean install, production build, static Storybook build, and all 833 Storybook Browser tests pass with the unpatched addon. Removing it would remove Storybook's Vitest coverage, while a local metadata patch would only hide the warning. This PR keeps it unchanged while storybookjs/storybook#36221 adds upstream support.

Performance

The same-head Node 24 comparison found no reproducible speed improvement. Four CI-shaped Node/jsdom shards were flat overall, and the three-run Storybook median was 0.4% slower. Vitest's shared-server optimization does not apply to Gutenberg's current project layout.

Measurements
Workload Vitest 4 Vitest 5 Result
Four Node/jsdom shards, total 228.19 s 228.68 s +0.2%
Four shards, critical path 59.49 s 62.40 s +4.9%
Storybook Browser tests, median 102.60 s 102.97 s +0.4%

Both versions used Node 24 and base commit b21da9b2199.

Follow-ups

Configuration, performance, and flake investigations
  • Prototype nested Node/jsdom projects with inherited configuration and a shared Vite server. An initial experiment kept all 1,068 files and reduced one shard's two-run mean by about 6.8%, but combining the integration and Browser projects discovered zero integration tests.
  • Re-test fsModuleCache. A warm-cache trial improved the median by about 4.7%, but worsened the mean, created about 93 MB of cache data, and does not support Browser Mode.
  • Investigate the load-sensitive jest-console subprocess timeout under CI-shaped contention. It reached the five-second limit during loaded full runs, but passed 20 isolated repetitions and the final complete run. Profile Jest startup before changing the test or timeout.
  • Consider injectCjsGlobals: false, Browser traces, and single-file HTML reports as separate hardening and diagnostics changes.
  • Adopt Temporal mocking if Gutenberg begins testing Temporal behavior.

Testing Instructions

  1. Install with Node 24 and run the dependency validation scripts.
  2. Run the typecheck, production build, convention tests, routing tests, and complete Vitest suite.
  3. Build Storybook and run its complete smoke suite.
  4. Run npm ls vitest @vitest/browser @vitest/browser-playwright. The only invalid peers should be the documented Storybook addon ranges.

Verified locally with a clean install, build, typecheck, dependency checks, 35,749 passing Vitest tests, and 833 passing Storybook Browser tests.

Testing Instructions for Keyboard

Not applicable. This PR changes development dependencies and test infrastructure only.

Use of AI Tools

Codex helped audit upstream releases and migration guidance, inspect consumers, implement the changes, run verification and performance comparisons, and draft this description. The author reviewed the changes and conclusions against the repository and official upstream sources.

@github-actions github-actions Bot added [Package] Element /packages/element [Package] Date /packages/date [Package] Data /packages/data [Package] Server Side Render /packages/server-side-render [Package] A11y /packages/a11y [Package] Autop /packages/autop [Tool] Babel preset /packages/babel-preset-default [Tool] Browserslist config /packages/browserslist-config [Package] DOM ready /packages/dom-ready [Package] Hooks /packages/hooks [Package] i18n /packages/i18n [Package] is-shallow-equal /packages/is-shallow-equal [Tool] Jest console /packages/jest-console [Tool] npm-package-json-lint config /packages/npm-package-json-lint-config [Tool] WP Scripts /packages/scripts [Package] Url /packages/url [Package] Word count /packages/wordcount [Package] Deprecated packages/deprecated [Tool] Babel plugin makepot /packages/babel-plugin-makepot [Tool] Babel plugin import JSX pragma /packages/babel-plugin-import-jsx-pragma [Package] Blob /packages/blob [Package] Compose /packages/compose [Package] Core data /packages/core-data [Package] API fetch /packages/api-fetch [Package] HTML entities /packages/html-entities [Package] Viewport /packages/viewport [Package] DOM /packages/dom [Tool] Jest preset /packages/jest-preset-default [Package] Keycodes /packages/keycodes labels Sep 9, 2026
@github-actions github-actions Bot added [Package] Escape HTML /packages/escape-html [Package] Interface /packages/interface [Package] Icons /packages/icons [Tool] stylelint config /packages/stylelint-config [Package] Edit Site /packages/edit-site [Package] Warning /packages/warning [Tool] Readable JS Assets Webpack Plugin /packages/readable-js-assets-webpack-plugin [Package] Style Engine /packages/style-engine [Package] Preferences /packages/preferences [Package] Media Utils /packages/media-utils [Package] Private APIs /packages/private-apis [Package] Core commands /packages/core-commands [Package] Interactivity /packages/interactivity [Package] DataViews /packages/dataviews [Package] Interactivity Router /packages/interactivity-router [Package] Fields /packages/fields [Package] Theme /packages/theme [Package] UI /packages/ui [Package] wp-build /packages/wp-build [Package] Sync /packages/sync labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: ciampo <mciampini@git.wordpress.org>
Co-authored-by: manzoorwanijk <manzoorwanijk@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

🤖 PR meta 🤖

📦 Bundle size

Size Change: +23 B (0%)

Total Size: 8 MB

📦 View Changed
Filename Size Change
build/scripts/compose/index.min.js 11.7 kB +23 B (+0.2%)

456acd0 Run

⚡ Performance

Show the results

Client side metrics exclude the server response time.

front-end-block-theme

Metric fcdc678 trunk % Change
timeToFirstByte 57.5 ms +10.87% -6.26% 54.75 ms +11.32% -3.74% 5.02%
largestContentfulPaint 98 ms +6.12% -8.16% 92 ms +8.7% -8.7% 6.52%
lcpMinusTtfb 35.6 ms +24.58% -2.39% 33.8 ms +22.04% -5.77% 5.33%
wpBeforeTemplate 27.79 ms +20.69% -3.49% 26.29 ms +19.67% -1.22% 5.71%
wpTemplate 25.19 ms +3.06% -7.66% 23.57 ms +2.5% -3.52% 6.87%
wpTotal 53.41 ms +11.48% -6.14% 51.08 ms +10.73% -4.46% 4.56%
wpMemoryUsage 7.52 MB +0% -0% 7.48 MB +0% -0% 0.46%
wpDbQueries 17 +0% -0% 17 +0% -0% 0%

front-end-classic-theme

Metric fcdc678 trunk % Change
timeToFirstByte 42.3 ms +12.29% -3.9% 41.1 ms +4.62% -1.95% 2.92%
largestContentfulPaint 98 ms +6.12% -2.04% 96 ms +0% -4.17% 2.08%
lcpMinusTtfb 54.3 ms +3.04% -1.84% 53.3 ms +3.94% -3.47% 1.88%
wpBeforeTemplate 24.43 ms +14.94% -4.67% 23.22 ms +3.96% -0.73% 5.21%
wpTemplate 14.55 ms +6.46% -3.64% 14.3 ms +3.36% -2.59% 1.75%
wpTotal 39.2 ms +12.88% -4.39% 38.01 ms +5.08% -2.21% 3.13%
wpMemoryUsage 5.51 MB +0% -0% 5.47 MB +0% -0% 0.64%
wpDbQueries 14 +0% -0% 14 +0% -0% 0%

media-processing

Metric fcdc678 trunk % Change
mediaProcessingJpeg 412.75 ms +2.88% -0.73% 413.74 ms +0.59% -0.92% -0.24%
mediaProcessingAvif 6164.88 ms +0.43% -0.11% 6179.01 ms +0.19% -0.22% -0.23%
mediaProcessingJpegToAvif 4301.03 ms +0.59% -0.16% 4302.01 ms +0.3% -0.23% -0.02%

media-upload

Metric fcdc678 trunk % Change
jpegUploadProcessing 1440.25 ms +34.98% -1.28% 1422.59 ms +1.6% -0.55% 1.24%
pngUploadProcessing 201.85 ms +2.24% -3.42% 200.4 ms +5.28% -1.37% 0.72%
largeJpegUploadProcessing 1411.57 ms +0.31% -0.41% 1414.77 ms +0.47% -0.35% -0.23%
multipleImageUploadProcessing 1545.61 ms +1.53% -2.07% 1571.48 ms +1.33% -1.42% -1.65%

post-editor

Metric fcdc678 trunk % Change
serverResponse 311.11 ms +16.87% -1.92% 335.17 ms +3.3% -6.68% -7.18%
firstPaint 207.79 ms +24.18% -16.58% 179.58 ms +19.38% -10.74% 15.71%
domContentLoaded 1022.47 ms +3.9% -0.63% 1027.92 ms +1.31% -0.73% -0.53%
loaded 1023.56 ms +3.9% -0.62% 1029.04 ms +1.32% -0.72% -0.53%
firstContentfulPaint 354.35 ms +0.28% -1.55% 347.74 ms +2.22% -4.21% 1.9%
firstBlock 2763.47 ms +2.5% -1.1% 2766.73 ms +0.77% -0.59% -0.12%
type 13.54 ms +4.95% -6.57% 13.25 ms +6.11% -1.43% 2.19%
typeWithoutInspector 13.33 ms +4.58% -6.23% 13.17 ms +5.01% -5.09% 1.21%
typeWithTopToolbar 15.25 ms +2.1% -4.92% 16.72 ms +4.43% -6.88% -8.79%
typeContainer 7.56 ms +16.27% -3.7% 7.95 ms +11.32% -7.04% -4.91%
focus 66.06 ms +14.58% -9.11% 67.01 ms +4.92% -6.92% -1.42%
firstFocus 142.44 ms +0% -0% 148.48 ms +0% -0% -4.07%
selectAll 464.81 ms +0.35% -1.8% 456.82 ms +3.93% -2.59% 1.75%
listViewOpen 55.8 ms +10.41% -10.84% 57.56 ms +4.99% -11.5% -3.06%
inserterOpen 21.21 ms +10.14% -9.24% 20.43 ms +2.64% -7.34% 3.82%
inserterHover 3.43 ms +8.75% -6.41% 3.54 ms +15.25% -7.06% -3.11%
inserterSearch 7.53 ms +8.9% -5.71% 7.26 ms +10.33% -2.89% 3.72%
loadPatterns 527.98 ms +5.51% -3.25% 543.33 ms +2.38% -3.71% -2.83%
wpTotal 302.26 ms +17.32% -1.91% 326.2 ms +3.52% -6.82% -7.34%
wpMemoryUsage 13.05 MB +0% -0% 13.02 MB +0% -0% 0.28%
wpDbQueries 54 +0% -0% 54 +0% -0% 0%

site-editor

Metric fcdc678 trunk % Change
serverResponse 310.07 ms +4.68% -3.62% 325.42 ms +1.93% -2.73% -4.72%
firstPaint 204.68 ms +64.26% -12.69% 191.37 ms +13.15% -16.31% 6.96%
domContentLoaded 1094.84 ms +0.78% -0.4% 1091.5 ms +1.13% -1.23% 0.31%
loaded 1095.88 ms +0.79% -0.39% 1092.63 ms +1.12% -1.24% 0.3%
firstContentfulPaint 345.12 ms +7.79% -2.58% 352 ms +1.64% -4.27% -1.95%
firstBlock 3427.83 ms +1.44% -0.4% 3432.76 ms +0.52% -0.67% -0.14%
type 13.44 ms +4.61% -5.65% 13.53 ms +5.47% -7.46% -0.67%
navigate 76.86 ms +18.62% -14.42% 67.64 ms +17.42% -2.57% 13.63%
loadPatterns 1070.77 ms +23.27% -3.44% 1107.34 ms +17.83% -2.22% -3.3%
loadPages 1104.57 ms +1.52% -0.81% 1083.47 ms +3.48% -2.7% 1.95%
wpTotal 301.67 ms +4.87% -3.74% 317.02 ms +1.94% -2.83% -4.84%
wpMemoryUsage 12.06 MB +0% -0% 12.03 MB +0% -0% 0.3%
wpDbQueries 44 +0% -2.27% 43.5 +1.15% -1.15% 1.15%

456acd0 Run

@manzoorwanijk manzoorwanijk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks straighforward if the CI is happy.

Comment thread test/unit/vitest.config.mjs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aduth another fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Package] A11y /packages/a11y [Package] Abilities /packages/abilities [Package] Admin UI /packages/admin-ui [Package] API fetch /packages/api-fetch [Package] Autop /packages/autop [Package] Blob /packages/blob [Package] Block editor /packages/block-editor [Package] Block library /packages/block-library [Package] Blocks /packages/blocks [Package] Components /packages/components [Package] Compose /packages/compose [Package] Core commands /packages/core-commands [Package] Core data /packages/core-data [Package] Data Controls /packages/data-controls [Package] Data /packages/data [Package] DataViews /packages/dataviews [Package] Date /packages/date [Package] Deprecated packages/deprecated [Package] Design System MCP /packages/design-system-mcp [Package] DOM ready /packages/dom-ready [Package] DOM /packages/dom [Package] Edit Post /packages/edit-post [Package] Edit Site /packages/edit-site [Package] Edit Widgets /packages/edit-widgets [Package] Editor /packages/editor [Package] Element /packages/element [Package] Escape HTML /packages/escape-html [Package] Fields /packages/fields [Package] Format library /packages/format-library [Package] Grid /packages/grid [Package] Hooks /packages/hooks [Package] HTML entities /packages/html-entities [Package] i18n /packages/i18n [Package] Icons /packages/icons [Package] Interactivity Router /packages/interactivity-router [Package] Interactivity /packages/interactivity [Package] Interface /packages/interface [Package] is-shallow-equal /packages/is-shallow-equal [Package] Keycodes /packages/keycodes [Package] Media Utils /packages/media-utils [Package] Notices /packages/notices [Package] Plugins /packages/plugins [Package] Preferences /packages/preferences [Package] Priority Queue /packages/priority-queue [Package] Private APIs /packages/private-apis [Package] Project management automation /packages/project-management-automation [Package] Redux Routine /packages/redux-routine [Package] Rich text /packages/rich-text [Package] Server Side Render /packages/server-side-render [Package] Style Engine /packages/style-engine [Package] Sync /packages/sync [Package] Theme /packages/theme [Package] Token List /packages/token-list [Package] UI /packages/ui [Package] Url /packages/url [Package] Viewport /packages/viewport [Package] Views /packages/views [Package] Warning /packages/warning [Package] Widget Dashboard /packages/widget-dashboard [Package] Widget primitives /packages/widget-primitives [Package] Word count /packages/wordcount [Package] wp-build /packages/wp-build [Tool] Babel plugin import JSX pragma /packages/babel-plugin-import-jsx-pragma [Tool] Babel plugin makepot /packages/babel-plugin-makepot [Tool] Babel preset /packages/babel-preset-default [Tool] Browserslist config /packages/browserslist-config [Tool] Dependency Extraction Webpack Plugin /packages/dependency-extraction-webpack-plugin [Tool] Docgen /packages/docgen [Tool] Env /packages/env [Tool] ESLint plugin /packages/eslint-plugin [Tool] Jest console /packages/jest-console [Tool] Jest preset /packages/jest-preset-default [Tool] npm-package-json-lint config /packages/npm-package-json-lint-config [Tool] Prettier config /packages/prettier-config [Tool] Readable JS Assets Webpack Plugin /packages/readable-js-assets-webpack-plugin [Tool] stylelint config /packages/stylelint-config [Tool] WP Scripts /packages/scripts [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants