Automated Testing: Add Storybook component documentation regression testing

.github/workflows/storybook-check.yml

@@ -44,6 +44,24 @@ jobs:
             - name: Validate manifest matches design-system-mcp contract
               run: node packages/design-system-mcp/scripts/validate-manifest.mjs
 
+            # Regenerates the committed snapshot of components/props/descriptions
+            # from the freshly built manifest. Fails if a component or prop is
+            # newly missing a description (see the allowlist for known gaps).
+            - name: Generate manifest snapshot
+              run: npm run storybook:manifest-snapshot
+
+            # Fails when the generated snapshot or pruned allowlist differs from
+            # what's committed, surfacing documentation changes for sign-off.
+            # This is similar to the "Check local changes" step of the static
+            # checks workflow, but the other workflow does not build Storybook,
+            # which is an expensive operation and only happens here.
+            - name: Check manifest snapshot is up to date
+              run: |
+                  if ! git diff --exit-code -- storybook/components-manifest.yml storybook/prop-description-allowlist.json; then
+                      echo "::error::The Storybook manifest snapshot is out of date. Review the diff above to confirm the change is expected, then run 'npm run storybook:build && npm run storybook:manifest-snapshot' locally and commit the updated storybook/components-manifest.yml and storybook/prop-description-allowlist.json."
+                      exit 1
+                  fi
+
             - name: Build E2E Storybook
               env:
                   NODE_ENV: test

package.json

@@ -157,6 +157,7 @@
 		"storybook:dev": "concurrently \"npm run dev\" \"wait-on .dev-ready && npm run --workspace @wordpress/storybook storybook:dev\"",
 		"storybook:e2e:dev": "concurrently \"npm run dev\" \"wait-on .dev-ready && npm run --workspace @wordpress/storybook-playwright storybook:dev\"",
 		"storybook:e2e:build": "npm run --workspace @wordpress/storybook-playwright storybook:build",
+		"storybook:manifest-snapshot": "npm run --workspace @wordpress/storybook manifest-snapshot",
 		"test": "npm-run-all lint test:unit",
 		"test:create-block": "npm run --workspace @wordpress/validation-tools test-create-block --",
 		"test:e2e": "npm exec --workspace @wordpress/e2e-tests-playwright -- wp-scripts test-e2e --config playwright.config.ts",

storybook/README.md

@@ -7,3 +7,21 @@ The Gutenberg project uses Storybook to view and work with the UI components dev
 View online at: https://wordpress.github.io/gutenberg/
 
 Run locally in your development environment running: `npm run storybook:dev` from the top-level Gutenberg directory.
+
+## Manifest snapshot regression testing
+
+Storybook upgrades and inocuous code refactoring have been a frequent source of accidental documentation regressions, resulting in component or prop descriptions being accidentally removed.
+
+To catch these, a consolidated snapshot of the components manifest is committed:
+
+-   `storybook/components-manifest.yml` is a list of each component and its props.
+-   `storybook/prop-description-allowlist.json` is a set of known components and props that are currently missing a description. The generator script fails on any new undocumented component or prop that isn't listed here, so this allowlist is expected to shrink over time.
+
+Description text is not stored in the snapshot, since it is noisy to diff and increases the burden on developers. Instead, the snapshot reflects and enforces on presence rather than specific values.
+
+When a snapshot changes due to code changes or Storybook upgrades, a pull request's checks will fail and expect the developer to commit the changes after acknowledging that they are expedcted:
+
+```bash
+npm run storybook:build
+npm run storybook:manifest-snapshot
+```