Skip to content

Bench: Simplify and harden docgen performance measurement - #35698

Closed
valentinpalkovic wants to merge 1 commit into
nextfrom
codex/docgen-perf-tools
Closed

valentinpalkovic wants to merge 1 commit into
nextfrom
codex/docgen-perf-tools

Conversation

@valentinpalkovic

Copy link
Copy Markdown
Contributor

Follow-up to #35689

What I did

  • Split fresh-process macro latency from the forced-GC memory/OOM gate, retaining every raw cold observation and ordered warm-save trajectory.
  • Replace designated repetitions and unpaired ratios with explicit work-equivalence states plus an opt-in, seeded, locally balanced paired comparison and confidence interval.
  • Add a narrow resettable Vue Component Meta lane using exactly pinned Tinybench 6.1.2; keep Mitata and Vitest benchmark mode out of the integration harness because they do not replace its process isolation, workload validation, or stateful trajectories.
  • Normalize the Storybook-owned JSON artifacts, harden child workload validation and cleanup, and document which lane is descriptive versus gating.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

Caution

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

From the repository root with Node 22.22.3:

  1. Run yarn workspace @storybook/docgen-harness bench:docgen-perf --quick. Confirm all default engines complete and ../storybook-sandboxes/docgen-perf/results.json contains raw repetitions with gating: false.
  2. Run yarn workspace @storybook/docgen-harness bench:docgen-perf --quick --compare vue-component-meta-version --seed 42 --max-regression 100. Confirm each scenario reports 10 pairs, same-work, and a smoke verdict.
  3. Run yarn workspace @storybook/docgen-harness bench:docgen-latency --quick. Confirm both Vue Component Meta pins report exactly three samples and write a Storybook-owned result artifact.
  4. Run yarn workspace @storybook/docgen-harness bench:docgen-memory. Confirm retained-memory budgets pass, recycling survives, the no-recycling negative control OOMs, and the command ends with docgen memory gate passed.

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Declare whether manual QA will be needed for this PR during the next release, through qa:needed or qa:skip

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

@valentinpalkovic valentinpalkovic added build Internal-facing build tooling & test updates ci:normal Run our default set of CI jobs (choose this for most PRs). qa:skip Pull Requests that do not need any QA. (e.g. documentation) labels Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Internal-facing build tooling & test updates ci:normal Run our default set of CI jobs (choose this for most PRs). qa:skip Pull Requests that do not need any QA. (e.g. documentation)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant