Skip to content

Maintenance: Fix self healing payload#34782

Merged
yannbf merged 1 commit into
nextfrom
yann/fix-payload-names
May 13, 2026
Merged

Maintenance: Fix self healing payload#34782
yannbf merged 1 commit into
nextfrom
yann/fix-payload-names

Conversation

@yannbf
Copy link
Copy Markdown
Member

@yannbf yannbf commented May 13, 2026

Closes #

What I did

This PR removes the "run" prefix from the payload for the test events, an unintentional breaking change.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

Caution

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

Summary by CodeRabbit

  • Refactor

    • Streamlined internal naming conventions for test result analysis fields, removing legacy prefixes to improve code consistency and maintainability.
  • Tests

    • Updated test coverage across telemetry reporting and result processing systems to validate refactored internal data structures.

Review Change Stack

@yannbf yannbf self-assigned this May 13, 2026
@yannbf yannbf added maintenance User-facing maintenance tasks ci:normal labels May 13, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 13, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f5eb771d-cad0-43c0-adb7-18c1e1a56d75

📥 Commits

Reviewing files that changed from the base of the PR and between 97716bd and 3ebdd02.

📒 Files selected for processing (9)
  • code/addons/vitest/src/vitest-plugin/agent-telemetry-reporter.test.ts
  • code/core/src/core-server/server-channel/ai-setup-channel.test.ts
  • code/core/src/core-server/server-channel/ghost-stories-channel.test.ts
  • code/core/src/core-server/utils/ghost-stories/parse-vitest-report.test.ts
  • code/core/src/shared/utils/analyze-test-results.test.ts
  • code/core/src/shared/utils/analyze-test-results.ts
  • code/core/src/shared/utils/test-result-types.ts
  • scripts/eval/lib/grade.ts
  • scripts/eval/lib/story-render.ts

📝 Walkthrough

Walkthrough

This PR renames fields in the TestRunAnalysis interface and all consuming code by removing the run prefix (e.g., runTotaltotal, runCssCheckcssCheck), propagating the changes through implementations, unit tests, integration tests, and evaluation scripts.

Changes

TestRunAnalysis refactoring

Layer / File(s) Summary
Type definition
code/core/src/shared/utils/test-result-types.ts
TestRunAnalysis interface fields are renamed by removing the run prefix: runTotal/runPassed/runPassedButEmptyRender/runSuccessRate/runSuccessRateWithoutEmptyRender/runUniqueErrorCount/runCategorizedErrors/runCssCheck become total/passed/passedButEmptyRender/successRate/successRateWithoutEmptyRender/uniqueErrorCount/categorizedErrors/cssCheck.
analyzeTestResults implementation and unit tests
code/core/src/shared/utils/analyze-test-results.ts, code/core/src/shared/utils/analyze-test-results.test.ts
analyzeTestResults populates the renamed fields from the run summary; unit tests verify the new field names across passing tests, failures, cumulative analysis, CSS check outcomes (pass/fail/not-run), and first-match-wins scenarios.
Parse vitest report test expectations
code/core/src/core-server/utils/ghost-stories/parse-vitest-report.test.ts
Tests validate the new summary field names in parsed Vitest reports across all-passing, failed, categorized-error, empty-render, multi-suite, zero-total, and CSS check scenarios.
Agent telemetry reporter test expectations
code/addons/vitest/src/vitest-plugin/agent-telemetry-reporter.test.ts
Telemetry payload assertions updated to expect renamed fields in multiple test runs, cumulative caching, example-story filtering, and results reset scenarios.
Ghost stories and AI setup channel integration tests
code/core/src/core-server/server-channel/ghost-stories-channel.test.ts, code/core/src/core-server/server-channel/ai-setup-channel.test.ts
Telemetry payloads in ghost-story discovery and AI-generated story scoring tests now assert the renamed results fields.
Evaluation script consumers
scripts/eval/lib/grade.ts, scripts/eval/lib/story-render.ts
Ghost-story grading and story render grading read the renamed summary fields (passedButEmptyRender, total, passed, cssCheck) when constructing grades from parsed reports.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes


Possibly related PRs

  • storybookjs/storybook#34699: Both PRs update the same TestRunAnalysis and related consumers, switching between run*-prefixed metrics and the new unprefixed shape.
  • storybookjs/storybook#34600: Both PRs modify analyzeTestResults/TestRunAnalysis output shape, specifically around the cssCheck telemetry field renaming.
  • storybookjs/storybook#34537: The PR's renaming of TestRunAnalysis fields and corresponding telemetry test expectations directly overlaps with this PR's new analyzeTestResults telemetry flow.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands and usage tips.

@yannbf yannbf merged commit a5d6744 into next May 13, 2026
142 of 147 checks passed
@yannbf yannbf deleted the yann/fix-payload-names branch May 13, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:normal maintenance User-facing maintenance tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants