Skip to content

Test: Improve stack trace when failing on console.warn/error#34697

Merged
JReinhold merged 5 commits into
nextfrom
jeppe/improve-fail-on-error
May 4, 2026
Merged

Test: Improve stack trace when failing on console.warn/error#34697
JReinhold merged 5 commits into
nextfrom
jeppe/improve-fail-on-error

Conversation

@JReinhold
Copy link
Copy Markdown
Contributor

@JReinhold JReinhold commented May 4, 2026

Closes #

What I did

Ran into a test failing because of a console.warn. Couldn't figure out why or where. Used vi.defineHelper instead to cause the failure from console.error/warn, to get the actual stack trace instead. It worked.
Later found out the test was already fixed on next..

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

Don't.

Caution

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

Summary by CodeRabbit

  • Chores

    • Updated Vitest test framework dependencies to v4.1.5 across all packages to keep test tooling current and consistent.
  • Refactor

    • Improved test setup to more robustly filter and surface unexpected console warnings/errors during tests, reducing noise and making test failures clearer.

@JReinhold JReinhold self-assigned this May 4, 2026
@JReinhold JReinhold added build Internal-facing build tooling & test updates test utilities labels May 4, 2026
@JReinhold JReinhold requested review from AriPerkkio and yannbf May 4, 2026 09:00
@JReinhold JReinhold moved this to In Progress in Core Team Projects May 4, 2026
@JReinhold JReinhold added the ci:normal Run our default set of CI jobs (choose this for most PRs). label May 4, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 4, 2026

📝 Walkthrough

Walkthrough

This PR bumps Vitest-related package versions from ^4.1.0 to ^4.1.5 across multiple manifests, widens a TypeScript coverage-options type, and refactors test-time console mocking in code/vitest-setup.ts from an ignore-list + throw helpers approach to a predicate-based helper that fails on unexpected console calls.

Changes

Vitest Dependency Upgrades & Type Adjustment

Layer / File(s) Summary
Dependency Updates
code/addons/vitest/package.json, code/package.json, package.json, scripts/package.json
Bumped Vitest and related packages (vitest, @vitest/browser, @vitest/browser-playwright, @vitest/coverage-istanbul, @vitest/coverage-v8, @vitest/runner) from ^4.1.0^4.1.5.
Type Widening
code/addons/vitest/src/node/coverage-reporter.ts
StorybookCoverageReporterOptions.coverageOptions type changed from `ResolvedCoverageOptions<'v8'>

Console Mocking Refactor

Layer / File(s) Summary
Behavioral Implementation
code/vitest-setup.ts
Replaced ignoreList + throwWarning/throwError mock implementations with an ALLOWED_CONSOLE_PREDICATES array and a failOnConsole helper that installs per-level vi.defineHelper to call expect.fail(...) for disallowed console calls.
Predicate Changes
code/vitest-setup.ts
Updated allowed-message predicates: removed a React stack-based ignore condition and converted several match checks to includes()/test()-style string checks.
Wiring
code/vitest-setup.ts
Unified wiring for console.warn and console.error via ['warn','error'].forEach(...) to use the new helper.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Review rate limit: 4/5 reviews remaining, refill in 12 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
code/vitest-setup.ts (1)

9-38: 💤 Low value

Consider adding a type guard for non-string console arguments.

The predicates assume the first argument is a string (calling .includes() / .test()). If console.warn or console.error is called with a non-string first argument (e.g., console.warn(someObject)), this will throw a TypeError before reaching the expect.fail() check, potentially producing a confusing error.

🛡️ Suggested defensive approach
 const ALLOWED_CONSOLE_PREDICATES: ((...args: any[]) => boolean)[] = [
-  (message) => message.includes('":nth-child" is potentially unsafe'),
+  (message) => typeof message === 'string' && message.includes('":nth-child" is potentially unsafe'),
   // ... apply same pattern to other predicates
 ];

Alternatively, add a single type guard at the start of the mock implementation before checking predicates.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@code/vitest-setup.ts` around lines 9 - 38, The predicates in
ALLOWED_CONSOLE_PREDICATES assume the first console argument is a string and
call .includes()/.test() on "message", which can throw for non-string values;
update the predicates (or add a single wrapper where they are used) to guard for
typeof message === 'string' (or coerce via String(message)) before calling
.includes()/.test(), e.g., ensure each predicate first returns false if message
is not a string so non-string console calls (objects/errors) won't cause a
TypeError; refer to ALLOWED_CONSOLE_PREDICATES and the individual predicate
functions that reference "message".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@code/vitest-setup.ts`:
- Around line 9-38: The predicates in ALLOWED_CONSOLE_PREDICATES assume the
first console argument is a string and call .includes()/.test() on "message",
which can throw for non-string values; update the predicates (or add a single
wrapper where they are used) to guard for typeof message === 'string' (or coerce
via String(message)) before calling .includes()/.test(), e.g., ensure each
predicate first returns false if message is not a string so non-string console
calls (objects/errors) won't cause a TypeError; refer to
ALLOWED_CONSOLE_PREDICATES and the individual predicate functions that reference
"message".

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f0ffe168-fd1a-4ddb-8182-2d2c9a81b7e7

📥 Commits

Reviewing files that changed from the base of the PR and between 3999e3c and be84464.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (5)
  • code/addons/vitest/package.json
  • code/package.json
  • code/vitest-setup.ts
  • package.json
  • scripts/package.json

Copy link
Copy Markdown
Member

@AriPerkkio AriPerkkio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. PR title's subject should likely be Test: instead of Build: though.

@JReinhold JReinhold changed the title Build: Improve stack trace when failing on console.warn/error Test: Improve stack trace when failing on console.warn/error May 4, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
code/addons/vitest/src/node/coverage-reporter.ts (1)

11-11: Align the widened reporter type with the manager callsite cast.

Line 11 now accepts ResolvedCoverageOptions (without provider generic), but code/addons/vitest/src/node/vitest-manager.ts:64 still asserts ResolvedCoverageOptions<'v8'>. Remove the 'v8' assertion to keep types consistent and avoid masking provider assumptions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@code/addons/vitest/src/node/coverage-reporter.ts` at line 11, The type
widening on the CoverageReporter field coverageOptions now uses
ResolvedCoverageOptions without a provider generic, so update the callsite in
vitest-manager where it currently asserts ResolvedCoverageOptions<'v8'>: remove
the explicit '<"v8">' cast and use ResolvedCoverageOptions (or let inference
infer the type) so the manager and coverage-reporter types remain consistent and
do not hardcode the 'v8' provider assumption.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@code/addons/vitest/src/node/coverage-reporter.ts`:
- Line 11: The type widening on the CoverageReporter field coverageOptions now
uses ResolvedCoverageOptions without a provider generic, so update the callsite
in vitest-manager where it currently asserts ResolvedCoverageOptions<'v8'>:
remove the explicit '<"v8">' cast and use ResolvedCoverageOptions (or let
inference infer the type) so the manager and coverage-reporter types remain
consistent and do not hardcode the 'v8' provider assumption.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 76c21b41-774a-49fe-ad49-7c3d564fc8d3

📥 Commits

Reviewing files that changed from the base of the PR and between be84464 and 7b7e8d5.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (1)
  • code/addons/vitest/src/node/coverage-reporter.ts

@JReinhold JReinhold enabled auto-merge May 4, 2026 10:26
@JReinhold JReinhold merged commit 5393fa9 into next May 4, 2026
42 of 71 checks passed
@JReinhold JReinhold deleted the jeppe/improve-fail-on-error branch May 4, 2026 10:31
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Core Team Projects May 4, 2026
@github-actions github-actions Bot mentioned this pull request May 4, 2026
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-scan:human build Internal-facing build tooling & test updates ci:normal Run our default set of CI jobs (choose this for most PRs). test utilities

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants