Skip to content

CI: Add internal Storybook E2E job#35052

Merged
ndelangen merged 8 commits into
nextfrom
ci/internal-storybook-e2e
Jun 5, 2026
Merged

CI: Add internal Storybook E2E job#35052
ndelangen merged 8 commits into
nextfrom
ci/internal-storybook-e2e

Conversation

@ndelangen

@ndelangen ndelangen commented Jun 4, 2026

Copy link
Copy Markdown
Member

Closes #

What I did

This PR adds end-to-end test infrastructure for the internal Storybook UI (code/.storybook) and renames the existing sandbox Playwright suite directory for clarity.

Why

Today, Playwright coverage runs almost exclusively against generated sandboxes via the e2e-tests / e2e-tests-dev tasks. The internal Storybook we use for core/manager/addon development (yarn storybook:ui) has no equivalent CI gate. That makes it easy to regress internal-only demos (open-service examples, manager stories, etc.) without noticing until someone runs the UI locally.

We want a stable, documented place for internal UI regressions before landing feature-specific tests on other branches.

Why this approach

  1. Two directories, two Playwright projects

    • code/e2e-sandbox/ — specs for sandbox templates (renamed from e2e-tests/)
    • code/e2e-internal/ — specs for code/.storybook
    • Renaming avoids ambiguity: the folder name now matches what it tests. NX/task names e2e-tests and e2e-tests-dev are unchanged (they still mean “run sandbox E2E”).
  2. New e2e-tests-internal task (non-sandbox)

    • Registered in scripts/task.ts and excluded from isSandboxTask, so it does not require a --template.
    • Waits for http://localhost:6006/index.json, then runs playwright test --project=internal-storybook.
    • Mirrors the sandbox E2E pattern: CI starts Storybook in the background, then runs the task.
  3. Minimal smoke test only

    • internal-storybook.smoke.spec.ts loads a stable core template story (core-basics--basic) and asserts the preview renders.
    • Intentionally not a feature-specific test (e.g. open-service background sync) — those belong on the branches that introduce the feature, once this plumbing is on next.
  4. Playwright config

    • Removed the global testDir; each project sets its own (e2e-sandbox for setup/chromium, e2e-internal for internal-storybook).
  5. CI

    • New CircleCI job Internal storybook E2E: yarn storybook:ui (background) → port 6006 → yarn task e2e-tests-internal.
    • Vitest unit/story test globs exclude both e2e-sandbox/ and e2e-internal/ (Playwright specs are not Vitest tests).

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

Internal Storybook E2E (new path)

  1. From repo root: yarn nx compile core (or yarn task compile --no-link) if code/core/dist is stale.
  2. Terminal 1: cd code && yarn storybook:ui — wait until Storybook is listening on port 6006.
  3. Terminal 2 (repo root): yarn task e2e-tests-internal --no-link -s e2e-tests-internal
    • Or, from code/: yarn playwright test --project=internal-storybook
  4. Expect the smoke test to pass: navigates to core-basics--basic and finds the Click Me! button in the preview iframe.

Sandbox E2E rename (regression check)

  1. Confirm sandbox E2E still resolves specs from the renamed folder, e.g.:
    • yarn task e2e-tests-dev --template react-vite/default-ts --start-from auto
    • Or run a single spec: cd code && STORYBOOK_URL=… yarn playwright test --project=chromium e2e-sandbox/navigation.spec.ts with a sandbox already served.

CI

  1. On CircleCI, verify the Internal storybook E2E job completes (starts internal Storybook, runs Playwright, publishes JUnit/artifacts on failure).

No product behavior changes are expected beyond test/CI wiring; manual UI spot-check of yarn storybook:ui is optional.

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update MIGRATION.MD

CONTRIBUTING.md and CODEOWNERS paths were updated for e2e-sandbox/. No user-facing migration: task CLI names are unchanged.

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts
  • Declare whether manual QA will be needed for this PR during the next release, through qa:needed or qa:skip
  • Make sure this PR contains one of the labels below:

Suggested label: build (CI/test tooling only; no runtime Storybook behavior change).

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

Summary by CodeRabbit

  • New Features

    • Added internal Storybook UI smoke test to the e2e test suite.
  • Tests

    • Reorganized test discovery and configuration for improved maintainability.
    • Added new CI job to validate internal Storybook functionality.
  • Documentation

    • Updated project documentation to reflect test structure changes.

Wire Playwright tests against code/.storybook into CircleCI so we have a stable home for internal UI regressions, starting with a minimal smoke test on a core template story.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copilot AI review requested due to automatic review settings June 4, 2026 10:58
Clarifies that Playwright specs under e2e-sandbox target generated sandboxes, distinct from e2e-internal for the internal Storybook UI.

Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new CircleCI job to run Playwright E2E smoke coverage against the repo’s internal Storybook UI (code/.storybook) by introducing an e2e-tests-internal task and a minimal internal UI spec suite.

Changes:

  • Added a new e2e-tests-internal task that waits for the internal UI and runs Playwright with JUnit output support.
  • Introduced a new code/e2e-internal/ Playwright smoke spec and a corresponding Playwright “project”.
  • Wired a new “Internal storybook E2E” job into the generated CircleCI workflow and excluded e2e-internal from unit/story test file globbing.

Reviewed changes

Copilot reviewed 16 out of 40 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/tasks/e2e-tests-internal.ts New task to run Playwright against the internal Storybook UI after it’s available.
scripts/task.ts Registers the new task and marks it as non-sandbox-scoped.
scripts/ci/main.ts Adds the new CI job to the generated workflow config.
scripts/ci/common-jobs.ts Defines the new CircleCI job and excludes e2e-internal from unit/story globbing.
code/playwright.config.ts Adds a new Playwright project for internal Storybook tests.
code/e2e-internal/internal-storybook.smoke.spec.ts Minimal smoke test to assert the internal preview iframe renders a known story.
code/.eslintrc.js Applies Playwright ESLint rules to the new internal E2E specs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread code/playwright.config.ts Outdated
Comment thread scripts/tasks/e2e-tests-internal.ts
@ndelangen ndelangen marked this pull request as draft June 4, 2026 11:03
@ndelangen ndelangen self-assigned this Jun 4, 2026
@ndelangen ndelangen added build Internal-facing build tooling & test updates ci:normal Run our default set of CI jobs (choose this for most PRs). qa:skip Pull Requests that do not need any QA. labels Jun 4, 2026
@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds an internal Storybook Playwright smoke test suite with supporting CI infrastructure. The test verifies the internal Storybook preview iframe renders correctly. Changes include new test specs, Playwright configs, a CLI task, CircleCI job integration, updated test globs, and helper import migrations across portable-stories test files to use the consolidated e2e-sandbox utility path.

Changes

Internal Storybook E2E Testing

Layer / File(s) Summary
Test spec and Playwright/ESLint configuration
code/e2e-internal/internal-storybook.smoke.spec.ts, code/e2e-internal/playwright.config.ts, code/playwright.config.ts, code/.eslintrc.js
Internal Storybook smoke test loads a configurable Storybook URL, asserts the preview iframe is visible, and checks for a button inside the iframe. New internal Playwright config defines timeouts, CI-aware retry/parallel settings, conditional JUnit reporting, and Chrome project targeting. Main Playwright config removes top-level testDir and sets per-project directories to ./e2e-sandbox with per-project file matching. ESLint Playwright override expanded to include ./e2e-internal/*.ts.
CLI task implementation and registration
scripts/tasks/e2e-tests-internal.ts, scripts/task.ts
New e2eTestsInternal task builds Storybook URL on port 6006, sets Playwright environment variables, conditionally waits for ${STORYBOOK_URL}/index.json unless dryRun, then runs yarn playwright test with internal Playwright config and internal-storybook project. Task registered in CLI task map and marked non-sandbox to exclude template/addon selection prompts.
CircleCI job integration and test-glob updates
scripts/ci/common-jobs.ts, scripts/ci/main.ts
New internalStorybookE2e CI job restores Linux workspace, starts internal Storybook in background, waits for port 6006, runs the CLI task with JUnit output, and persists test and Playwright results. Job imported and included in CircleCI workflow for non-docs pipelines. Linux unit and story test globs updated to exclude e2e-internal paths instead of e2e-tests paths.
Sandbox globs, test import migration, and documentation
scripts/ci/sandboxes.ts, test-storybooks/portable-stories-kitchen-sink/*/e2e-tests/*.spec.ts, CODEOWNERS, CONTRIBUTING.md
Sandbox E2E test file discovery globs updated to code/e2e-sandbox/* pattern. Six portable-stories e2e test files migrate SbPage helper imports from code/e2e-tests/util to code/e2e-sandbox/util path. CODEOWNERS entry activated for /code/e2e-sandbox/ with assigned owners. Repository documentation directory tree updated to reflect e2e-sandbox instead of e2e-tests.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/tasks/e2e-tests-internal.ts`:
- Around line 15-23: The run method currently calls waitOn() unconditionally
which blocks when --dry-run is used; modify the async run({ codeDir,
junitFilename }, { dryRun, debug }) function to skip the waitOn call (and any
dependent Storybook readiness logic that follows) when dryRun is true—i.e.,
check the dryRun flag before constructing storybookUrl/await waitOn(...) and
return or proceed without waiting so the task exits immediately in dry-run mode.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 27159251-8cc1-4cd4-93b5-2c32cc3a4cf1

📥 Commits

Reviewing files that changed from the base of the PR and between 6ed450a and cd98d39.

📒 Files selected for processing (7)
  • code/.eslintrc.js
  • code/e2e-internal/internal-storybook.smoke.spec.ts
  • code/playwright.config.ts
  • scripts/ci/common-jobs.ts
  • scripts/ci/main.ts
  • scripts/task.ts
  • scripts/tasks/e2e-tests-internal.ts

Comment thread scripts/tasks/e2e-tests-internal.ts
@ndelangen ndelangen requested a review from JReinhold June 4, 2026 11:09
@ndelangen ndelangen marked this pull request as ready for review June 4, 2026 11:09
Only register the internal-storybook project when STORYBOOK_INTERNAL_E2E is set by the e2e-tests-internal task, so sandbox e2e runs are unchanged. Shorten readiness wait to 25s and skip waitOn during dry-run.

Co-authored-by: Norbert de Langen <norbert@chromatic.com>
Comment thread code/playwright.config.ts Outdated
Comment thread code/playwright.config.ts Outdated
Use playwright.internal.config.ts for e2e-internal so sandbox e2e keeps a single sandbox-only playwright.config.ts without env gating.

Co-authored-by: Norbert de Langen <norbert@chromatic.com>
Comment thread code/e2e-internal/playwright.config.ts
cursoragent and others added 2 commits June 4, 2026 12:43
Colocate playwright.config.ts with internal UI e2e specs instead of the code package root.

Co-authored-by: Norbert de Langen <norbert@chromatic.com>
Comment thread scripts/tasks/e2e-tests-internal.ts Outdated
Internal Playwright config is dedicated to e2e-internal, so a single default project is enough.

Co-authored-by: Norbert de Langen <norbert@chromatic.com>

@JReinhold JReinhold left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments, nothing blocking.

Comment thread code/e2e-sandbox/addon-a11y.spec.ts
Comment thread code/playwright.config.ts Outdated
Comment thread CONTRIBUTING.md Outdated
Use top-level testDir in sandbox playwright.config.ts again now that internal e2e has its own config. List e2e-internal in CONTRIBUTING.md repo overview.

Co-authored-by: Norbert de Langen <norbert@chromatic.com>
@ndelangen ndelangen merged commit 4ac69fb into next Jun 5, 2026
142 checks passed
@ndelangen ndelangen deleted the ci/internal-storybook-e2e branch June 5, 2026 07:37
@github-actions github-actions Bot mentioned this pull request Jun 5, 2026
20 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Internal-facing build tooling & test updates ci:normal Run our default set of CI jobs (choose this for most PRs). qa:skip Pull Requests that do not need any QA.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants