Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storybook: Set up local visual regression testing #43393

Merged
merged 11 commits into from
Sep 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ phpcs.xml
phpunit.xml
phpunit-watcher.yml
.tool-versions
test/storybook-playwright/test-results
test/storybook-playwright/specs/__snapshots__
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -290,11 +290,13 @@
"storybook:build": "build-storybook -c ./storybook -o ./storybook/build",
"prestorybook:dev": "npm run build:packages",
"storybook:dev": "concurrently \"npm run dev:packages\" \"start-storybook -c ./storybook -p 50240\"",
"storybook:e2e:dev": "concurrently \"npm run dev:packages\" \"start-storybook -c test/storybook-playwright/storybook -p 50241\"",
"test": "npm-run-all lint test:unit",
"test:create-block": "bash ./bin/test-create-block.sh",
"test:e2e": "wp-scripts test-e2e --config packages/e2e-tests/jest.config.js",
"test:e2e:debug": "wp-scripts --inspect-brk test-e2e --config packages/e2e-tests/jest.config.js --puppeteer-devtools",
"test:e2e:playwright": "playwright test --config test/e2e/playwright.config.ts",
"test:e2e:storybook": "playwright test --config test/storybook-playwright/playwright.config.ts",
"test:e2e:watch": "npm run test:e2e -- --watch",
"test:performance": "wp-scripts test-e2e --config packages/e2e-tests/jest.performance.config.js",
"test:php": "npm-run-all lint:php test:unit:php",
Expand Down
47 changes: 47 additions & 0 deletions packages/components/src/font-size-picker/stories/e2e/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/**
* WordPress dependencies
*/
import { useState } from '@wordpress/element';

/**
* Internal dependencies
*/
import FontSizePicker from '../..';

export default {
title: 'Components/FontSizePicker',
component: FontSizePicker,
};

const FontSizePickerWithState = ( { initialValue, ...props } ) => {
const [ fontSize, setFontSize ] = useState( initialValue );
return (
<FontSizePicker
{ ...props }
value={ fontSize }
onChange={ setFontSize }
/>
);
};

export const Default = FontSizePickerWithState.bind( {} );
Default.args = {
fontSizes: [
{
name: 'Small',
slug: 'small',
size: 12,
},
{
name: 'Normal',
slug: 'normal',
size: 16,
},
{
name: 'Big',
slug: 'big',
size: 26,
},
],
initialValue: 16,
};
25 changes: 25 additions & 0 deletions packages/components/src/popover/stories/e2e/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/**
* WordPress dependencies
*/
import { useState } from '@wordpress/element';

/**
* Internal dependencies
*/
import Popover from '../..';

export default {
title: 'Components/Popover',
component: Popover,
};

export const Default = () => {
const [ isVisible, setIsVisible ] = useState( false );

return (
<button onClick={ () => setIsVisible( ( state ) => ! state ) }>
Toggle Popover!
{ isVisible && <Popover>Popover is toggled!</Popover> }
</button>
);
};
29 changes: 29 additions & 0 deletions test/storybook-playwright/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Storybook Playwright Tests

This is currently set up for testing visual regressions in the `components` package. The tests do not run on CI, and is meant as a testing tool for local development.

## How to run

First, build and serve the E2E Storybook.
mirka marked this conversation as resolved.
Show resolved Hide resolved

```sh
npm run storybook:e2e:dev
```

You are now ready to run the tests. The first run will generate the reference images, and subsequent runs will compare against them. (On the first run, you may be prompted to first install Playwright. If so, follow the instructions.)

```sh
npm run test:e2e:storybook
```

To update the reference images, pass the `--update-snapshots` flag.

```sh
npm run test:e2e:storybook -- --update-snapshots
```

## How to write tests

Any stories matching the glob patterns listed in the [E2E Storybook config](./storybook/main.js) will be included in the special build. Note that these are exclusive fixtures for our tests, and are separate from the stories included in the [main Storybook build](../../storybook/main.js) to be [published online](https://wordpress.github.io/gutenberg/).

The Playwright test files live in the [`specs`](./specs/) folder. See the [E2E Tests README](../e2e/README.md) for general best practices.
13 changes: 13 additions & 0 deletions test/storybook-playwright/playwright.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/**
* External dependencies
*/
import { PlaywrightTestConfig } from '@playwright/test';

const config: PlaywrightTestConfig = {
outputDir: 'test-results/output',
reporter: [
[ 'html', { open: 'on-failure', outputFolder: 'test-results/report' } ],
],
};

export default config;
37 changes: 37 additions & 0 deletions test/storybook-playwright/specs/font-size-picker.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/**
* WordPress dependencies
*/
import { test, expect } from '@wordpress/e2e-test-utils-playwright';

/**
* Internal dependencies
*/
import { gotoStoryId } from '../utils';

const waitUntilButtonHighlightStable = async ( page ) => {
const handle = await page.waitForSelector(
'[aria-label="Font size"] > div[role=presentation]'
);

await handle?.waitForElementState( 'stable' );

return handle;
};

test.describe.parallel( 'FontSizePicker', () => {
test.beforeEach( async ( { page } ) => {
await gotoStoryId( page, 'components-fontsizepicker--default' );
} );

// This isn't a meaningful test, just some example code to demonstrate a way to
// wait until a certain element has finished animating.
// We can remove it once we have real tests.
test( 'with value', async ( { page } ) => {
const button = await page.locator( 'button[aria-label="Normal"]' );

await waitUntilButtonHighlightStable( page );

expect( button ).toHaveAttribute( 'aria-checked', 'true' );
expect( await page.screenshot() ).toMatchSnapshot();
} );
} );
25 changes: 25 additions & 0 deletions test/storybook-playwright/specs/popover.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/**
* WordPress dependencies
*/
import { test, expect } from '@wordpress/e2e-test-utils-playwright';

/**
* Internal dependencies
*/
import { gotoStoryId } from '../utils';

test.describe( 'Popover', () => {
// This isn't a meaningful test, just some example code.
// We can remove it once we have real tests.
test( 'should render', async ( { page } ) => {
await gotoStoryId( page, 'components-popover--default', {
decorators: { marginChecker: 'show' },
} );

await page.click( 'role=button' );
const popover = await page.waitForSelector( '.components-popover' );
await popover.waitForElementState( 'stable' );

expect( await page.screenshot() ).toMatchSnapshot();
} );
} );
14 changes: 14 additions & 0 deletions test/storybook-playwright/storybook/main.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* Internal dependencies
*/
const baseConfig = require( '../../../storybook/main' );

const config = {
...baseConfig,
addons: [ '@storybook/addon-toolbars' ],
stories: [
'../../../packages/components/src/**/stories/e2e/*.@(js|tsx|mdx)',
],
};

module.exports = config;
1 change: 1 addition & 0 deletions test/storybook-playwright/storybook/preview.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from '../../../storybook/preview';
6 changes: 6 additions & 0 deletions test/storybook-playwright/storybook/webpack.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
/**
* Internal dependencies
*/
const baseConfig = require( '../../../storybook/webpack.config' );

module.exports = baseConfig;
40 changes: 40 additions & 0 deletions test/storybook-playwright/utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/**
* External dependencies
*/
import type { Page } from '@playwright/test';

const STORYBOOK_PORT = '50241';

type Decorators = {
css?: 'none' | 'basic' | 'wordpress';
direction?: 'ltr' | 'rtl';
marginChecker?: 'show' | 'hide';
};
type Options = { decorators?: Decorators };

const buildDecoratorString = ( decorators: Decorators = {} ) => {
const decoratorParamStrings = Object.entries( decorators ).map(
( keyValue ) => keyValue.join( ':' )
);
return decoratorParamStrings.join( ';' );
};

export const gotoStoryId = (
page: Page,
storyId: string,
{ decorators }: Options = {}
) => {
const params = new URLSearchParams();
const decoratorString = buildDecoratorString( decorators );

if ( decoratorString ) {
params.set( 'globals', decoratorString );
}

params.set( 'id', storyId );

page.goto(
`http://localhost:${ STORYBOOK_PORT }/iframe.html?${ params.toString() }`,
{ waitUntil: 'load' }
);
};