[harness eval #34775] Add Stories block support for of and docs.stories options#25
Open
valentinpalkovic wants to merge 3 commits into
Open
[harness eval #34775] Add Stories block support for of and docs.stories options#25valentinpalkovic wants to merge 3 commits into
valentinpalkovic wants to merge 3 commits into
Conversation
|
Verify HarnessVerdict: PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
fe2f521 to
e537022
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6711 passed, 1 failed across 2118 suite(s) Files: vitest output (last 4KB)Replay: |
valentinpalkovic
added
ci:verify
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
80ccd7d to
745162d
Compare
Verify HarnessVerdict: Reason: Compile output (last 4KB)PR-added unit tests: ❌ failed — 6711 passed, 1 failed across 2118 suite(s) Files: vitest output (last 4KB)Replay: |
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
a11176d to
9de9d5b
Compare
valentinpalkovic
added
ci:verify
Verify HarnessNo verdict produced — the workflow failed before the harness ran (likely recipe-author dispatch, deny-regex, or lint). See run log for details. |
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6711 passed, 1 failed across 2118 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('addon-docs Stories block renders example/button autodocs', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/docs/example-button--docs`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const previewIframe = recipe.previewIframe();
const docsRoot = previewIframe.locator('#storybook-docs');
await expect(docsRoot).toBeVisible();
const storiesHeading = previewIframe.getByRole('heading', { name: /^stories$/i });
await expect(storiesHeading).toBeVisible({ timeout: 15000 });
const storyHeadings = previewIframe.locator('#storybook-docs h3');
await expect(storyHeadings.first()).toBeVisible({ timeout: 15000 });
const headingCount = await storyHeadings.count();
expect(headingCount).toBeGreaterThan(0);
await previewIframe.locator('body').screenshot({
path: testInfo.outputPath('docs-stories-block.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
ad75ba9 to
099b6f7
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff is primarily a large cleanup/revert of build infrastructure, CI config, and version numbers. While it does modify the Stories block component ( PR-added unit tests: ❌ failed — 6711 passed, 1 failed across 2118 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('Stories block renders autodocs with multiple stories without errors', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/docs/example-button--docs`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = recipe.previewIframe();
const docsRoot = previewIframe.locator('#storybook-docs');
await expect(docsRoot).toBeVisible();
const storiesHeading = previewIframe.getByRole('heading', { name: /^stories$/i });
await expect(storiesHeading).toBeVisible({ timeout: 15000 });
const secondary = previewIframe.getByRole('heading', { name: /^secondary$/i });
const large = previewIframe.getByRole('heading', { name: /^large$/i });
await expect(secondary).toBeVisible({ timeout: 10000 });
await expect(large).toBeVisible({ timeout: 10000 });
await previewIframe.locator('#storybook-docs').screenshot({
path: testInfo.outputPath('docs-stories-block.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34775
branch
from
35037bc to
59a9972
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6759 passed, 1 failed across 2128 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('Stories block supports of + docs.stories.* parameters', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(
`${baseURL}/?path=/story/blocks-stories--with-docs-stories-parameters`
);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const preview = sb.previewIframe();
const previewRoot = sb.previewRoot();
await expect(previewRoot).toBeVisible();
const configuredHeading = preview.getByRole('heading', { name: 'Configured stories' });
await expect(configuredHeading).toBeVisible();
await expect(preview.getByRole('heading', { name: /^Secondary$/ })).toBeVisible();
await expect(preview.getByRole('heading', { name: /^Tertiary$/ })).toBeVisible();
await expect(preview.getByRole('heading', { name: /^Primary$/ })).toHaveCount(0);
await previewRoot.screenshot({
path: testInfo.outputPath('with-docs-stories-parameters.png'),
});
await page.goto(`${baseURL}/?path=/story/blocks-stories--of-csf-file`);
await sb.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const ofPreview = sb.previewIframe();
await expect(ofPreview.getByRole('heading', { name: 'Stories' })).toBeVisible();
await expect(ofPreview.getByRole('heading', { name: /^Primary$/ })).toBeVisible();
await sb.previewRoot().screenshot({
path: testInfo.outputPath('of-csf-file.png'),
});
await page.goto(
`${baseURL}/?path=/story/blocks-stories--without-primary-story`
);
await sb.waitUntilLoadedReplay: Screenshots
|
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6759 passed, 1 failed across 2128 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('Stories block renders with docs.stories parameters via of prop', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
await page.goto(
`${baseURL}/?path=/story/addons-docs-blocks-stories--with-docs-stories-parameters`
);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const configuredTitle = previewIframe.getByText('Configured stories', { exact: false });
await expect(configuredTitle).toBeVisible({ timeout: 15_000 });
await previewIframe.locator('body').screenshot({
path: testInfo.outputPath('stories-block-with-parameters.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34775
branch
from
59a9972 to
da9cfe3
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6939 passed, 2 failed across 2178 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('Stories doc block renders autodocs page with stories list', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/docs/example-button--docs`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const docsRoot = previewIframe.locator('#storybook-docs');
await expect(docsRoot).toBeVisible();
const storiesHeading = previewIframe.getByRole('heading', { name: /^stories$/i }).first();
await expect(storiesHeading).toBeVisible({ timeout: 15000 });
const docsStoryPreviews = previewIframe.locator('.docs-story, [class*="docs-story"]');
await expect(docsStoryPreviews.first()).toBeVisible({ timeout: 15000 });
const previewCount = await docsStoryPreviews.count();
expect(previewCount).toBeGreaterThan(0);
await previewIframe.locator('body').screenshot({
path: testInfo.outputPath('docs-stories-block.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
expect(filterConsoleErrors(consoleErrors)).toEqual([]);
});Replay: Screenshots
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Synthetic fork PR for agentic harness eval against storybookjs#34775.