[harness eval #34687] fix: narrow CSF factory decorator context types#33
[harness eval #34687] fix: narrow CSF factory decorator context types#33valentinpalkovic wants to merge 1 commit into
Conversation
|
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningRecipe produced no screenshots — cannot verify visible evidence. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
fe2f521 to
e537022
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff is primarily a type-level change to CSF factory decorator context (narrowing TS types for parameters/globals in story.ts) with no runtime behavior changes. The Playwright recipe explicitly confirms this is a pure type-level change with no observable UI differences, and the screenshots show a functional Button story rendering correctly—which is exactly what the recipe validates (that touching the types doesn't break story rendering). The screenshots contain the expected UI but cannot demonstrate the absence or presence of invisible type narrowing. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff consists entirely of non-user-visible changes: TypeScript type narrowing (TypedParameters, TypedGlobals), CI/build configuration simplifications, and test additions. The Playwright recipe explicitly acknowledges this is a type-only change with no runtime behavior changes, confirming the visible UI (a rendered button in the preview) should be identical before and after. The screenshots show the button rendering correctly, which is the expected outcome—not evidence of a visible change. PR-added unit tests: ✅ passed — 6713 passed, 0 failed across 2118 suite(s) Files: Replay: Screenshots
|
valentinpalkovic
added
ci:verify
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
80ccd7d to
745162d
Compare
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff contains pure TypeScript type-narrowing changes (TypedParameters/TypedGlobals type aliases, cast fix in portable-stories.ts) and infrastructure/documentation changes (GitHub Actions workflow simplifications, recipe authoring guide cleanup, script refactoring). These are not user-visible changes. The screenshots show a Storybook preview with a Button component, which correctly renders and contains no errors, confirming the types work at runtime. However, type narrowing itself produces no observable UI difference. PR-added unit tests: ✅ passed — 6713 passed, 0 failed across 2118 suite(s) Files: Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
a11176d to
9de9d5b
Compare
valentinpalkovic
added
ci:verify
Verify HarnessNo verdict produced — the workflow failed before the harness ran (likely recipe-author dispatch, deny-regex, or lint). See run log for details. |
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff is primarily composed of non-user-visible changes: CI/workflow configuration files, changelog entries, version number bumps in package.json files, test file deletions, and removal of package manager utility functions. While the Playwright recipe tests sidebar rendering and preview functionality, the actual user-visible changes (if any) in the diff are unclear—the comments mention Tree.tsx and ReviewChangesButton refactors, but the provided diff excerpt is truncated and doesn't show the relevant component changes. PR-added unit tests: ✅ passed — 6713 passed, 0 failed across 2118 suite(s) Files: How Playwright validated thistest('sidebar Tree renders consolidated status icons without runtime errors', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
const sidebar = page.locator('.sidebar-container');
await expect(sidebar).toBeVisible();
const exampleGroup = page.locator('[data-item-id="example"]');
await expect(exampleGroup).toBeVisible();
await exampleGroup.click();
const buttonComponent = page.locator('[data-item-id="example-button"]');
await expect(buttonComponent).toBeVisible();
await buttonComponent.click();
const primaryStory = page.locator('[data-item-id="example-button--primary"]');
await expect(primaryStory).toBeVisible();
const secondaryStory = page.locator('[data-item-id="example-button--secondary"]');
await expect(secondaryStory).toBeVisible();
await secondaryStory.click();
await expect(page).toHaveURL(/example-button--secondary/);
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewRoot = recipe.previewRoot();
await expect(previewRoot).toBeVisible();
await sidebar.screenshot({
path: testInfo.outputPath('sidebar.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
ad75ba9 to
099b6f7
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ✅ passed — 6713 passed, 0 failed across 2118 suite(s) Files: How Playwright validated thistest('nextjs-vite sandbox renders a story without runtime errors', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/`);
const sidebar = page.locator('.sidebar-container');
await expect(sidebar).toBeVisible({ timeout: 30000 });
const firstStoryLink = page
.locator('a[data-nodetype="story"], a[data-nodetype="document"]')
.first();
if ((await firstStoryLink.count()) === 0) {
const firstGroup = page
.locator('button[data-nodetype="group"], button[data-nodetype="component"]')
.first();
await firstGroup.click();
}
const storyLink = page
.locator('a[data-nodetype="story"], a[data-nodetype="document"]')
.first();
await expect(storyLink).toBeVisible({ timeout: 15000 });
await storyLink.click();
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewRoot = recipe.previewRoot();
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
await page.locator('.sidebar-container').screenshot({
path: testInfo.outputPath('sidebar.png'),
});
await page.frameLocator('#storybook-preview-iframe').locator('body').screenshot({
path: testInfo.outputPath('preview.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'applReplay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34687
branch
from
aeda830 to
5176877
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ✅ passed — 6761 passed, 0 failed across 2128 suite(s) Files: How Playwright validated thistest('CSF factory story boots with parameters/globals context after type narrowing', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/core-decorators--basics`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const previewIframe = sb.previewIframe();
const previewRoot = sb.previewRoot();
await expect(previewRoot).toBeVisible();
const ctxShape = await previewIframe.locator('body').evaluate(() => {
const w = window as unknown as {
__STORYBOOK_PREVIEW__?: {
storyStore?: unknown;
};
};
const preview = w.__STORYBOOK_PREVIEW__;
const hasStore = !!preview && typeof preview === 'object' && 'storyStore' in preview;
const store = preview?.storyStore as
| { lastRenderedStory?: { parameters?: unknown; globals?: unknown } }
| undefined;
const last = store?.lastRenderedStory;
return {
hasStore,
parametersIsObject:
last == null ? hasStore : typeof last.parameters === 'object' && last.parameters !== null,
globalsIsObject:
last == null ? hasStore : typeof last.globals === 'object' && last.globals !== null,
};
});
expect(ctxShape.hasStore).toBe(true);
expect(ctxShape.parametersIsObject).toBe(true);
expect(ctxShape.globalsIsObject).toBe(true);
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await teReplay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ✅ passed — 6761 passed, 0 failed across 2128 suite(s) Files: How Playwright validated thistest('csf type-narrowing diff does not break story boot or portable-stories runtime', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/core-basics--default`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
await page.goto(`${baseURL}/?path=/story/core-args--default`);
await sb.waitUntilLoaded();
await expect(errorDisplay).toBeHidden();
await expect(previewRoot).toBeVisible();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
const relevantConsoleErrors = consoleErrors.filter(
(msg) => !/ERR_INTERNET_DISCONNECTED|Failed to load resource/i.test(msg),
);
expect(relevantConsoleErrors).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34687
branch
from
5176877 to
8fec210
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6941 passed, 1 failed across 2178 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('csf decorator context typing change does not break story boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeAttached();
const button = previewIframe.getByRole('button', { name: /button/i });
await expect(button).toBeVisible();
await page.goto(`${baseURL}/?path=/story/example-button--secondary`);
await sb.waitUntilLoaded();
await expect(errorDisplay).toBeHidden();
const secondaryButton = previewIframe.getByRole('button', { name: /button/i });
await expect(secondaryButton).toBeVisible();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
expect(filterConsoleErrors(consoleErrors)).toEqual([]);
});Replay: Screenshots
|
2 participants
Synthetic fork PR for agentic harness eval against storybookjs#34687.