[harness eval #34777] Fix layout.showPanel manager config#24
Open
valentinpalkovic wants to merge 1 commit into
Open
[harness eval #34777] Fix layout.showPanel manager config#24valentinpalkovic wants to merge 1 commit into
valentinpalkovic wants to merge 1 commit into
Conversation
|
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningRecipe produced no screenshots — cannot verify visible evidence. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
fe2f521 to
e537022
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff modifies layout.ts to properly handle PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6712 passed, 2 failed across 2119 suite(s) Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
80ccd7d to
745162d
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6713 passed, 1 failed across 2119 suite(s) Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
a11176d to
9de9d5b
Compare
valentinpalkovic
added
ci:verify
Verify HarnessNo verdict produced — the workflow failed before the harness ran (likely recipe-author dispatch, deny-regex, or lint). See run log for details. |
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff is primarily infrastructure/CI changes (GitHub Actions, version bumps, test file deletions) with no user-visible UI changes. The Playwright recipe tests PR-added unit tests: ❌ failed — 6713 passed, 1 failed across 2119 suite(s) Files: vitest output (last 4KB)How Playwright validated thisinterface ManagerWindow {
__STORYBOOK_ADDONS_MANAGER?: {
api?: {
setOptions?: (opts: Record<string, unknown>) => void;
getCurrentLayoutState?: () => {
layout?: {
bottomPanelHeight?: number;
rightPanelWidth?: number;
recentVisibleSizes?: { bottomPanelHeight?: number; rightPanelWidth?: number };
};
};
};
};
}
test('layout.showPanel:false via setOptions hides addon panel', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
const panel = page.locator('#storybook-panel-root');
await expect(panel).toBeVisible();
await page.waitForFunction(() => {
const w = globalThis as unknown as ManagerWindow;
return !!w.__STORYBOOK_ADDONS_MANAGER?.api?.setOptions;
}, undefined, { timeout: 15000 });
const sizesBefore = await page.evaluate(() => {
const w = globalThis as unknown as ManagerWindow;
const layout = w.__STORYBOOK_ADDONS_MANAGER?.api?.getCurrentLayoutState?.()?.layout;
return {
bottomPanelHeight: layout?.bottomPanelHeight,
rightPanelWidth: layout?.rightPanelWidth,
};
});
expect((sizesBefore.bottomPanelHeight ?? 0) > 0 || (sizesBefore.rightPanelWidth ?? 0) > 0).toBe(
true
);
await page.evaluate(() => {
const w = globalThis as unknown as ManagerWindow;
w.__STORYBOOK_ADDONS_MANAGER?.api?.setOptions?.({ layout: { showPanel: false } });
});
await expect
.pollReplay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
ad75ba9 to
099b6f7
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6713 passed, 1 failed across 2119 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('layout.showPanel:false via setOptions hides the addon panel', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const panelRoot = page.locator('#storybook-panel-root');
await expect(panelRoot).toBeVisible({ timeout: 15000 });
await page.evaluate(() => {
const w = window as unknown as {
__STORYBOOK_ADDONS_MANAGER?: { getChannel?: () => unknown };
};
const api = (
window as unknown as { __STORYBOOK_API__?: { setOptions: (o: unknown) => void } }
).__STORYBOOK_API__;
if (api && typeof api.setOptions === 'function') {
api.setOptions({ layout: { showPanel: false } });
}
void w;
});
await expect(panelRoot).toBeHidden({ timeout: 10000 });
await page.evaluate(() => {
const api = (
window as unknown as { __STORYBOOK_API__?: { setOptions: (o: unknown) => void } }
).__STORYBOOK_API__;
if (api && typeof api.setOptions === 'function') {
api.setOptions({ layout: { showPanel: true } });
}
});
await expect(panelRoot).toBeVisible({ timeout: 10000 });
await page.screenshot({ path: testInfo.outputPath('manager-panel-restored.png') });
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrorReplay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34777
branch
from
cfb3bbf to
7eb513b
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)How Playwright validated thistest('layout.showPanel option hides and restores the addon panel', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
const panelRoot = page.locator('#storybook-panel-root');
await expect(panelRoot).toBeVisible();
await page.locator('html').press('Alt+a');
await expect(panelRoot).toBeHidden();
await page.locator('html').press('Alt+a');
await expect(panelRoot).toBeVisible();
const restoredHeight = await panelRoot.evaluate((el) => el.getBoundingClientRect().height);
expect(restoredHeight).toBeGreaterThan(0);
await expect(page.locator('#sb-errordisplay')).toBeHidden();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6761 passed, 1 failed across 2129 suite(s) Files: vitest output (last 4KB)How Playwright validated thistype LayoutSnapshot = {
bottomPanelHeight: number;
rightPanelWidth: number;
navSize: number;
recentVisibleSizes: {
bottomPanelHeight: number;
rightPanelWidth: number;
navSize: number;
};
};
test('layout.showPanel option hides and restores panel sizes via setOptions', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
await page.waitForFunction(() => {
const w = window as unknown as { __STORYBOOK_ADDONS_MANAGER?: unknown };
return Boolean(w.__STORYBOOK_ADDONS_MANAGER);
});
await page.evaluate(() => {
const w = window as unknown as {
__STORYBOOK_ADDONS_MANAGER: { getChannel: () => unknown };
};
const apiHolder = window as unknown as {
__STORYBOOK_ADDONS_MANAGER: unknown;
};
void apiHolder;
void w;
});
const readLayout = async (): Promise<LayoutSnapshot> => {
return await page.evaluate(() => {
const w = window as unknown as {
__STORYBOOK_PREVIEW__?: unknown;
__STORYBOOK_ADDONS_MANAGER?: {
getChannel: () => { last?: unknown };
};
};
const stateHook = (window as unknown as {
__STORYBOOK_STATE__?: () => LayoutSnapshot;
}).__STORYBOOK_STATE__;
if (typeof stateHook === 'function') {
return stateHook();
}
const panel = document.getElementById('storybook-panel-root');
const sidebar = document.querySelector('.sidebar-coReplay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34777
branch
from
7eb513b to
ff752a8
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6941 passed, 2 failed across 2179 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('layout.showPanel:false via setOptions zeros panel sizes and preserves recent ones', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
await page.waitForFunction(() => {
const w = window as any;
return !!(w.__STORYBOOK_ADDONS_MANAGER && typeof w.__STORYBOOK_ADDONS_MANAGER.getChannel === 'function');
}, { timeout: 15000 });
const result = await page.evaluate(() => {
const mgr: any = (window as any).__STORYBOOK_ADDONS_MANAGER;
const api: any =
(window as any).__STORYBOOK_API__ ??
(window as any).api ??
mgr?.api;
if (!api || typeof api.setSizes !== 'function' || typeof api.setOptions !== 'function') {
return { ok: false, reason: 'manager-api not exposed', [REDACTED]: Object.[REDACTED](window).filter((k) => k.includes('STORY') || k.includes('api')) };
}
api.setSizes({ bottomPanelHeight: 220, rightPanelWidth: 260 });
const before = api.getState().layout;
api.setOptions({ layout: { showPanel: false } });
const afterHidden = api.getState().layout;
api.setOptions({ layout: { showPanel: true } });
const afterShown = api.getState().layout;
return {
ok: true,
before: {
bottomPanelHeight: before.bottomPanelHeight,
rightPanelWidth: before.rightPanelWidth,
},
afterHidden: {
bottomPanelHeight: afterHidden.bottomPanelHeight,
rightPanelWidth: afterHidden.rightPanelReplay: Screenshots
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Synthetic fork PR for agentic harness eval against storybookjs#34777.