[harness eval #34765] Vitest: Reset playwright cursor position to avoid hover bug#26
[harness eval #34765] Vitest: Reset playwright cursor position to avoid hover bug#26valentinpalkovic wants to merge 4 commits into
Conversation
|
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningRecipe produced no screenshots — cannot verify visible evidence. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
fe2f521 to
e537022
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff's core changes are non-visual: they modify Vitest plugin setup hooks (resetMousePositionBeforeTests, resetMousePosition command), CI workflow paths, and screenshot directory resolution logic. The Playwright recipe explicitly documents that these are 'node-side Vitest plugin code paths' with 'no UI surface in Storybook itself.' The screenshots show Storybook loading correctly with the Button story and sidebar visible, which is consistent with the smoke-test intent, but the diff's functional changes (mouse position reset, Vitest caching, directory path simplification) have no observable PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff is primarily a Vitest runtime change (mouse cursor reset in browser mode, test infrastructure simplifications, and CI workflow changes). The Playwright recipe explicitly tests only that 'the vitest addon panel registers and example story boots cleanly' — a smoke test verifying no module-resolution errors or page crashes. The screenshots show the Storybook manager and a Button example story rendering correctly, which is consistent with the addon booting cleanly, but the actual behavioral change (mouse.move(-1000,-1000) executed before each Vitest browser-mode test) is non-observable in PR-added unit tests: ✅ passed — 6716 passed, 0 failed across 2119 suite(s) Files: Replay: Screenshots
|
valentinpalkovic
added
ci:verify
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
80ccd7d to
745162d
Compare
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff consists entirely of non-user-visible changes: vitest browser-mode setup code, CI workflow simplifications, documentation removals, and test infrastructure updates. The Playwright recipe explicitly documents that these changes are 'build-time/test-runner-only code paths' with 'NO runtime surface' observable in the preview. The screenshots show a working Storybook preview with an example button, confirming baseline boot health, but cannot reveal vitest setup-file or browser command registration state. PR-added unit tests: ❌ failed — 6715 passed, 1 failed across 2119 suite(s) Files: vitest output (last 4KB)Replay: |
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
a11176d to
9de9d5b
Compare
valentinpalkovic
added
ci:verify
Verify HarnessNo verdict produced — the workflow failed before the harness ran (likely recipe-author dispatch, deny-regex, or lint). See run log for details. |
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff is primarily composed of version downgrades, changelog removals, and internal refactoring (BUNProxy changes, vitest plugin updates, workflow/script modifications). While there are sidebar-related changes mentioned in the recipe comments (Tree.tsx status icon logic and ReviewChangesButton clear button icon swap), the actual user-visible UI changes in the diff are minimal and not clearly identifiable in the provided screenshots. The screenshots show a sidebar with story hierarchy but don't provide sufficient resolution or focus to confirm the specific icon changes or status logic modifications. PR-added unit tests: ✅ passed — 6716 passed, 0 failed across 2119 suite(s) Files: How Playwright validated thistest('sidebar status icons + review-changes Clear button render correctly', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const exampleGroup = page.locator('[data-item-id="example"]');
if (await exampleGroup.count()) {
await exampleGroup.first().click().catch(() => {});
}
const primaryLeaf = page.locator('[data-item-id="example-button--primary"]');
await expect(primaryLeaf).toBeVisible();
await page.locator('.sidebar-container').screenshot({
path: testInfo.outputPath('sidebar-baseline.png'),
});
const controlsTab = page.getByRole('tab', { name: /controls/i });
if (await controlsTab.count()) {
await controlsTab.first().click();
const labelInput = page
.locator('input[name="label"], textarea[name="label"]')
.first();
if (await labelInput.count()) {
await labelInput.fill('Verify harness saved this');
const saveButton = page.getByRole('button', {
name: /save changes to story|update story/i,
});
if (await saveButton.count()) {
await saveButton.first().click();
const reviewToggle = page.getByRole('switch', { name: /review.+stories/i });
await reviewToggle
.waitFor({ state: 'visible', timeout: 15000 })
.catch(() => {});
if (await reviewToggle.count()) {
await reviewToggle.firReplay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
ad75ba9 to
099b6f7
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Evidence (vision-check, Vision reasoningThe diff is primarily composed of non-UI changes: CI/workflow configuration removals, version downgrades, test file deletions, and internal refactoring. The recipe's Playwright test targets a UI change (SweepIcon swap in ReviewChangesButton), but the diff shown does not contain any visible UI code changes to support this—the actual manager/sidebar code modifications are not included in the provided diff excerpt. PR-added unit tests: ✅ passed — 6716 passed, 0 failed across 2119 suite(s) Files: How Playwright validated this//
//
test('sidebar Tree renders without runtime errors after status-icon refactor', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const sidebar = page.locator('.sidebar-container');
await expect(sidebar).toBeVisible();
const exampleGroup = page.locator('[data-item-id="example"]');
if (await exampleGroup.count()) {
await exampleGroup.click();
}
const selectedLeaf = page.locator(
'[data-item-id="example-button--primary"][data-selected="true"]',
);
await expect(selectedLeaf).toBeVisible();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const previewIframe = page.frameLocator('#storybook-preview-iframe');
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
await sidebar.screenshot({
path: testInfo.outputPath('sidebar-tree.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34765
branch
from
540f19a to
6fb2a28
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)How Playwright validated thistest('vitest addon setup-file changes do not break internal-ui boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = page.frameLocator('#storybook-preview-iframe');
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ✅ passed — 6764 passed, 0 failed across 2129 suite(s) Files: How Playwright validated thistest('addon-vitest setup-file changes do not break internal-ui boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/core-args--basic`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewRoot = sb.previewRoot();
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
await page.goto(`${baseURL}/?path=/story/core-decorators--basic`);
await sb.waitUntilLoaded();
await expect(errorDisplay).toBeHidden();
await expect(sb.previewRoot()).toBeVisible();
await sb.previewIframe().locator('body').screenshot({
path: testInfo.outputPath('preview.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34765
branch
from
6fb2a28 to
0d45949
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6944 passed, 1 failed across 2179 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('addon-vitest setup-file change does not break Storybook boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
await expect(page.locator('.sidebar-container')).toBeVisible();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
expect(filterConsoleErrors(consoleErrors)).toEqual([]);
});Replay: Screenshots
|
Synthetic fork PR for agentic harness eval against storybookjs#34765.