[harness eval #34686] Addon-Vitest: Support non-ASCII project paths#34
Open
valentinpalkovic wants to merge 1 commit into
Open
[harness eval #34686] Addon-Vitest: Support non-ASCII project paths#34valentinpalkovic wants to merge 1 commit into
valentinpalkovic wants to merge 1 commit into
Conversation
|
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningRecipe produced no screenshots — cannot verify visible evidence. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
fe2f521 to
e537022
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff's primary user-visible change is in addon-vitest's path decoding logic for non-ASCII project paths, which is a build-time Vite plugin behavior exercised by unit tests rather than runtime UI. The Playwright recipe is a smoke test that verifies internal-ui boots and preview renders cleanly, not a test designed to observe the path-decoding change itself. The screenshots show a Button story rendering correctly, but this does not directly demonstrate the decodeURI() path-matching fix since that code path is not exercised during normal story interaction. PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6712 passed, 1 failed across 2120 suite(s) Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
added
ci:verify
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
80ccd7d to
745162d
Compare
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff is primarily a Node-side change to the vitest-plugin's URL decoding logic and GitHub Actions workflow/script simplifications. The Playwright recipe confirms internal-ui boots cleanly and renders a button story, but the specific behavioral change (handling URL-encoded paths in vitest-plugin transforms) has no observable in-browser effect. The screenshots show a properly rendered button component, which is consistent with the change not breaking anything, but cannot directly validate the URL-decoding logic itself (which is tested via unit tests instead). PR-added unit tests: ❌ failed — 6712 passed, 1 failed across 2120 suite(s) Files: vitest output (last 4KB)Replay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
3 times, most recently
from
a11176d to
9de9d5b
Compare
valentinpalkovic
added
ci:verify
Verify HarnessNo verdict produced — the workflow failed before the harness ran (likely recipe-author dispatch, deny-regex, or lint). See run log for details. |
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: Evidence (vision-check, Vision reasoningThe diff is primarily infrastructure/CI changes (GitHub Actions workflows, version bumps, package.json updates, test file deletions/modifications) with minimal user-visible UI changes. The Playwright recipe comment mentions a SweepIcon swap in ReviewChangesButton, but the provided screenshots show only the Storybook manager sidebar and a test completion notification—neither clearly displays the specific icon change or the clear button with the new icon. The screenshots are insufficient to confirm the UI diff is present. PR-added unit tests: ❌ failed — 6712 passed, 1 failed across 2120 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('ReviewChangesButton clear button renders SweepIcon after change-detection trigger', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
await page.locator('.sidebar-container').screenshot({
path: testInfo.outputPath('sidebar-baseline.png'),
});
const controlsTab = page.getByRole('tab', { name: /controls/i });
await controlsTab.click();
const labelInput = page.locator('input[name="label"], textarea[name="label"]').first();
await expect(labelInput).toBeVisible({ timeout: 10000 });
await labelInput.fill('Verify harness saved this');
const saveButton = page.getByRole('button', {
name: /save changes to story|update story/i,
});
await expect(saveButton).toBeVisible({ timeout: 10000 });
await saveButton.click();
const reviewToggle = page.getByRole('switch', { name: /review.+stories/i });
await expect(reviewToggle).toBeVisible({ timeout: 20000 });
await reviewToggle.click();
const clearButton = page.getByRole('button', { name: /^clear$/i });
await expect(clearButton).toBeVisible({ timeout: 10000 });
const clearIcon = clearButton.locator('svg');
await expect(clearIcon).toBeVisible();
await page.locator('.sidebar-container').screenshot({
path: testInfo.outputPath('sidebar-with-clear-button.png'),
});
await clearButton.screenshot({
path: testInfo.outputPath('clear-buReplay: Screenshots
|
valentinpalkovic
force-pushed
the
next
branch
2 times, most recently
from
ad75ba9 to
099b6f7
Compare
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6712 passed, 1 failed across 2120 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('sidebar Tree status icon and ReviewChangesButton render without runtime errors', async ({
page,
}, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(
`${baseURL}/?path=/story/sidebar-tree--with-new`,
);
const recipe = new RecipePage(page, expect);
await recipe.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
const previewIframe = recipe.previewIframe();
const previewRoot = recipe.previewRoot();
await expect(previewRoot).toBeVisible();
const changeStatusButton = previewIframe
.locator('[data-testid="tree-change-status-button"]')
.first();
await expect(changeStatusButton).toBeVisible({ timeout: 15000 });
await previewRoot.screenshot({
path: testInfo.outputPath('tree-with-new.png'),
});
await page.goto(
`${baseURL}/?path=/story/sidebar-tree--with-modified`,
);
await recipe.waitUntilLoaded();
await expect(page.locator('#sb-errordisplay')).toBeHidden();
await expect(previewRoot).toBeVisible();
const modifiedStatusButton = previewIframe
.locator('[data-testid="tree-change-status-button"]')
.first();
await expect(modifiedStatusButton).toBeVisible({ timeout: 15000 });
await previewRoot.screenshot({
path: testInfo.outputPath('tree-with-modified.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});Replay: |
valentinpalkovic
force-pushed
the
try-pr-34686
branch
from
366a43c to
d9f2562
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — vitest exited 1 without writing a JSON report (likely setup error); see Action log Files: vitest output (last 4KB)How Playwright validated thistest('addon-vitest plugin change does not break internal-ui boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = page.frameLocator('#storybook-preview-iframe');
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const button = previewIframe.getByRole('button', { name: /button/i }).first();
await expect(button).toBeVisible();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
added
ci:verify
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6760 passed, 1 failed across 2130 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('addon-vitest plugin diff: internal-ui boots cleanly', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = page.frameLocator('#storybook-preview-iframe');
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
await previewIframe.locator('body').screenshot({
path: testInfo.outputPath('preview.png'),
});
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
});Replay: Screenshots
|
valentinpalkovic
force-pushed
the
try-pr-34686
branch
from
d9f2562 to
7230baf
Compare
Verify HarnessVerdict: Reason: PR-added unit tests: ❌ failed — 6940 passed, 2 failed across 2180 suite(s) Files: vitest output (last 4KB)How Playwright validated thistest('addon-vitest plugin path-decoding change does not regress storybook boot', async ({ page }, testInfo) => {
const pageErrors: string[] = [];
const consoleErrors: string[] = [];
page.on('pageerror', (err) => {
pageErrors.push(err.stack ?? err.message ?? String(err));
});
page.on('console', (msg) => {
if (msg.type() === 'error') {
consoleErrors.push(msg.text());
}
});
const baseURL =
process.env.STORYBOOK_URL ?? testInfo.project.use.baseURL ?? 'http://localhost:6006';
try {
await page.goto(`${baseURL}/?path=/story/example-button--primary`);
const sb = new RecipePage(page, expect);
await sb.waitUntilLoaded();
const errorDisplay = page.locator('#sb-errordisplay');
await expect(errorDisplay).toBeHidden();
const previewIframe = sb.previewIframe();
const previewRoot = previewIframe.locator('#storybook-root, #root');
await expect(previewRoot).toBeVisible();
const childCount = await previewRoot.evaluate((el) => el.childElementCount);
expect(childCount).toBeGreaterThan(0);
const primaryButton = previewIframe.getByRole('button', { name: /button/i }).first();
await expect(primaryButton).toBeVisible();
} finally {
await testInfo.attach('pageErrors', {
body: JSON.stringify(pageErrors),
contentType: 'application/json',
});
await testInfo.attach('consoleErrors', {
body: JSON.stringify(consoleErrors),
contentType: 'application/json',
});
}
expect(filterPageErrors(pageErrors)).toEqual([]);
expect(filterConsoleErrors(consoleErrors)).toEqual([]);
});Replay: Screenshots
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Synthetic fork PR for agentic harness eval against storybookjs#34686.