Repository navigation
fix(visual-proof): outputPath escape + networkidle timeout (W8-14) - #194
Merged
Ghenghis merged 1 commit intoMay 10, 2026
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
3 tasks done
Two surgical fixes to the W6-6 visual-proof harness so it produces real
match/diff data against Images-GUI/.
1. outputPath bug. snapshotPathTemplate "{arg}{ext}" + spec segments that
walked UP from the test root into Images-GUI/ tripped Playwright's
"outputPath is not allowed outside of the parent directory" safety
check. Fix: new tests/visual/global-setup.ts mirrors Images-GUI/ into
tests/visual/__refs__/ on every run (idempotent, size+mtime check).
The spec now emits forward-only segments into __refs__/, fully inside
the test root. Images-GUI/ remains the single source of truth; the
mirror is one-way and gitignored.
2. networkidle timeout bug. The Hermes3D SPA's long-poll + SSE channels
(recovery_controller, agent updates, A2A) keep the network busy past
Playwright's 30s default. Per Playwright's docs, networkidle is
DISCOURAGED for this app shape. Fix: drop networkidle from
waitForStable(); rely on domcontentloaded + 500ms quiet + the
per-target wait_test_id toBeVisible(15s) assertion.
Re-run delta vs PR #189:
before: 0 match / 0 diff / 11 error / 20 skipped
after: 0 match / 11 diff / 0 error / 20 skipped (all 11 live targets
now produce real diff data; uniform ~0.29 ratio diagnosed as
1920x1080 viewport vs 1536x1024 reference, owned by W6-3 to
fix).
Stacks on PR #188 (W8-12 Vite Fast-Refresh fix). Builds on PR #189
(W6-6 first-run JSON summary).
Sources cited in code + handoff doc:
- https://playwright.dev/docs/api/class-testconfig#test-config-snapshot-path-template
- https://playwright.dev/docs/api/class-page#page-wait-for-load-state
Files
03_implementation/ui/playwright.visual.config.ts
03_implementation/ui/tests/visual/visual-proof.spec.ts
03_implementation/ui/tests/visual/global-setup.ts (NEW)
03_implementation/ui/tests/visual/__refs__/.gitkeep (NEW)
03_implementation/ui/tests/visual/.gitignore (NEW)
03_implementation/docs/evidence/visual_proof_2026-05-09/summary.json
03_implementation/docs/handoffs/W8-14_VISUAL_PROOF_HARNESS_FIX_2026-05-09.md (NEW)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ghenghis
force-pushed
the
claude/w8-14-visual-proof-harness-fix
branch
from
May 10, 2026 05:16
fb83e49 to
66aee96
Compare
Ghenghis
merged commit May 10, 2026
b2f7c54
into
feat/hermes3d-7-complete-gui-repo-wiring
3 checks passed
Ghenghis
added a commit
that referenced
this pull request
May 10, 2026
W8-14 left 11 live targets reporting a uniform ~0.28-0.31 diff ratio because the harness viewport (1920x1080) did not match the Images-GUI/ reference PNG dimensions (1536x1024 for 9 of 11). toHaveScreenshot is not resolution-tolerant, so the diff was geometric, not content. Surgical fix: drop the harness viewport to 1536x1024 to match the reference pack (single source of truth, PR #128 user-approved). 03_implementation/ui/playwright.visual.config.ts (use.viewport + projects[0].use.viewport) 03_implementation/ui/tests/visual/visual-targets.json (viewport + new reference_viewport field for clarity) Re-run delta vs PR #194: before: 0 match / 11 diff / 0 error / 20 skipped after: 9 match / 1 diff / 1 error / 20 skipped Two remaining failures are NOT viewport-related — both are reference PNGs captured at non-canonical sizes: 04_source_os_60_app_coverage_matrix : ref 1672x941 (diff) 04_source_os_remaining_categories : ref 1586x992 (error) Both need re-capture in PR #128 v2 (Images-GUI maintainer scope, NOT W8-15). Documented in 03_implementation/docs/handoffs/W8-15_VISUAL_VIEWPORT_ALIGN_2026-05-09.md with diff_pixels, ratio, evidence_path, and next_fix_attempt for each. Stacks on PR #194 (W8-14 harness fix). Builds on PR #188 (W8-12 Vite Fast-Refresh) for the SPA mount. No --update-snapshots, no reference PNGs touched, no paid services. visual-proof.spec.ts unchanged (no hardcoded viewport literals). Sources cited: - https://playwright.dev/docs/api/class-testoptions#test-options-viewport - Images-GUI/ folder pack from PR #128 (verified PNG IHDR dims) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merged
4 of 5 tasks
Ghenghis
added a commit
that referenced
this pull request
May 10, 2026
W8-14 left 11 live targets reporting a uniform ~0.28-0.31 diff ratio because the harness viewport (1920x1080) did not match the Images-GUI/ reference PNG dimensions (1536x1024 for 9 of 11). toHaveScreenshot is not resolution-tolerant, so the diff was geometric, not content. Surgical fix: drop the harness viewport to 1536x1024 to match the reference pack (single source of truth, PR #128 user-approved). 03_implementation/ui/playwright.visual.config.ts (use.viewport + projects[0].use.viewport) 03_implementation/ui/tests/visual/visual-targets.json (viewport + new reference_viewport field for clarity) Re-run delta vs PR #194: before: 0 match / 11 diff / 0 error / 20 skipped after: 9 match / 1 diff / 1 error / 20 skipped Two remaining failures are NOT viewport-related — both are reference PNGs captured at non-canonical sizes: 04_source_os_60_app_coverage_matrix : ref 1672x941 (diff) 04_source_os_remaining_categories : ref 1586x992 (error) Both need re-capture in PR #128 v2 (Images-GUI maintainer scope, NOT W8-15). Documented in 03_implementation/docs/handoffs/W8-15_VISUAL_VIEWPORT_ALIGN_2026-05-09.md with diff_pixels, ratio, evidence_path, and next_fix_attempt for each. Stacks on PR #194 (W8-14 harness fix). Builds on PR #188 (W8-12 Vite Fast-Refresh) for the SPA mount. No --update-snapshots, no reference PNGs touched, no paid services. visual-proof.spec.ts unchanged (no hardcoded viewport literals). Sources cited: - https://playwright.dev/docs/api/class-testoptions#test-options-viewport - Images-GUI/ folder pack from PR #128 (verified PNG IHDR dims) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Surgical fixes to W6-6's Playwright visual-proof harness so it produces real match/diff data against
Images-GUI/. Stacks on PR #188 (W8-12 Vite Fast-Refresh). Builds on PR #189 (W6-6 first-run JSON summary).outputPath is not allowed outside of the parent directory—snapshotPathTemplate: \"{arg}{ext}\"plus spec segments that escaped the test root tripped Playwright's safety check. Fix: newtests/visual/global-setup.tsmirrorsImages-GUI/intotests/visual/__refs__/(idempotent, size+mtime check, one-way only). Spec now emits forward-only segments into__refs__/. The mirror is gitignored;Images-GUI/remains the single source of truth.networkidletimeouts — the Hermes3D SPA's long-poll + SSE channels never go idle. Per Playwright's docs,networkidleis DISCOURAGED for this shape. Fix: dropnetworkidlefromwaitForStable(); keepdomcontentloaded+ 500ms quiet + the per-targetwait_test_id toBeVisible(15s)assertion that already proves the route mounted.Re-run Delta
03_implementation/docs/evidence/visual_proof_2026-05-09/summary.json:All 11 live targets now produce real pixel-diff data. The harness end-to-end works.
Persistence — 11 unfixed diffs (single shared cause)
All 11 live targets exceed tolerance with ratio ~0.28-0.31. Single shared root cause: reference PNGs were captured at 1536x1024, harness viewport is 1920x1080. Not 11 independent UI defects. Owned by W6-3 to fix (either re-render references at 1920x1080 or drop viewport to 1536x1024). See handoff doc table for per-target evidence_path / diff_path.
Sources
Test plan
npx playwright test --config playwright.visual.config.ts -g \"01_dashboard_advanced_a\"— completes in 2.3s with real diff data (diff_pixels=594010 ratio=0.29).summary.jsonwith 11 diff / 20 skipped / 0 error.(copied=0 skipped=31 pruned=0).git check-ignoreagainsttests/visual/.gitignore).updateSnapshots: \"none\", fail-on-missing-baseline all unchanged.Hermes evidence chain: PASS
🤖 Generated with Claude Code