test(visual): commit golden baselines and provenance - #138
Conversation
|
Warning Review limit reached
Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request promotes the Playwright visual regression tests from an optional stub to a standard part of the test suite. It introduces a PROVENANCE.md file to track approved golden baselines, updates documentation to reflect automatic server startup and the use of npm ci, and enhances the validate.ps1 script to enforce the presence of baseline files and their documentation. Feedback is provided regarding the use of backslashes in path strings within validate.ps1, which can cause execution failures on non-Windows CI runners; using forward slashes is recommended for cross-platform compatibility.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| $visualReadme = Join-Path $repoRoot "tests\visual\README.md" | ||
| $goldenDir = Join-Path $repoRoot "tests\visual\golden" | ||
| $goldenBaseline = Join-Path $goldenDir "e1-bundle-empty.png" | ||
| $goldenProvenance = Join-Path $repoRoot "tests\visual\PROVENANCE.md" |
There was a problem hiding this comment.
Using backslashes (\) in path strings passed to Join-Path can cause failures on non-Windows platforms (like Ubuntu CI runners), as Linux/macOS treat backslashes as literal characters in filenames rather than directory separators. Use forward slashes (/) instead to ensure cross-platform compatibility.
$goldenProvenance = Join-Path $repoRoot "tests/visual/PROVENANCE.md"
| $goldenBaseline = Join-Path $goldenDir "e1-bundle-empty.png" | ||
| $goldenProvenance = Join-Path $repoRoot "tests\visual\PROVENANCE.md" | ||
| $a11ySpec = Join-Path $repoRoot "tests\visual\harness\a11y.spec.js" | ||
| $visualSpecHarness = Join-Path $repoRoot "tests\visual\harness\visual.spec.js" |
There was a problem hiding this comment.
Using backslashes (\) in path strings passed to Join-Path can cause failures on non-Windows platforms (like Ubuntu CI runners), as Linux/macOS treat backslashes as literal characters in filenames rather than directory separators. Use forward slashes (/) instead to ensure cross-platform compatibility.
$visualSpecHarness = Join-Path $repoRoot "tests/visual/harness/visual.spec.js"
Co-authored-by: Cursor <cursoragent@cursor.com>
ded7021 to
a1da7b0
Compare
Raise maxDiffPixelRatio to 0.03 so CI Chromium can match Windows-authored goldens without weakening layout regression detection.
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| // tolerate that AA noise. | ||
| await expect(page).toHaveScreenshot("e1-bundle-empty.png", { | ||
| animations: "disabled", | ||
| maxDiffPixelRatio: 0.03, |
There was a problem hiding this comment.
WARNING: maxDiffPixelRatio: 0.03 is extremely permissive
A 3% pixel difference tolerance can easily mask significant visual regressions. The README documents "≤ 0.1% pixel diff on non-AA regions" but this value applies to all pixels. If the intent is only to absorb font AA variance, consider lowering this or scoping the tolerance to specific regions.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Files Reviewed (7 files)
Fix these issues in Kilo Cloud Reviewed by step-3.7-flash · Input: 125.5K · Output: 42K · Cached: 2.2M |
1 participant
Summary
Test plan
Made with Cursor