Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] image comparison diff detection and threshold unintuitiveness #9444

Closed
dimkin-eu opened this issue Oct 12, 2021 · 1 comment
Closed

Comments

@dimkin-eu
Copy link

dimkin-eu commented Oct 12, 2021

Context:

  • Playwright Version: "@playwright/test": "^1.15.2"
  • Operating System: Catalina 10.15.7
  • Node.js version: v14.16.0
  • Browser: doesn't matter

Code Snippet

I've created repro here

Describe the bug

image comparator treats major changes in the same way as small ones,
so, minor changes in the picture must pass on lower threshold values comparing to major changed picture, but for now - both pass for the same 0.8

Screenshot 2021-10-12 at 18 18 01

Screenshot 2021-10-12 at 23 48 31

@yury-s
Copy link
Member

yury-s commented Oct 12, 2021

This is how pixelmatch works. Both of the images have significant differences and they are flagged as such even with rather lax threshold 0.7 (note that by default Playwright uses more strict 0.2).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants