[Question] Is there a way to ignore small difference between snapshots?

[DZakh]

I have an expected image snapshot and the only different thing comparing to an actual one is the text cursor. Because of this my test suit fails. I've tryed to increase the threshold option to the max but it didn't work. Do I understand it correctly, that threshold is only related to how accurate different pixels are found, not to the percentage of omitted overlapping pixels.

const count = pixelmatch_1.default(expected.data, actual.data, diff.data, expected.width, expected.height, { threshold: 0.2, ...options });

In my case the variable count from the source always returns 16 and threshold doesn't change anything.

In my opinion it should work something like this:

const count = // get the number of different pixels ...
const pass = count / (width * height) < threshold

Is there a way to ignore small difference between snapshots?

[pavelfeldman]

I wonder if you would benefit from us hiding the cursor when capturing the screenshot instead.

[DZakh]

I wonder if you would benefit from us hiding the cursor when capturing the screenshot instead.

No, I think it's a partial soltion, that may even harm other people.
Also, for instance I have other cases not related to the text cursor in my app, these are the app version that changes every commit and a lot of dates from server. Of course I can twik my app logic a little bit to always show the same result during tests, but I find the ability to set some kind of ignorance level when I match snapshots more appealing.
By ignorance level I mean that I can set some percentage, and if the number of different pixels devided to the number of all pixels is less than ignorance level then consider the test as successful.

[dgozman]

@DZakh This is an interesting idea, but we have to gather more feedback before changing how snapshot matching works.

Meanwhile, you can also capture an element screenshot to avoid the area with the ever-changing version, or anything else that is not stable. Would that work for you?

[sergioariveros]

This is also a feature that will help us, as we need to ignore some parts in our screenshoot that always are moving and making the comparison fails

[aslushnikov]

I have an expected image snapshot and the only different thing comparing to an actual one is the text cursor.

@DZakh as of 547a328 we now hide blinking cursor when making the screenshot. This is already available on @playwright/test@next and will be released in the upcoming @playwright/[email protected].

Do I understand it correctly, that threshold is only related to how accurate different pixels are found, not to the percentage of omitted overlapping pixels.

You understand correctly, and that's pretty unfortunate. We have another tracking bug for this: #10219

This is also a feature that will help us, as we need to ignore some parts in our screenshoot that always are moving and making the comparison fails

@sergioariveros this is coming via #11913

I'll close this in favor of mentioned separate issues.

[ritz078]

@aslushnikov It looks like the cursor is not hidden on firefox (Linux). I had to add color: transparent to hide them.

[aslushnikov]

@ritz078 interesting! Could you please share a repro? (Preferrably, in a new issue)