-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TST: migrate image comparison tests from nose to pytest-mpl (⏰ wait on #4122) #4118
TST: migrate image comparison tests from nose to pytest-mpl (⏰ wait on #4122) #4118
Conversation
2a2fd8f
to
f96bd77
Compare
I was not originally planning to fully replace the "answer tests" job on GHA but I see that after removing all image tests from it, it now runs only a very small set of tests and takes about 20s to run (and 10min to build), so I think I should just clear that out completely and replace the job with a new one 100% focused on image-comparison tests. |
456d539
to
21e510a
Compare
I think I should split this in two parts and separate tests that require real data (running on Jenkins) from the rest (running on GH) |
This is probably beyond saving at this point. In any case, #4122 has most of this work in a mergeable state, so I think I should just close this one. |
Can you describe the difference with #4122? |
Mostly, I was initially going for a wider target, and wanted to migrate all image tests at once, including the ones that require large datasets that are only run on Jenkins. In #4122 I just focused on GHA |
PR Summary
Partially addresses #4102
This is far from done, but I need early iterations with CI to see what problems I encounter.
Right now I'm only aiming at image tests being skipped on nose
approach:
the main difficulty is that our
yield
base image tests are built via loops, whilepytest-mpl
relies on return values of test functions. Whenpytest.mark.parametrize
is appropriate, this can be a one-to-one translation, but otherwise I rely on generated code to reproduce existing test (by which I mean that I write many alike plain test functions, just not purely by hand).TODO:
GenericImageTest
VRImageComparisonTest
PhasePlotAttributeTest
PixelizedParticleProjectionValuesTest
PixelizedProjectionValuesTest
PlotWindowAttributeTest
tests/report_failed_answers.py
filename=...
in mpl_image_compar markerspytest.mark.parametrize
opened questions: