test order affects test success/speed. it shouldn't

[leondz]

PR #472 is a hack.

pytest-8.0.0 changed the ordering of tests, and this exposed a stateful? dependency that led to tests hanging/taking incredibly long.

py 3.11.5
pytest 7.4.2
python3 -m pytest tests/test_cli.py OK
python3 -m pytest tests/ OK
pytest 8.0.0
python3 -m pytest tests/test_cli.py OK
python3 -m pytest tests/ vv slow

OK. The order in 8.0.0 is:

tests/detectors/test_detectors_packagehallucination.py .....                                                                                          [  0%]
tests/generators/test_huggingface.py ...                                                                                                              [  1%]
tests/probes/test_probe_format.py ................................................................................................................... [ 16%]
...............................................................                                                                                       [ 25%]
tests/probes/test_probe_tags.py .........................................................................................                             [ 37%]
tests/probes/test_probes_encoding.py ...                                                                                                              [ 37%]
tests/probes/test_probes_packagehallucination.py .                                                                                                    [ 38%]
tests/test_analyze.py ..                                                                                                                              [ 38%]
tests/test_attempt.py .                                                                                                                               [ 38%]
tests/test_cli.py ....

The order in 7.4.2 is:

tests/test_analyze.py ..                                                                                                                              [  0%]
tests/test_attempt.py .                                                                                                                               [  0%]
tests/test_cli.py ......                                                                                                                              [  1%]
tests/test_config.py ........................................                                                                                         [  6%]
tests/test_docs.py .................................................................................................................................. [ 24%]
..............................................................................................................                                        [ 39%]
tests/test_plugin_load.py ........................................................................................................................... [ 56%]
..........................................                                                                                                            [ 61%]
tests/test_reqs.py .                                                                                                                                  [ 61%]
tests/detectors/test_detectors_packagehallucination.py .....                                                                                          [ 62%]
tests/generators/test_huggingface.py ...                                                                                                              [ 63%]
tests/probes/test_probe_format.py ................................................................................................................... [ 78%]
...............................................................                                                                                       [ 87%]
tests/probes/test_probe_tags.py .........................................................................................                             [ 99%]
tests/probes/test_probes_encoding.py ...                                                                                                              [ 99%]
tests/probes/test_probes_packagehallucination.py .                    

extra in 8.0.0 before test_cli are:

tests/detectors/test_detectors_packagehallucination.py
tests/generators/test_huggingface.py
tests/probes/test_probe_format.py
tests/probes/test_probe_tags.py
tests/probes/test_probes_encoding.py
tests/probes/test_probes_packagehallucination.py

the slowness seems to happen in the encoding module (or at least, that's where ctrl-c catches us)
test_plugin_load is also slow

to troubleshoot, try adding in tests from the "extra" list (just above) before test_cli, and see which one (or ones...) make it slow down. then we can work out what we don't know about how tests are run / how state is preserved or not removed.

seeing as test_cli seems to get a bit stuck on encoding, maybe test_probes_encoding is the trigger. maybe we're creating way too many combinations of encodings. ¯_(ツ)_/¯