Parallel testing support (xdist)

[dex6]

Thanks for great plugin. However, there's one problem for me: my test suite uses xdist to run tests on multiple CPU cores, and it completely breaks with pytest-random-order.

Here's some fake tests to demonstrate the problem:

# cat /tmp/some_tests.py 
def test_1():
        pass
def test_2():
        pass
def test_3():
        pass
def test_4():
        pass

And here it is executed in two parallel processes:

# pytest /tmp/some_tests.py -n2
============================ test session starts ============================
platform linux2 -- Python 2.7.5, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
Using --randomly-seed=1503989180
Using --random-order-bucket=module
Using --random-order-seed=249480

rootdir: /tmp, inifile:
plugins: xdist-1.20.0, timeout-1.2.0, repeat-0.4.1, randomly-1.2.1, random-order-0.5.4, raisesregexp-2.1, pythonpath-0.7.1, mock-1.6.2, logging-2015.11.4, instafail-0.3.0, forked-0.2, faker-2.0.0, cov-2.5.1, browser-0.2.0
gw0 [4] / gw1 [4]
scheduling tests via LoadScheduling
collecting 0 items / 1 errors                                                
================================== ERRORS ===================================
___________________________ ERROR collecting gw1 ____________________________
Different tests were collected between gw0 and gw1. The difference is:
--- gw0

+++ gw1

@@ -1,4 +1,4 @@

 some_tests.py::test_3
-some_tests.py::test_4
 some_tests.py::test_2
 some_tests.py::test_1
+some_tests.py::test_4
========================== 1 error in 0.51 seconds ==========================

It gets worse the more tests and CPU cores you have. I suppose that every subprocess generates a different seed and executes other test cases than it was expected to, because when a seed is specified using --random-order-seed, it always seem to work.

[adamchainz]

https://github.com/adamchainz/pytest-randomly supports xdist :)

[jbasko]

Fixed.

[Alexander-Serov]

I know it's an old and closed issue, but I seem to be having the same problem with
pytest-random-order==1.0.4
pytest-xdist==2.5.0.

gw1:None (gw1)
Different tests were collected between gw0 and gw1. The difference is:
--- gw0

+++ gw1

@@ -1,276 +1,276 @@
...

Should I open a new issue?