Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

binbinlv · 2022-01-13T03:54:29Z

Could you help to support the scenario of multiple process (the plugin "pytest-xdist": -n 6)?

Thanks.

vsisl · 2022-02-22T14:44:40Z

For me, trying to use pytest-xdist and pyters-random-order together results into an error during collection of tests. (ERROR collecting gwX). Tests are not executed at all.

Being able to run tests in randomised order in parallel would be great...

Anyways, thanks for creating this plugin, it is very helpful even without support for parallel execution!

Some info about my setup:

platform linux -- Python 3.7.7, pytest-6.2.5, py-1.9.0, pluggy-0.13.1
plugins: dash-1.16.1, timeout-2.1.0, random-order-1.0.4, forked-1.3.0, xdist-2.4.0

`pytest-random-order` is a pytest plugin that randomises the order of tests. This can be useful to detect a test that passes just because it happens to run after an unrelated test that leaves the system in a favourable state. To use it add the `--random-order` to your pytest run. It [can't yet be used](pytest-dev/pytest-random-order#46) with `pytest-xdist` though :(. feat(linux_snippets#Clean old kernels): Clean old kernels The full command is ```bash dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get -y purge ``` To test what packages will it remove use: ```bash dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get --dry-run remove ``` Remember that your running kernel can be obtained by `uname -r`.

Alexander-Serov · 2022-07-27T08:03:29Z

For me, trying to use pytest-xdist and pyters-random-order together results into an error during collection of tests. (ERROR collecting gwX). Tests are not executed at all.

Being able to run tests in randomised order in parallel would be great...

Anyways, thanks for creating this plugin, it is very helpful even without support for parallel execution!

Some info about my setup:
platform linux -- Python 3.7.7, pytest-6.2.5, py-1.9.0, pluggy-0.13.1
plugins: dash-1.16.1, timeout-2.1.0, random-order-1.0.4, forked-1.3.0, xdist-2.4.0

I am having the same issue with pytest-xdist: #22 (comment) (pytest-random-order==1.0.4, pytest-xdist==2.5.0)

matejsp · 2022-10-12T17:49:47Z

I have a solution for this. Ideally I would prepare PR but this project seems not maintained anymore.

Put the following to conftest.py and it should work:

class XdistHooks:

    def pytest_configure_node(self, node: pytest.Item) -> None:
        seed = node.config.getoption('random_order_seed')
        node.workerinput['random_order_seed'] = seed  # type: ignore [attr-defined]


def pytest_configure(config: pytest.Config) -> None:
    if config.pluginmanager.hasplugin('xdist'):
        config.pluginmanager.register(XdistHooks())

    if hasattr(config, "workerinput"):  # pragma: no cover
        # pytest-xdist: use seed generated on main.
        seed = config.workerinput["random_order_seed"]  # type: ignore [attr-defined]
        if hasattr(config, "cache"):
            assert config.cache is not None
            config.cache.set("random_order_seed", seed)
        config.option.random_order_seed = seed

jbasko · 2022-12-03T16:36:41Z

This is fixed in v1.1.0 thanks to the code provided by @matejsp (apologies for too many mentions in the last few hours, had trouble updating to do the build and publish with github actions)

binbinlv · 2022-12-05T09:33:25Z

Verified, it is fixed in pytest-random-order v1.1.0.

jbasko mentioned this issue Dec 3, 2022

Seed is not consistent in different xdist processes if not specified on the command line #49

Closed

jbasko closed this as completed Dec 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

binbinlv commented Jan 13, 2022

vsisl commented Feb 22, 2022

Alexander-Serov commented Jul 27, 2022 •

edited

Loading

matejsp commented Oct 12, 2022 •

edited

Loading

jbasko commented Dec 3, 2022

binbinlv commented Dec 5, 2022 •

edited

Loading

Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

Comments

binbinlv commented Jan 13, 2022

vsisl commented Feb 22, 2022

Alexander-Serov commented Jul 27, 2022 • edited Loading

matejsp commented Oct 12, 2022 • edited Loading

jbasko commented Dec 3, 2022

binbinlv commented Dec 5, 2022 • edited Loading

Alexander-Serov commented Jul 27, 2022 •

edited

Loading

matejsp commented Oct 12, 2022 •

edited

Loading

binbinlv commented Dec 5, 2022 •

edited

Loading