Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random order does not take effect with parameter "-n 4" (module pytest-xdist) #46

Closed
binbinlv opened this issue Jan 13, 2022 · 5 comments

Comments

@binbinlv
Copy link

Could you help to support the scenario of multiple process (the plugin "pytest-xdist": -n 6)?

Thanks.

@vsisl
Copy link

vsisl commented Feb 22, 2022

For me, trying to use pytest-xdist and pyters-random-order together results into an error during collection of tests. (ERROR collecting gwX). Tests are not executed at all.

Being able to run tests in randomised order in parallel would be great...

Anyways, thanks for creating this plugin, it is very helpful even without support for parallel execution!

Some info about my setup:

platform linux -- Python 3.7.7, pytest-6.2.5, py-1.9.0, pluggy-0.13.1
plugins: dash-1.16.1, timeout-2.1.0, random-order-1.0.4, forked-1.3.0, xdist-2.4.0

lyz-code added a commit to lyz-code/blue-book that referenced this issue Mar 14, 2022
`pytest-random-order` is a pytest plugin that randomises the order of tests.
This can be useful to detect a test that passes just because it happens to run
after an unrelated test that leaves the system in a favourable state.

To use it add the `--random-order` to your pytest run.

It [can't yet be used](pytest-dev/pytest-random-order#46)
with `pytest-xdist` though :(.

feat(linux_snippets#Clean old kernels): Clean old kernels

The full command is

```bash
dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get -y purge
```

To test what packages will it remove use:

```bash
dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | grep -E "(image|headers)" | xargs sudo apt-get --dry-run remove
```

Remember that your running kernel can be obtained by `uname -r`.
@Alexander-Serov
Copy link

Alexander-Serov commented Jul 27, 2022

For me, trying to use pytest-xdist and pyters-random-order together results into an error during collection of tests. (ERROR collecting gwX). Tests are not executed at all.

Being able to run tests in randomised order in parallel would be great...

Anyways, thanks for creating this plugin, it is very helpful even without support for parallel execution!

Some info about my setup:

platform linux -- Python 3.7.7, pytest-6.2.5, py-1.9.0, pluggy-0.13.1
plugins: dash-1.16.1, timeout-2.1.0, random-order-1.0.4, forked-1.3.0, xdist-2.4.0

I am having the same issue with pytest-xdist: #22 (comment) (pytest-random-order==1.0.4, pytest-xdist==2.5.0)

@matejsp
Copy link

matejsp commented Oct 12, 2022

I have a solution for this. Ideally I would prepare PR but this project seems not maintained anymore.

Put the following to conftest.py and it should work:

class XdistHooks:

    def pytest_configure_node(self, node: pytest.Item) -> None:
        seed = node.config.getoption('random_order_seed')
        node.workerinput['random_order_seed'] = seed  # type: ignore [attr-defined]


def pytest_configure(config: pytest.Config) -> None:
    if config.pluginmanager.hasplugin('xdist'):
        config.pluginmanager.register(XdistHooks())

    if hasattr(config, "workerinput"):  # pragma: no cover
        # pytest-xdist: use seed generated on main.
        seed = config.workerinput["random_order_seed"]  # type: ignore [attr-defined]
        if hasattr(config, "cache"):
            assert config.cache is not None
            config.cache.set("random_order_seed", seed)
        config.option.random_order_seed = seed

@jbasko
Copy link
Collaborator

jbasko commented Dec 3, 2022

This is fixed in v1.1.0 thanks to the code provided by @matejsp (apologies for too many mentions in the last few hours, had trouble updating to do the build and publish with github actions)

@jbasko jbasko closed this as completed Dec 3, 2022
@binbinlv
Copy link
Author

binbinlv commented Dec 5, 2022

Verified, it is fixed in pytest-random-order v1.1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants