Pytest 7.3.2 changes in behaviour regarding conftest.py and testpaths
#11104
Comments
RonnyPfannschmidt
added
the
type: regression
|
a recent bugfix made a hidden mistake in your conftest layout surface the basic gist is, that with testpaths, pytest now correctly consider the conf-tests in those root test paths as possible early loaded conftests (to supply addopts & co) in turn making sure that all options are always registred as far as i understand you previously ran either one or the other parts of the testsuite, thus it was never possible that both conftests where loaded at the same time (which was a bug in pytest) now that pytest more correctly considers all sources of options, an error pops up as previously you actually absolutely had to register in both places since pytest was not picking up all sources of options for a testsuite the recommended fix would be to move shared options into a plugin module to list it in the pytest_plugins of the conftests |
|
thank you for the clarification! that does make sense, it's true that we never seemed to run both test suites together, we'd always do I'll close this out, as it looks to me that the solution is for us to fix our config. |
|
However, I do think there's an issue in pytest here. I think that if argument paths are to be used for finding initial conftests, then we should use the @nicoddemus WDYT? (Reopening for discussion) |
You are right, I did not realize that at the time.
At first glance seems reasonable indeed. |
|
Agreed this seems a fundamental change in behavior in how Is it desirable that the solution in #10988 should also treat What do we think of something like this? diff --git a/src/_pytest/config/__init__.py b/src/_pytest/config/__init__.py
index 85d8830e7..fa1924ce5 100644
--- a/src/_pytest/config/__init__.py
+++ b/src/_pytest/config/__init__.py
@@ -546,7 +546,10 @@ class PytestPluginManager(PluginManager):
)
self._noconftest = namespace.noconftest
self._using_pyargs = namespace.pyargs
- testpaths = namespace.file_or_dir + testpaths_ini
+ testpaths = namespace.file_or_dir
+ if not testpaths:
+ # source testpaths_ini value only when command-line files absent
+ testpaths = testpaths_ini
foundanchor = False
for testpath in testpaths:
path = str(testpath)If folks believe that new |
|
Thanks for the extra details I consider this a overreaching bugfix We should restore part of the old behavior until a major release We also should ensure all test path related conftests are considered for pytest configuration and addoption for consistency in a major release |
|
Agreed. Sorry I won't be able to work on this today (likely tomorrow), so if anybody wants to contribute a fix, it would be greatly appreciated! |
|
I can handle this one. |
Fixes pytest-dev#11104. See the issue for a description of the problem. Now, we use the same logic for initial conftest paths as we do for deciding the initial args, which was the idea behind checking `namespace.file_or_dir` and `testpaths` previously. This fixes the issue of `testpaths` being considered for initial conftests even when it's not used for the args. (Another issue in faeb161 was that the `testpaths` were not glob-expanded, this is also fixed.)
Fixes pytest-dev#11104. See the issue for a description of the problem. Now, we use the same logic for initial conftest paths as we do for deciding the initial args, which was the idea behind checking `namespace.file_or_dir` and `testpaths` previously. This fixes the issue of `testpaths` being considered for initial conftests even when it's not used for the args. (Another issue in faeb161 was that the `testpaths` were not glob-expanded, this is also fixed.)
In cibuildwheel, we have two test suites - the unit tests at
/unit_testand the integration test suite at/test. Both/unit_testand/testare listed in testpaths-pyproject.toml
We then run either
unit_testortestusingpytest unit_test/pytest test.Each
unit_test/testdir contains a conftest.py file, which adds some options usingparser.addoption. One option that is common to both test suites is--run-podman. Before 7.3.2, this setup seemed to work, we could run both unit tests and integration tests without issue. But on 7.3.2 (perhaps since #10988?) we get the following error:Is this an issue in our configuration, or a bug? Should we no longer use testpaths to list all the test suites?
pip list output
Xref pypa/cibuildwheel#1518
The text was updated successfully, but these errors were encountered: