Description
This is a bit of a strange one, in that it’s hard to reproduce. I’m the maintainer of the python-pytest-bdd
package in Fedora Linux, and I’m seeing a test failure associated with Fedora’s update to pip 21.3.
=================================== FAILURES ===================================
_____________________________ test_apply_tag_hook ______________________________
testdir = <Testdir local('/tmp/pytest-of-mockbuild/pytest-0/test_apply_tag_hook0')>
def test_apply_tag_hook(testdir):
testdir.makeconftest(
"""
import pytest
@pytest.hookimpl(tryfirst=True)
def pytest_bdd_apply_tag(tag, function):
if tag == 'todo':
marker = pytest.mark.skipif(True, reason="Not implemented yet")
marker(function)
return True
else:
# Fall back to pytest-bdd's default behavior
return None
"""
)
testdir.makefile(
".feature",
test="""
Feature: Customizing tag handling
@todo
Scenario: Tags
Given I have a bar
@xfail
Scenario: Tags 2
Given I have a bar
""",
)
testdir.makepyfile(
"""
from pytest_bdd import given, scenarios
@given('I have a bar')
def i_have_bar():
return 'bar'
scenarios('test.feature')
"""
)
result = testdir.runpytest("-rsx")
result.stdout.fnmatch_lines(["SKIP*: Not implemented yet"])
> result.stdout.fnmatch_lines(["*= 1 skipped, 1 xpassed * =*"])
E Failed: nomatch: '*= 1 skipped, 1 xpassed * =*'
E and: '============================= test session starts =============================='
E and: 'platform linux -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0'
E and: 'rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_apply_tag_hook0'
E and: 'plugins: bdd-4.1.0'
E and: 'collected 2 items'
E and: ''
E and: 'test_apply_tag_hook.py sX [100%]'
E and: ''
E and: '=============================== warnings summary ==============================='
E and: '../../../../usr/lib/python3.10/site-packages/_pytest/config/__init__.py:1114'
E and: ' /usr/lib/python3.10/site-packages/_pytest/config/__init__.py:1114: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: pytest_bdd'
E and: ' self._mark_plugins_for_rewrite(hook)'
E and: ''
E and: '-- Docs: https://docs.pytest.org/en/stable/warnings.html'
E and: '=========================== short test summary info ============================'
E and: 'SKIPPED [1] ../../../../builddir/build/BUILDROOT/python-pytest-bdd-4.1.0-6.fc36.x86_64/usr/lib/python3.10/site-packages/pytest_bdd/scenario.py:163: Not implemented yet'
E and: '=================== 1 skipped, 1 xpassed, 1 warning in 0.01s ==================='
E remains unmatched: '*= 1 skipped, 1 xpassed * =*'
result = <RunResult ret=ExitCode.OK len(stdout.lines)=17 len(stderr.lines)=0 duration=0.04s>
testdir = <Testdir local('/tmp/pytest-of-mockbuild/pytest-0/test_apply_tag_hook0')>
/builddir/build/BUILD/pytest-bdd-4.1.0/tests/feature/test_tags.py:162: Failed
All of the other tests still pass. As you can see, the glob '*= 1 skipped, 1 xpassed * =*'
does not match = 1 skipped, 1 xpassed, 1 warning in 0.01s =
due to the unexpected PytestAssertRewriteWarning
. I’m not sure why this should be triggered by an update of pip.
Annoyingly, I can’t seem to reproduce this in a plain Python 3.10 virtualenv, using pip 21.3 and pip-installed dependency instead of Fedora-packaged ones.
I found that pytest-cov had a similar issue several years ago and worked around it by adding PYTEST_DONT_REWRITE
to the module docstring pytest_cov/__init__.py
.
I can confirm that adding PYTEST_DONT_REWRITE
to the docstring in pytest_bdd/__init__.py
does fix the unexpected warning, but I don’t really understand the pytest rewriting system, so I don’t know if this would have unintended consequences.
Alternatively, either skipping test_apply_tag_hook
or loosening the glob very subtly to '*= 1 skipped, 1 xpassed* =*'
can work around the issue.
For now, I’m loosening the glob in the test.
I’m curious:
- if you have any better understanding of why this might be happening than I do
- if you have any thoughts on the downstream options
- if you think
pytest-bdd
needs any changes upstream based on this report
Thanks!