Fix incompatibility with pytest == 8.1.0 #199
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
|
The idea of the change here looks good but it has some syntax errors and codestyle errors. Could you clean those up? |
the suggestions above should fix the errors |
Fixes incompatibility with pytest == 8.1.0 by internalizing call_runtest_hook function.
|
FYI, 8.1.0 has been yanked and the removals have been reverted for the time being, FWIU at least until they're better documented. |
|
Tested this PR on top of current master and pytest fails with + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.1.0, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0
plugins: flaky-3.7.0
collected 74 items / 1 error
==================================== ERRORS ====================================
__________________ ERROR collecting test/test_flaky_plugin.py __________________
/usr/lib/python3.9/site-packages/_pytest/python.py:524: in importtestmodule
mod = import_path(
/usr/lib/python3.9/site-packages/_pytest/pathlib.py:580: in import_path
importlib.import_module(module_name)
/usr/lib64/python3.9/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1030: in _gcd_import
???
<frozen importlib._bootstrap>:1007: in _find_and_load
???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:680: in _load_unlocked
???
/usr/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:169: in exec_module
source_stat, co = _rewrite_test(fn, self.config)
/usr/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:351: in _rewrite_test
tree = ast.parse(source, filename=strfn)
/usr/lib64/python3.9/ast.py:50: in parse
return compile(source, filename, mode, flags,
E File "/home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_flaky_plugin.py", line 13
E "default_not_started": TestCaseDataset(2, 1, 0, 0, False),
E ^
E SyntaxError: invalid syntax
=========================== short test summary info ============================
ERROR test/test_flaky_plugin.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.30s =============================== |
|
After add test/test_flaky_plugin.py to --ignore list pytest passes units scan but it fails in 6 units Here is pytest output:+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network' --ignore test/test_flaky_plugin.py
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.1.0, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0
plugins: flaky-3.7.0
collected 74 items
test/test_flaky_decorator.py ... [ 4%]
test/test_multiprocess_string_io.py FF [ 6%]
test/test_pytest/pytest_generate_example/test_pytest_generate_example.py [ 8%]
.. [ 9%]
test/test_pytest/test_flaky_pytest_plugin.py .....................FFF... [ 45%]
..................... [ 74%]
test/test_pytest/test_pytest_example.py .xx.x...s......s [ 94%]
test/test_pytest/test_pytest_options_example.py F.. [ 98%]
test/test_pytest/test_pytester_plugin.py . [100%]
=================================== FAILURES ===================================
________________ TestMultiprocessStringIO.test_write_then_read _________________
self = <test.test_multiprocess_string_io.TestMultiprocessStringIO testMethod=test_write_then_read>
def test_write_then_read(self):
> for name in _test_values:
E NameError: name '_test_values' is not defined
test/test_multiprocess_string_io.py:26: NameError
______________ TestMultiprocessStringIO.test_writelines_then_read ______________
self = <test.test_multiprocess_string_io.TestMultiprocessStringIO testMethod=test_writelines_then_read>
def test_writelines_then_read(self):
> for name in _test_values:
E NameError: name '_test_values' is not defined
test/test_multiprocess_string_io.py:34: NameError
__________ test_flaky_plugin_raises_errors_in_fixture_setup[instance] __________
flaky_test = <test.test_pytest.test_flaky_pytest_plugin.MockTestItem object at 0x7f3fcebccc10>
flaky_plugin = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f3fcebcc7f0>
string_io = <_io.StringIO object at 0x7f3fcebce4c0>
mock_io = <_io.StringIO object at 0x7f3fcebce670>
def test_flaky_plugin_raises_errors_in_fixture_setup(
flaky_test,
flaky_plugin,
string_io,
mock_io,
):
"""
Test for Issue #57 - fixtures which raise an error should show up as
test errors.
This test ensures that exceptions occurring when running a test
fixture are copied into the call info's excinfo field.
"""
def error_raising_setup_function(item):
assert item is flaky_test
item.ran_setup = True
return 5 / 0
flaky()(flaky_test)
flaky_test.ihook = Mock()
flaky_test.ihook.pytest_runtest_setup = error_raising_setup_function
flaky_plugin._call_infos[flaky_test] = {} # pylint:disable=protected-access
> call_info = runner.call_runtest_hook(flaky_test, 'setup')
E AttributeError: module '_pytest.runner' has no attribute 'call_runtest_hook'
test/test_pytest/test_flaky_pytest_plugin.py:242: AttributeError
___________ test_flaky_plugin_raises_errors_in_fixture_setup[module] ___________
flaky_test = <test.test_pytest.test_flaky_pytest_plugin.MockTestItem object at 0x7f3fcf62bd60>
flaky_plugin = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f3fcf4fc9a0>
string_io = <_io.StringIO object at 0x7f3fcf7c4430>
mock_io = <_io.StringIO object at 0x7f3fcf7c4550>
def test_flaky_plugin_raises_errors_in_fixture_setup(
flaky_test,
flaky_plugin,
string_io,
mock_io,
):
"""
Test for Issue #57 - fixtures which raise an error should show up as
test errors.
This test ensures that exceptions occurring when running a test
fixture are copied into the call info's excinfo field.
"""
def error_raising_setup_function(item):
assert item is flaky_test
item.ran_setup = True
return 5 / 0
flaky()(flaky_test)
flaky_test.ihook = Mock()
flaky_test.ihook.pytest_runtest_setup = error_raising_setup_function
flaky_plugin._call_infos[flaky_test] = {} # pylint:disable=protected-access
> call_info = runner.call_runtest_hook(flaky_test, 'setup')
E AttributeError: module '_pytest.runner' has no attribute 'call_runtest_hook'
test/test_pytest/test_flaky_pytest_plugin.py:242: AttributeError
___________ test_flaky_plugin_raises_errors_in_fixture_setup[parent] ___________
flaky_test = <test.test_pytest.test_flaky_pytest_plugin.MockTestItem object at 0x7f3fcebcc400>
flaky_plugin = <flaky.flaky_pytest_plugin.FlakyPlugin object at 0x7f3fcebcc2b0>
string_io = <_io.StringIO object at 0x7f3fcf7c2ee0>
mock_io = <_io.StringIO object at 0x7f3fcec9faf0>
def test_flaky_plugin_raises_errors_in_fixture_setup(
flaky_test,
flaky_plugin,
string_io,
mock_io,
):
"""
Test for Issue #57 - fixtures which raise an error should show up as
test errors.
This test ensures that exceptions occurring when running a test
fixture are copied into the call info's excinfo field.
"""
def error_raising_setup_function(item):
assert item is flaky_test
item.ran_setup = True
return 5 / 0
flaky()(flaky_test)
flaky_test.ihook = Mock()
flaky_test.ihook.pytest_runtest_setup = error_raising_setup_function
flaky_plugin._call_infos[flaky_test] = {} # pylint:disable=protected-access
> call_info = runner.call_runtest_hook(flaky_test, 'setup')
E AttributeError: module '_pytest.runner' has no attribute 'call_runtest_hook'
test/test_pytest/test_flaky_pytest_plugin.py:242: AttributeError
_____________________________ test_something_flaky _____________________________
dummy_list = [0]
def test_something_flaky(dummy_list=[]):
# pylint:disable=dangerous-default-value
dummy_list.append(0)
> assert len(dummy_list) > 1
E assert 1 > 1
E + where 1 = len([0])
test/test_pytest/test_pytest_options_example.py:11: AssertionError
================================== XFAILURES ===================================
________________ test_something_good_with_failing_setup_fixture ________________
@pytest.fixture(scope='function')
def failing_setup_fixture():
> assert False
E assert False
test/test_pytest/test_pytest_example.py:21: AssertionError
________________ test_something_good_with_failing_setup_fixture ________________
@pytest.fixture(scope='function')
def failing_setup_fixture():
> assert False
E assert False
test/test_pytest/test_pytest_example.py:21: AssertionError
___________________ TestExample.test_non_flaky_failing_thing ___________________
self = <test.test_pytest.test_pytest_example.TestExample object at 0x7f3fced01df0>
@pytest.mark.xfail
def test_non_flaky_failing_thing(self):
"""Flaky will also not interact with this test"""
> assert self == 1
E assert <test.test_pytest.test_pytest_example.TestExample object at 0x7f3fced01df0> == 1
test/test_pytest/test_pytest_example.py:40: AssertionError
=============================== warnings summary ===============================
test/test_flaky_decorator.py::TestFlakyDecorator::test_flaky_adds_flaky_attributes_to_test_method
/usr/lib64/python3.9/unittest/case.py:1140: DeprecationWarning: assertDictContainsSubset is deprecated
warnings.warn('assertDictContainsSubset is deprecated',
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===Flaky Test Report===
Flaky report texttest_something_flaky failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 1 > 1
+ where 1 = len([0])
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:16>]
test_something_flaky passed 1 out of the required 1 times. Success!
test_something_good_with_failing_setup_fixture failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert False
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:85>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:843>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:878>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:82>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:158>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:515>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:512>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1777>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:687>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:541>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:570>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:647>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:1079>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/setuponly.py:36>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:1132>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:913>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:21>]
test_something_good_with_failing_setup_fixture failed; it passed 0 out of the required 1 times.
<class 'AssertionError'>
assert False
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:85>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:843>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:878>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:82>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:158>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:515>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:512>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1777>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:687>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:541>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:570>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:647>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:1079>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/setuponly.py:36>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:1132>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/fixtures.py:913>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:21>]
test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExample._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:50>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times.
test_flaky_thing_that_fails_then_succeeds passed 2 out of the required 2 times. Success!
test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 1 out of the required 2 times. Running test again until it passes 2 times.
test_flaky_thing_that_succeeds_then_fails_then_succeeds passed 2 out of the required 2 times. Success!
test_flaky_thing_that_always_passes passed 1 out of the required 2 times. Running test again until it passes 2 times.
test_flaky_thing_that_always_passes passed 2 out of the required 2 times. Success!
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExampleFlakyTests._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:87>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExampleFlakyTestCase._threshold
[<TracebackEntry /usr/lib64/python3.9/unittest/case.py:59>, <TracebackEntry /usr/lib64/python3.9/unittest/case.py:592>, <TracebackEntry /usr/lib64/python3.9/unittest/case.py:550>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:101>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestMarkedClass._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:119>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
test_requiring_my_fixture failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 1 > 1
+ where 1 = len([0])
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:140>]
test_requiring_my_fixture passed 1 out of the required 1 times. Success!
test_something_flaky failed (1 runs remaining out of 2).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExampleRerunFilter._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_example.py:155>]
test_something_flaky passed 1 out of the required 1 times. Success!
test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3).
<class 'AssertionError'>
assert -1 >= 1
+ where -1 = TestExample._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_options_example.py:27>]
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 3).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExample._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_options_example.py:27>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
test_flaky_thing_that_fails_then_succeeds failed (2 runs remaining out of 3).
<class 'AssertionError'>
assert -1 >= 1
+ where -1 = TestExampleFlakyTests._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_options_example.py:43>]
test_flaky_thing_that_fails_then_succeeds failed (1 runs remaining out of 3).
<class 'AssertionError'>
assert 0 >= 1
+ where 0 = TestExampleFlakyTests._threshold
[<TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:340>, <TracebackEntry /home/tkloczko/rpmbuild/BUILDROOT/python-flaky-3.7.0-15.fc36.x86_64/usr/lib/python3.9/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:87>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/threadexception.py:63>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:90>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:65>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:849>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/logging.py:832>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/capture.py:883>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:121>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/skipping.py:256>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:182>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/runner.py:172>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:1774>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_hooks.py:501>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_manager.py:119>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:138>, <TracebackEntry /usr/lib/python3.9/site-packages/pluggy/_callers.py:102>, <TracebackEntry /usr/lib/python3.9/site-packages/_pytest/python.py:199>, <TracebackEntry /home/tkloczko/rpmbuild/BUILD/flaky-3.7.0/test/test_pytest/test_pytest_options_example.py:43>]
test_flaky_thing_that_fails_then_succeeds passed 1 out of the required 1 times. Success!
===End Flaky Test Report===
=========================== short test summary info ============================
SKIPPED [1] test/test_pytest/test_pytest_example.py:66: This really fails! Remove skipif to see the test failure.
SKIPPED [1] ../../../../../usr/lib/python3.9/site-packages/_pytest/unittest.py:357: This test always fails
XFAIL test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture
XFAIL test/test_pytest/test_pytest_example.py::test_something_good_with_failing_setup_fixture
XFAIL test/test_pytest/test_pytest_example.py::TestExample::test_non_flaky_failing_thing
FAILED test/test_multiprocess_string_io.py::TestMultiprocessStringIO::test_write_then_read
FAILED test/test_multiprocess_string_io.py::TestMultiprocessStringIO::test_writelines_then_read
FAILED test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[instance]
FAILED test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[module]
FAILED test/test_pytest/test_flaky_pytest_plugin.py::test_flaky_plugin_raises_errors_in_fixture_setup[parent]
FAILED test/test_pytest/test_pytest_options_example.py::test_something_flaky
======== 6 failed, 64 passed, 2 skipped, 3 xfailed, 1 warning in 0.42s ========= |
ananth102
added a commit
to ananth102/ack-sagemaker-controller
that referenced
this pull request
Mar 10, 2024
The minor version bump seems to cause issues with flakey which might not be maintained anymore (box/flaky#199)
This was referenced Mar 10, 2024
blink1073
added a commit
to jupyter-server/jupyter_server
that referenced
this pull request
Mar 10, 2024
We'll need box/flaky#199 before flaky is compatible.
|
@jonathan-eq and others, thanks for the help. I needed to fix some problems with flaky on Python 3.12 in order to get CI passing, so I did that on another branch in #201, but I was able to use the commit from this PR. I'll try to create a new release later today. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #198
Fixes incompatibility with pytest == 8.1.0 by internalizing call_runtest_hook function.