pytest 8.2.2 breaks pytest-rerunfailures for tests that inherit unittest.TestCase

[mdmintz]

pytest 8.2.2 breaks pytest-rerunfailures for tests that inherit unittest.TestCase

The code: (placed in new_file.py)

from unittest import TestCase

class MyTestClass(TestCase):
    def test_base(self):
        self.fail()

Run command:

pytest new_file.py --reruns=1

Expected output: (Using pytest==8.2.1 / previous version)

self = <examples.new_file.MyTestClass testMethod=test_base>

    def test_base(self):
>       self.fail()
E       AssertionError: None

new_file.py:5: AssertionError

Regression output: (Using pytest==8.2.2 / latest version)

self = <TestCaseFunction test_base>

    def runtest(self) -> None:
        from _pytest.debugging import maybe_wrap_pytest_function_for_tracing
    
        testcase = self.instance
>       assert testcase is not None
E       AssertionError

The error occurs during the rerun of the test.

The failure line should be self.fail(). Instead, it's assert testcase is not None.
This causes the rerun to never happen. (The test only ran the first time)

Tested on a Mac. Here's the pip list:

Package              Version
-------------------- -------
iniconfig            2.0.0
packaging            24.0
pip                  24.0
pluggy               1.5.0
pytest               8.2.2
pytest-rerunfailures 14.0

[hb2638]

Thank you for filing this with succinct replication steps. I spent hours yesterday trying to figure out why our CI/CD was failing

[bluetech]

We can probably accommodate rerunfailures here. I will take a look.

[hb2638]

I think switching pytest_runtest_makereport to the "new style" hook will make the issue go away

BEFORE

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()
    if result.when == "setup":
        # clean failed statuses at the beginning of each test/rerun
        setattr(item, "_test_failed_statuses", {})

        # create a dict to store error-check results for each stage
        setattr(item, "_terminal_errors", {})

    _test_failed_statuses = getattr(item, "_test_failed_statuses", {})
    _test_failed_statuses[result.when] = result.failed
    item._test_failed_statuses = _test_failed_statuses

    item._terminal_errors[result.when] = _should_hard_fail_on_error(item, result)

AFTER

@pytest.hookimpl(wrapper=True)
def pytest_runtest_makereport(item, call):
    result = yield
    if result.when == "setup":
        # clean failed statuses at the beginning of each test/rerun
        setattr(item, "_test_failed_statuses", {})

        # create a dict to store error-check results for each stage
        setattr(item, "_terminal_errors", {})

    _test_failed_statuses = getattr(item, "_test_failed_statuses", {})
    _test_failed_statuses[result.when] = result.failed
    item._test_failed_statuses = _test_failed_statuses

    item._terminal_errors[result.when] = _should_hard_fail_on_error(item, result)
    return result

[hb2638]

I think switching pytest_runtest_makereport to the "new style" hook will make the issue go away

BEFORE

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    result = outcome.get_result()
    if result.when == "setup":
        # clean failed statuses at the beginning of each test/rerun
        setattr(item, "_test_failed_statuses", {})

        # create a dict to store error-check results for each stage
        setattr(item, "_terminal_errors", {})

    _test_failed_statuses = getattr(item, "_test_failed_statuses", {})
    _test_failed_statuses[result.when] = result.failed
    item._test_failed_statuses = _test_failed_statuses

    item._terminal_errors[result.when] = _should_hard_fail_on_error(item, result)

AFTER

@pytest.hookimpl(wrapper=True)
def pytest_runtest_makereport(item, call):
    result = yield
    if result.when == "setup":
        # clean failed statuses at the beginning of each test/rerun
        setattr(item, "_test_failed_statuses", {})

        # create a dict to store error-check results for each stage
        setattr(item, "_terminal_errors", {})

    _test_failed_statuses = getattr(item, "_test_failed_statuses", {})
    _test_failed_statuses[result.when] = result.failed
    item._test_failed_statuses = _test_failed_statuses

    item._terminal_errors[result.when] = _should_hard_fail_on_error(item, result)
    return result

disregard, the issue is still there :^(