Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't add fixture finalizer if the value is cached #11833

Merged
merged 17 commits into from
Mar 16, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/1489.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix some instances where teardown of higher-scoped fixtures was not happening in the reverse order they were initialized in.
29 changes: 19 additions & 10 deletions src/_pytest/fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -538,6 +538,8 @@ def getfixturevalue(self, argname: str) -> Any:
:raises pytest.FixtureLookupError:
If the given fixture could not be found.
"""
# Note: This is called during setup for evaluating fixtures defined via
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
# function arguments as well.
fixturedef = self._get_active_fixturedef(argname)
assert fixturedef.cached_result is not None, (
f'The fixture value for "{argname}" is not available. '
Expand Down Expand Up @@ -574,9 +576,8 @@ def _compute_fixture_value(self, fixturedef: "FixtureDef[object]") -> None:
"""Create a SubRequest based on "self" and call the execute method
of the given FixtureDef object.

This will force the FixtureDef object to throw away any previous
results and compute a new fixture value, which will be stored into
the FixtureDef object itself.
If the FixtureDef has cached the result it will do nothing, otherwise it will
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous comment is wrong, however saying that if the FixtureDef has cached the result it does nothing is not right either. It registers finalizers, and recomputes if the cache key no longer matches,

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, registering finalizers regardless of if the value is cached seems dumb... if it's cached then we've already registered a finalizer when we computed the value. And this can also cause bad teardown ordering:

import pytest


@pytest.fixture(scope="module")
def fixture_1(request):
    ...

@pytest.fixture(scope="module")
def fixture_2(fixture_1):
    print("setup 2")
    yield
    print("teardown 2")

@pytest.fixture(scope="module")
def fixture_3(fixture_1):
    print("setup 3")
    yield
    print("teardown 3")

def test_1(fixture_2):
    ...
def test_2(fixture_3):
    ...

# this will reschedule fixture_2's finalizer in the parent fixture, causing it to be
# torn down before fixture 3
def test_3(fixture_2):
    ...

# trigger finalization of fixture_1, otherwise the cleanup would sequence 3&2 before 1 as normal
@pytest.mark.parametrize("fixture_1", [None], indirect=["fixture_1"])
def test_4(fixture_1):
    ...

this prints

setup 2
setup 3
teardown 2
teardown 3

but if we remove test_3 we get 2-3-3-2.

But this is also a different issue+PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakkdl would you mind opening a fresh issue for this case and the one you describe in #11833 (comment)?

Copy link
Member Author

@jakkdl jakkdl Mar 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: #12134 and #12135

setup and run the fixture, cache the value, and schedule a finalizer for it.
"""
# prepare a subrequest object before calling fixture function
# (latter managed by fixturedef)
Expand Down Expand Up @@ -641,11 +642,8 @@ def _compute_fixture_value(self, fixturedef: "FixtureDef[object]") -> None:

# Check if a higher-level scoped fixture accesses a lower level one.
subrequest._check_scope(argname, self._scope, scope)
try:
# Call the fixture function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it originally ensured teardown even when setup failed

I suspect that we are missing a edge case tests there

Copy link
Member Author

@jakkdl jakkdl Feb 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the fixture need finalizing in case we never run ihook.pytest_fixture_setup? I thought it was tightly coupled to the user-supplied code.

But I could change it to

try:
    fixturedef.execute(...)
except:
    self._schedule_finalizers(...)
    raise

or specifically schedule a finalizer where I made a comment on line 1076

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The specific schedule is slightly better,

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On further consideration, changing that seems... bad to me? It means we'll be running the finalizer multiple times for a fixture with a cached exception, even if its setup was only run once. That is how it worked in the past though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies, what I meant is that a single schedule is better than the current mechanism

Ideally this logic would move to a data structure instead of the code where it is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right - no yeah I agree that the current implementation is quite messy and would benefit from an overhaul, but that seems out of scope for this PR.
I'm not sure I follow your original comment in that case, the try/except that previously was on this line is now moved one step deeper into execute() - and so there should be no change wrt to behaviour of scheduling finalizer if setup fails.

The only real change should ™️ be that the finalizer isn't scheduled if the value is cached (regardless of if it's an exception or not). If the setup code for the fixture fails, it's catched by the new try/finally.
If some other unrelated code in execute raises an exception the finalizer will not be scheduled, but in that case the setup has not run either.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is there a problem I should address, or was your comment just a musing on how this should be generally overhauled? Or do you consider that a requirement before modifying any logic?

fixturedef.execute(request=subrequest)
finally:
self._schedule_finalizers(fixturedef, subrequest)
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
# Make sure the fixture value is cached, running it if it isn't
fixturedef.execute(request=subrequest)

def _schedule_finalizers(
self, fixturedef: "FixtureDef[object]", subrequest: "SubRequest"
Expand Down Expand Up @@ -1055,15 +1053,17 @@ def finish(self, request: SubRequest) -> None:
self.cached_result = None
self._finalizers.clear()

# Note: the return value is entirely unused, no tests depend on it
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
def execute(self, request: SubRequest) -> FixtureValue:
finalizer = functools.partial(self.finish, request=request)
# Get required arguments and register our own finish()
# with their finalization.
for argname in self.argnames:
fixturedef = request._get_active_fixturedef(argname)
if argname != "request":
# PseudoFixtureDef is only for "request".
assert isinstance(fixturedef, FixtureDef)
fixturedef.addfinalizer(functools.partial(self.finish, request=request))
fixturedef.addfinalizer(finalizer)

my_cache_key = self.cache_key(request)
if self.cached_result is not None:
Expand All @@ -1073,6 +1073,7 @@ def execute(self, request: SubRequest) -> FixtureValue:
if my_cache_key is cache_key:
if self.cached_result[2] is not None:
exc = self.cached_result[2]
# this would previously trigger adding a finalizer. Should it?
raise exc
else:
result = self.cached_result[0]
Expand All @@ -1083,7 +1084,15 @@ def execute(self, request: SubRequest) -> FixtureValue:
assert self.cached_result is None

ihook = request.node.ihook
result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
try:
# Setup the fixture, run the code in it, and cache the value
# in self.cached_result
result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
finally:
# schedule our finalizer, even if the setup failed
request.node.addfinalizer(finalizer)

# note: unused
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
return result

def cache_key(self, request: SubRequest) -> object:
Expand Down
33 changes: 33 additions & 0 deletions testing/python/test_scope_fixture_teardown_order.py
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import pytest

last_executed = ""


@pytest.fixture(scope="module")
def fixture_1():
global last_executed
assert last_executed == ""
last_executed = "autouse_setup"
yield
assert last_executed == "noautouse_teardown"
last_executed = "autouse_teardown"


@pytest.fixture(scope="module")
def fixture_2():
global last_executed
assert last_executed == "autouse_setup"
last_executed = "noautouse_setup"
yield
assert last_executed == "run_test"
last_executed = "noautouse_teardown"


def test_autouse_fixture_teardown_order(fixture_1, fixture_2):
global last_executed
assert last_executed == "noautouse_setup"
last_executed = "run_test"


def test_2(fixture_1):
pass
Loading