Add a more obvious way to pass parameters to fixture functions

[oakkitten]

note that there were major edits of this comment

pytest supports passing arguments to fixture functions using so-called indirect parametrization:

@pytest.fixture
def service(request):
    return f"Service launched with parameter {request.param!r}"


@pytest.mark.parametrize("service", ["parameter"], indirect=True)
def test_with_service(service):
    assert service == "Service launched with parameter 'parameter'"

There are some problems with this:

• This is buried in the documentation and hard to find
• This makes passing arguments a parametrization api. Parametrization, as I understand, usually refers to running the same test multiple times with different arguments. In the above example test_with_service is only run once. This test may only work with a service launched in this particular way. It may not make sense to run it against different kinds of service. In other words, this test doesn't require parametrization.
• The syntax, especially the indirect=... bit, is rather confusing
• The fixture has to manually deal with request.param object. It somehow has to verify that it contains the things that it can use, choose default values if not, and throw exceptions in case of errors. some of this behavior looks very much like regular function calls and could benefit from better syntax

So please add a more obvious way to pass parameters to fixture functions.

Additionally, we should note that there's a similar syntax to parametrize fixtures:

@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])
def smtp_connection(request):
    smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)
    ...

It is also problematic:

• This is more or less the same idea as with test parametrization, but it uses a different syntax
• Due to the fact that conceptually there is only a single argument (requst.param), you can't easily create a “matrix” of arguments like you can do with pytest.mark.parametrize. Consider this:

  @pytest.mark.parametrize("one", ["1", "uno"])
  @pytest.mark.parametrize("two", ["2", "dos"])
  fun test_one_two(one, two):
      ...

  This will run the test four times using all possible combinations of arguments. This is not possible with pytest.fixture
• You can't pass arguments to other fixtures with pytest.fixture. There's no indirect keyword

It would be nice if the two syntaxes were unified

---

If I may suggest something, here's my take on a better syntax. Most of it is already possible with pytest; here's a runnable proof of concept — see below regarding what it can't do.

You can define fixture like this:

from ... import fixture

@fixture
def dog(request, /, name, age=69):
    return f"{name} the dog aged {age}"

Here, to the left of / you have other fixtures, and to the right you have parameters that are supplied using:

@dog.arguments("Buddy", age=7)
def test_with_dog(dog):
    assert dog == "Buddy the dog aged 7"

Note: This works the same way function arguments work. If you don't supply the age argument, the default one, 69, is used instead. if you don't supply name, or omit the dog.arguments decorator, you get the regular TypeError: dog() missing 1 required positional argument: 'name'. If you have another fixture that takes argument name, it doesn't conflict with this one.

Also note: / is not strictly required, but it allows easily building this on top of existing pytest api, and also should prevent a few user errors

If for some reason you can't import dog directly, you can reference it by name:

dog = fixture.by_name("dog")

To run a test several times with different dogs, use:

from ... import arguments

@dog.argumentize(arguments("Champion"), arguments("Buddy", age=7))
def test_with_dog(dog):
    ...

Note: I'm using “argumentize” here for a lack of a better word. Pytest documentation says,

The builtin pytest.mark.parametrize decorator enables parametrization of arguments for a test function.

But arguments (e.g. "Champion") are not the subject but the object of this action. I'm not sure if “parametrize” fits here; the test test_with_dog is already parametrized by virtue of having the parameter dog. The subject must be the parameter here, so perhaps this should say “argumentization of parameters”? (Disclaimer: English isn't my mother tongue)

...it would be reasonable to not require arguments() in case of a single positional argument:

@dog.argumentize("Champion", arguments("Buddy", age=7))
def test_with_dog(dog):
    ...

To pass parameters to another fixture, stack arguments() decorators:

@cat.arguments("Mittens")
@dog.arguments("Buddy", age=7)
def test_with_cat_and_dog(cat, dog):    # this test is run once
    ...

To run a matrix of tests, stack argumentize() decorators:

@cat.argumentize("Tom", "Mittens")
@dog.argumentize("Champion", arguments("Buddy", age=7))
def test_with_cat_and_dog(cat, dog):    # this test is run four times
    ...

To argumentize several parameters as a group, use this:

from ... import parameters

@parameters(dog, cat).argumentize(
    ("Champion", arguments("Tom", age=420)), 
    ("Buddy", "Whiskers")
)
def test_with_dogs_and_cats(dog, cat):  # this test is run twice
    ...

To argumentize a parameter directly, without passing arguments to other fixtures, use a special “literal” fixture:

@fixture.literal("weasel").argumentize("Bob", "Mike")
def test_with_weasel(weasel):
    assert weasel in ["Bob", "Mike"]

This fixture works exactly like regular fixtures. In parameters(), you can omit the call to fixture.literal():

@parameters("expression", "result").argumentize(
    ("1 + 1", 2),
    ("2 * 2", 4),
)
def test_math(expression, result):
    assert eval(expression) == result

You can mix it all together. Also, you can use pytest.param() as usual:

@parameters(dog, owner, "poop").argumentize(
    (arguments("Champion", age=1337), "Macaulay Culkin", "brown"),
    ("Buddy", arguments(), "round"),
    pytest.param("Buddy", "Name", 123, marks=pytest.mark.xfail())
)
def test_with_dogs_and_owners_and_poop(dog, owner, poop):
    assert f"{dog=}, {owner=}, {poop=}" in [
        "dog='Champion the dog aged 1337', owner='Macaulay Culkin, owner of Champion the dog aged 1337', poop='brown'",
        "dog='Buddy the dog aged 69', owner='John Doe, owner of Buddy the dog aged 69', poop='round'",
        "dog='Buddy the dog aged 69', owner='Name, owner of Buddy the dog aged 69', poop=123"
    ]
    assert isinstance(poop, str)

Pass a keyword argument ids to argumentize() to have readable test case descriptions. I think you can't do this better than pytest is doing it at the time being.

You can also parametrize fixtures using the same syntax, including parameter matrices, etc

    @fixture.literal("name").argumentize("Veronica", "Greta")
    @fixture.literal("word").argumentize("boo", "meow")
    @fixture
    def hedgehog(self, request, /, name, word):
        return f"{name} the hedgehog says: {word}"

    def test_with_hedgehog(self, hedgehog):     # this test is run twice
        assert hedgehog in [
            "Veronica the hedgehog says: boo",
            "Greta the hedgehog says: boo",
            "Veronica the hedgehog says: meow",
            "Greta the hedgehog says: meow",
        ]

---

Notes on my proof of concept:

• With the exception of reading and mutating _pytestfixturefunction, it uses the regular pytest api. It doesn't require any breaking changes. It would require very few changes in existing pytest code if implemented properly.
• It is alrady usable, and also supports async fixtures. Also it produces a neat setup plan:

  $ pytest --setup-plan
  
  SETUP    F dog['Buddy', age=7]
  ...
  SETUP    F dog['Champion']
  SETUP    F owner (fixtures used: dog)['John Travolta']
  ...
  

• It doesn't support ids, although it could. Matricizing fixture function arguments is a bit hacky and throwing ids into that would complicate things too much for a proof of concept
• When argumentizing fixture functions, you can only use literal parameters, as pytest doesn't provide syntax to pass arguments to other fixtures in @pytest.fixture()
• When matricizing fixture function parameters, tuples of marks are simply merged into one. This probably isn't right but works for the purpose of this POC
• In the case you write the following, which you shouldn't do, the test will fail:

  @fixture.literal("foo").argumentize(arguments("bar"))
  def test_with_foo(foo):
      assert foo == "bar"

  Instead, foo will be == Arguments("bar"). I'm not sure if one other other behavior should be preferred here

[nicoddemus]

Hi @oakkitten,

Thanks for the well written post!

If I may suggest something, here's a proof of concept decorator that allows writing fixtures like this:

@fixture_taking_arguments
def dog(request, /, name, age=69):
    return f"{name} the dog aged {age}"

At first glance looks good, one issue is that the / is new in Python 3.8 only.

Fixture parametrization currently is driven either by tests (using indirect parametrization) or directly in the fixture using the params argument to @pytest.fixture. I agree both are not as friendly as they could be.

Here, to the left of / you have other fixtures, and to the right you have parameters that are supplied using:

@dog.arguments("Buddy", age=7)
def test_with_dog(dog):
    assert dog == "Buddy the dog aged 7"

I like how it looks, however fixtures in pytest should not really be imported, which makes the @dog.arguments impracticable, as test modules won't have the dog fixture available.

However we can get the same benefits by using a new mark:

@pytest.mark.fixture_args("dog", "Buddy", age=7)
def test_with_dog(dog):
    assert dog == "Buddy the dog aged 7"def

This however passes a single set of parameters to the dog fixture. If we also want to parametrize this so a set of parameters are sent to dog, resulting in multiple calls to test_with_dog (like a normal parametrization with indirect=True would), we need to think a bit more.

The other examples you propose have the same "problem" of needing access to the fixture function, as I mentioned is a bit problematic given the current good practices.

Another thought: the problem of specifying fixture parameters in tests and in fixtures can be tackled separately. I mean, currently one accesses fixtures parameters using request.param; we can improve that aspect independently from changing how parameters are passed from test functions to fixtures, and vice versa.

Btw: I know it can be a bit frustrating when you come up with a new syntax which seems better overall, however pytest has backward compatibility and complexity concerns as well (more than one way to do the same thing), so we need to balance those out.

Again thanks for the detailed proposal!

[oakkitten]

I like how it looks, however fixtures in pytest should not really be imported

Why not?

Either way, the call to dog.arguments or similar methods only really needs the name of the fixture. In the scenario when one can't import a fixture directly (not sure how that would be the case) they could do something like this perhaps:

dog = fixture.by_name("dog")

If we also want to parametrize this so a set of parameters are sent to dog

As I mention above, the following syntax could work; it will also work with the fixture produced by fixture.by_name()

@dog.argumentize("Champion", arguments("Buddy", age=7))
def test_with_dog(dog):
    ...

pytest has backward compatibility and complexity concerns as well

Unless I'm missing something this syntax would not introduce any breaking changes

Also, I hope that new syntax should be simple and straightforward enough to completely supercede the old one and become the “only” way to pass arguments around

---

By the way, I didn't think of fixure parametrization using params=... argument, but why not use the same syntax? For instance, the documentation gives the following example:

@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])
def smtp_connection(request):
    smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)
    ...

This could be written as:

@fixture(scope="module")
@parameters("server").argumentize(("smtp.gmail.com", "mail.python.org"))
def smtp_connection(request, /, server):
    smtp_connection = smtplib.SMTP(server, 587, timeout=5)
    ...

In fact, in the case where one wants to directly parametrize argumentize a single parameter, a syntax similar to fixture.by_name() could be used instead:

server = fixture.literal("server")

@fixture(scope="module")
@server.argumentize("smtp.gmail.com", "mail.python.org")
def smtp_connection(request, /, server):
    smtp_connection = smtplib.SMTP(server, 587, timeout=5)
    ...

Here you can think of literal() as of a special fixture that returns its parameters verbatim, and of @parameters(dog, owner, "poop") as of a shorthand for @parameters(dog, owner, literal("poop"))...

[RonnyPfannschmidt]

a agree wit the general intent but am really hesitant to currently introduce new and far-fetching apis,

my main driver for the hesitation is the amount of technical debt we have lingering in the fixture/fixture scoping/parameterize system that makes such changes extremely fragile right now

as far as i gather here the general desire is to have more and better ways to declarative configure pytests dependency injection/parameterize

there is a number of potential pitfalls/documentation-al pain points about how to configure things

so in order to give this effort a realistic chance of success (because i love the direction this is going, even if i don't necessarily agree about how this should look like yet, i want this to proceed to a success)
we should do a number of things

a) set up a basic project for consolidating the histroic issues of the fixture system, migrating them to a more flexible setupstate/caching system
b) iterate a a RFC style document on how to deal with referencing fixtures, passing parameters to fixtures and parameterize of fixtures and their parameters

its important to run trough a few iterations here as part of our internal issues is that the initial prototypes where driven by next features only and didn't have a design goal - with a growing api we would put ourselves into a bad mess if we didn't aim for a more consistent design while adding features one at a time

[oakkitten]

Just a note that I updated the proposal and also did a major update to my proof of concept to address comments by @nicoddemus and also to support fixture argumentization

[kousu]

Hey have you seen #3960 (available in pytest_cases)? It seems like a similar approach that could enhance your own thinking.

[oakkitten]

Is there anything in #3960 than this proposal/poc doesn't cover? With the exception of indirect parametrization of fixtures in fixture functions, which pytest_cases doesn't seem to support either, this poc allows argumentizing fixtures the in the exact same way that test functions are argumentized

---

old text

_pytest_cases_ also has `fixture_union` which should work but might benefit from first-class support. Perhaps something like this could work:

@fixture
def raccoon():
    return "Bob the raccoon"

@fixture.literal("animal").argumentize(
    "Todd the toad",
    raccoon,
    dog.arguments("Buddy", age=7),
    cat.argumentize("Tom", "Mittens")
)
def test_with_animal(animal):
    assert animal in [
        "Todd the toad",
        "Bob the raccoon",
        "Buddy the dog aged 7",
        "Tom the cat",
        "Mittens the cat"]

It would be kind of weird to not support argument*() here, I think, so this would naturally be a superset of the proposal “Parametrize with fixture”

...this, however, is only straightforward if the fixtures are function-scoped. If they are not, it would have to be decided when wider-scoped fixtures are set up and torn down, or whether the individual tests can be reused, or what happes when the value of literal().argumentize() is reused. On the other hand, this decision making could be left to pytest machinery if animal is instead made a fixture, perhaps like this:

animal = fixture.by_literal("animal", scope=...).argumentize("Todd the toad", raccoon, ...)

pytest_cases also has fixture_union which should work but might benefit from first-class support. I updated my poc to support a superset of this functionality:

@fixture
def raccoon(request, /):
    return "Bob the raccoon"

@fixture
def eagle(request, /, name="Jake"):
    return f"{name} the eagle"

@fixture.literal("name").argumentize("Matthew", "Bartholomew")
@fixture.literal("size").argumentize("big", "small")
@fixture
def sparrow(request, /, name, size):
    return f"{name} the {size} sparrow"

animal = fixture.create_sync("animal").argumentize(
    "Todd the toad",
    raccoon,
    eagle.arguments("William"),
    eagle.argumentize("Luke", arguments("Simon")),
    eagle,
    sparrow,
)

def test_with_animal(animal):
    assert animal in {
        "Todd the toad",
        "Bob the raccoon",
        "William the eagle",
        "Luke the eagle",
        "Simon the eagle",
        "Jake the eagle",
        "Matthew the big sparrow",
        "Matthew the small sparrow",
        "Bartholomew the big sparrow",
        "Bartholomew the small sparrow",
    }

This works in by creating a new non-async fixture in fixture.create_sync that calls getfixturevalue(). I'm not sure if there's a way to portably call getfixturevalue() in an async way? While this works, getfixturevalue() is probably an awful solution.

It probably is possible to make this work as a test decorator, e.g.

@fixture.literal("animal").argumentize(raccoon, dog.arguments("Buddy", age=7))
def test_with_animal(animal):
    ...

But this apparently requires different machinery than the one used by fixture.create_sync. Also, the implementation might complicated if one can argumentize fixtures with other fixtures, e.g. dog.arguments(cat.argumentize("Tom", "Mittens"))

[vorpal56]

one way I've been doing this:

@pytest.fixture
def fixture_1():
    return 1

@pytest.fixture
def service(some_other_fixture):
    def _fixture(service_name):
        return f"Service launched with parameter {service_name!r} {fixture_1}"

    return _fixture

def test_with_service(service):
    service_message = service("parameter")
    assert service_message == "Service launched with parameter 'parameter' 1"

@pytest.mark.parametrize("service_name", ["a", "b", "c"])
def test_with_varying_services(service_name, service):
    service_message = service(service_name)
    assert service_message == f"Service launched with parameter '{service_name}' 1"

[mahadi]

Stumbeled over this again today... Is this proposal still considered? Any further actions have been taken since 2021?

[network-shark]

Looks lovely ❤️ let's push it to main

[chrispy-snps]

I want to configure a fixture on a test-by-test basis. For most tests, I want the fixture configuration parameters to implicitly remain at their defaults. For some tests, I need to explicitly set one or more configuration parameters to nondefault values.

I just spent hours reading these links:

• https://docs.pytest.org/en/latest/example/parametrize.html#indirect-parametrization
• https://stackoverflow.com/questions/44677426/how-to-pass-arguments-to-pytest-fixtures
• https://stackoverflow.com/questions/18011902/how-to-pass-a-parameter-to-a-fixture-function-in-pytest

plus the discussion above, and wow... is it really this difficult to make a configurable fixture in Python? :(

Indirect fixture parameterization does not seem like a good fit. Its method of passing multiple parameters to a fixture is not intuitive, and the parameter values must be exhaustively enumerated instead of using defaults unless overridden.

The pattern of fixture-returns-a-function-that-makes-the-thing seems to be the closest match to what I need:

import pytest
from typing import Callable

# make a fixture that returns a function (with configurable parameters)
# that makes the thing (using those parameters)
@pytest.fixture
def make_thing() -> Callable:

  # the configuration parameters have default values
  def thing(
        a: int = 1,
        b: int = 1,
        c: int = 1
    ) -> int:
    return a*100 + b*10 + c

  return thing


# test at various default and non-default parameter behaviors
def test_default(make_thing: Callable):
  thing = make_thing()
  assert thing == 111

def test_hundreds(make_thing: Callable):
  thing = make_thing(a=3)
  assert thing == 311

def test_tens(make_thing: Callable):
  thing = make_thing(b=4)
  assert thing == 141

def test_ones(make_thing: Callable):
  thing = make_thing(c=5)
  assert thing == 115

Is there a better way to do this, either recently introduced in Python or under discussion somewhere?

[merlinz01]

Here's an option:

def get_param_from_request(
    request: pytest.FixtureRequest, name: str, default: Any = None
) -> Any:
    marker = request.node.get_closest_marker("fixture_params")
    if marker is None:
        return default
    return marker.kwargs.get(name, default)

@pytest.fixture
def fixt(request):
    return get_param_from_request(request, "some_parameter", "thisisthedefault")

@pytest.mark.fixture_params(some_parameter="some_value")
def test_parameterized_fixture(fixt):
    assert fixt == "some_value"

def test_unset_parameter(fixt):
    assert fixt == "thisisthedefault"