Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixtures: match fixtures based on actual node hierarchy, not textual nodeids #11785

Merged
merged 1 commit into from
Jan 8, 2024

Conversation

bluetech
Copy link
Member

@bluetech bluetech commented Jan 7, 2024

Refs #11662.

Problem

Each fixture definition has a "visibility", the FixtureDef.baseid attribute. This is nodeid-like string. When a certain node requests a certain fixture name, we match node's nodeid against the fixture definitions with this name.

The matching currently happens on the textual representation of the nodeid - we split node.nodeid to its "parent nodeids" and then check if the fixture's baseid is in there.

While this has worked so far, we really should try to avoid textual manipulation of nodeids as much as possible. It has also caused problem in an odd case of a Package in the root directory: the Package gets nodeid ., while a Module in it gets nodeid test_module.py. And textually, . is not a parent of test_module.py.

Solution

Avoid this entirely by just checking the node hierarchy itself. This is made possible by the fact that we now have proper Directory nodes (Dir or Package) for the entire hierarchy.

Also do the same for _getautousenames which is a similar deal.

The iterparentnodeids function is no longer used and is removed.

@bluetech
Copy link
Member Author

bluetech commented Jan 7, 2024

Looking at plugins from my local corpus:

Affected by getfixturedefs change:

  • pytest-tipsi-django
  • pytest-factoryboy
  • python-pytest-harvest
  • pytest-missing-fixtures
  • python-pytest-cases
  • pytest-bdd

Affected by _getautousenames change:

  • python-pytest-cases

Affected by iterparentnodeids removal:

  • pytest-bdd

I've considered deprecations but since these are private APIs and the fixes are not very difficult, I've decided not to. Instead, I added a changelog entry and will open issues for the projects above if this PR is merged.

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally 🎉🍾🎆

Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup!

changelog/11785.trivial.rst Outdated Show resolved Hide resolved
- ``FixtureManager._getautousenames()`` now takes a ``Node`` itself instead of the nodeid.
- ``FixtureManager.getfixturedefs()`` now takes the ``Node`` itself instead of the nodeid.
- The ``_pytest.nodes.iterparentnodeids()`` function is removed without replacement.
Prefer to traverse the node hierarchy itself instead.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prefer to traverse the node hierarchy itself instead.
Prefer to traverse the node hierarchy itself using ``_pytest.nodes.iterparentnodes()`` instead.

Copy link
Member

@nicoddemus nicoddemus Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I now realize that you might not want to advertise the internal function on purpose -- if that's the case feel free to ignore this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I don't mean to expose it. I think it can be nice as a Node method, maybe later.

…nodeids

Refs pytest-dev#11662.

--- Problem

Each fixture definition has a "visibility", the `FixtureDef.baseid`
attribute. This is nodeid-like string. When a certain `node` requests a
certain fixture name, we match node's nodeid against the fixture
definitions with this name.

The matching currently happens on the *textual* representation of the
nodeid - we split `node.nodeid` to its "parent nodeids" and then check
if the fixture's `baseid` is in there.

While this has worked so far, we really should try to avoid textual
manipulation of nodeids as much as possible. It has also caused problem
in an odd case of a `Package` in the root directory: the `Package` gets
nodeid `.`, while a `Module` in it gets nodeid `test_module.py`. And
textually, `.` is not a parent of `test_module.py`.

--- Solution

Avoid this entirely by just checking the node hierarchy itself. This is
made possible by the fact that we now have proper `Directory` nodes
(`Dir` or `Package`) for the entire hierarchy.

Also do the same for `_getautousenames` which is a similar deal.

The `iterparentnodeids` function is no longer used and is removed.
@bluetech
Copy link
Member Author

bluetech commented Jan 8, 2024

Temporarily disabled pytest-bdd plugin check until it's updated

@bluetech bluetech merged commit 97dfc34 into pytest-dev:main Jan 8, 2024
23 of 24 checks passed
@bluetech bluetech deleted the matchfactories-nodes branch January 8, 2024 20:23
bluetech added a commit to bluetech/pytest-factoryboy that referenced this pull request Jan 8, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785

I verified that all tests pass when run against pytest main.
bluetech added a commit to bluetech/pytest-bdd that referenced this pull request Jan 8, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785

Additionally, the `iterparentnodeids` function is removed, so
copy/pasting it for now.

I verified that all tests pass when run against pytest main.
bluetech added a commit to bluetech/python-pytest-harvest that referenced this pull request Jan 8, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785
bluetech added a commit to bluetech/python-pytest-harvest that referenced this pull request Jan 8, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785
bluetech added a commit to bluetech/python-pytest-cases that referenced this pull request Jan 8, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785
bluetech added a commit to bluetech/python-pytest-cases that referenced this pull request Jan 11, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785
smarie pushed a commit to smarie/python-pytest-cases that referenced this pull request Jan 12, 2024
The signature of this (private) function will change in the upcoming
pytest 8.1 release:
pytest-dev/pytest#11785
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants