2.3.1: pytest is failing in one unit

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-enabler-2.3.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-enabler-2.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.17, pytest-7.4.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-enabler-2.3.1
configfile: pytest.ini
plugins: enabler-2.3.1, black-0.3.12, cov-4.0.0
collected 6 items

pytest_enabler/__init__.py .                                                                                                                                                          [ 16%]
tests/test_enabler.py F....                                                                                                                                                           [100%]

========================================================================================= FAILURES ==========================================================================================
_______________________________________________________________________________ test_pytest_addoption_default _______________________________________________________________________________

    def test_pytest_addoption_default():
        config = mock.MagicMock()
        config.pluginmanager.has_plugin = lambda name: name == 'black'
        args = []
        enabler.pytest_load_initial_conftests(config, None, args)
>       assert args == ['--black']
E       AssertionError: assert [] == ['--black']
E         Right contains one more item: '--black'
E         Use -v to get more diff

tests/test_enabler.py:21: AssertionError
================================================================================== short test summary info ==================================================================================
FAILED tests/test_enabler.py::test_pytest_addoption_default - AssertionError: assert [] == ['--black']
================================================================================ 1 failed, 5 passed in 0.53s ================================================================================

Here is list of installed modules in build env

Package                       Version
----------------------------- -------
alabaster                     0.7.13
asttokens                     2.2.1
Babel                         2.12.1
backcall                      0.2.0
black                         23.7.0
build                         0.10.0
charset-normalizer            3.1.0
click                         8.1.3
coverage                      7.2.7
decorator                     5.1.1
distro                        1.8.0
docutils                      0.19
domdf-python-tools            3.6.1
exceptiongroup                1.1.1
executing                     1.2.0
gpg                           1.20.0
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.7.0
importlib-resources           6.0.0
iniconfig                     2.0.0
installer                     0.7.0
ipython                       8.12.0
jaraco.context                4.3.0
jaraco.functools              3.8.0
jaraco.packaging              9.4.0
jedi                          0.18.2
Jinja2                        3.1.2
libcomps                      0.1.19
MarkupSafe                    2.1.2
matplotlib-inline             0.1.6
more-itertools                9.1.0
mypy-extensions               1.0.0
natsort                       8.3.1
packaging                     23.1
parso                         0.8.3
pathspec                      0.11.1
pexpect                       4.8.0
pickleshare                   0.7.5
platformdirs                  3.8.1
pluggy                        1.0.0
prompt-toolkit                3.0.38
ptyprocess                    0.7.0
pure-eval                     0.2.2
Pygments                      2.15.1
pyproject_hooks               1.0.0
pytest                        7.4.0
pytest-black                  0.3.12
pytest-cov                    4.0.0
python-dateutil               2.8.2
pytz                          2023.2
requests                      2.31.0
rst                           0.1
rst.linker                    2.4.0
setuptools                    68.0.0
setuptools-scm                7.1.0
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        6.2.1
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
stack-data                    0.6.2
toml                          0.10.2
tomli                         2.0.1
traitlets                     5.9.0
typing_extensions             4.7.1
urllib3                       1.26.15
wcwidth                       0.2.6
wheel                         0.40.0
zipp                          3.15.0

[jaraco]

My guess is that default.toml is missing from your build (and install). Can you confirm in the environment with:

import importlib_resources
print(importlib_resources.files('pytest_enabler').joinpath('default.toml').is_file())

It should emit "True".

If it's false, it confirms my suspicion and you should ensure than a MANIFEST or manifest template exists to include that file in the build.

[kloczek]

My guess is that default.toml is missing from your build (and install). Can you confirm in the environment with:

import importlib_resources
print(importlib_resources.files('pytest_enabler').joinpath('default.toml').is_file())

It should emit "True".

If it's false, it confirms my suspicion and you should ensure than a MANIFEST or manifest template exists to include that file in the build.

This content of the generated rpm package

# rpm -ql python-pytest-enabler
/usr/lib/python3.8/site-packages/pytest_enabler
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/LICENSE
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/METADATA
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/RECORD
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/WHEEL
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/entry_points.txt
/usr/lib/python3.8/site-packages/pytest_enabler-2.3.1.dist-info/top_level.txt
/usr/lib/python3.8/site-packages/pytest_enabler/__init__.py
/usr/lib/python3.8/site-packages/pytest_enabler/__pycache__
/usr/lib/python3.8/site-packages/pytest_enabler/__pycache__/__init__.cpython-38.opt-1.pyc
/usr/lib/python3.8/site-packages/pytest_enabler/__pycache__/__init__.cpython-38.pyc
/usr/share/man/man3/python-pytest-enabler.3.gz

and indeed there is no default.toml. During package build on test suite execution above is in </install/prefix> which is passed over $PYTHONPATH.

Here is pep517 based build output:

+ SETUPTOOLS_SCM_PRETEND_VERSION=2.3.1
+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
running egg_info
creating pytest_enabler.egg-info
writing pytest_enabler.egg-info/PKG-INFO
writing dependency_links to pytest_enabler.egg-info/dependency_links.txt
writing entry points to pytest_enabler.egg-info/entry_points.txt
writing requirements to pytest_enabler.egg-info/requires.txt
writing top-level names to pytest_enabler.egg-info/top_level.txt
writing manifest file 'pytest_enabler.egg-info/SOURCES.txt'
reading manifest file 'pytest_enabler.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'pytest_enabler.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/pytest_enabler
copying pytest_enabler/__init__.py -> build/lib/pytest_enabler
running egg_info
writing pytest_enabler.egg-info/PKG-INFO
writing dependency_links to pytest_enabler.egg-info/dependency_links.txt
writing entry points to pytest_enabler.egg-info/entry_points.txt
writing requirements to pytest_enabler.egg-info/requires.txt
writing top-level names to pytest_enabler.egg-info/top_level.txt
reading manifest file 'pytest_enabler.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'pytest_enabler.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pytest_enabler
copying build/lib/pytest_enabler/__init__.py -> build/bdist.linux-x86_64/wheel/pytest_enabler
running install_egg_info
Copying pytest_enabler.egg-info to build/bdist.linux-x86_64/wheel/pytest_enabler-2.3.1-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/pytest_enabler-2.3.1.dist-info/WHEEL
creating '/home/tkloczko/rpmbuild/BUILD/pytest-enabler-2.3.1/dist/.tmp-nweq5r3n/pytest_enabler-2.3.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'pytest_enabler/__init__.py'
adding 'pytest_enabler-2.3.1.dist-info/LICENSE'
adding 'pytest_enabler-2.3.1.dist-info/METADATA'
adding 'pytest_enabler-2.3.1.dist-info/WHEEL'
adding 'pytest_enabler-2.3.1.dist-info/entry_points.txt'
adding 'pytest_enabler-2.3.1.dist-info/top_level.txt'
adding 'pytest_enabler-2.3.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built pytest_enabler-2.3.1-py3-none-any.whl

As you see default.toml does not come to generated .whl file.
In other words it is issue with description of what needs to be packaged.

[kloczek]

Looks like as pep517 backend is used setuptools-scm and in the repo there is no MANIFEST file so when it is build out of autogenerated from git tag tar ball nothing provides info that default.toml should be packaged into .whl file

[kloczek]

Looks like as pep517 backend is used setuptools-scm

No .. wrong. It is used setuptools (build-backend = "setuptools.build_meta") so MANIFEST files needs to be provided.

[kloczek]

.. or details that default.toml needs to be included in .whl can be added as well over entries in pyproject.toml.

[jaraco]

Right - skeleton-based projects require setuptools_scm and a checkout of the git metadata to resolve tracked files and the version. Downloading a git tarball will produce an invalid artifact. Since you're working from a git tarball, it's the downstream builder's responsibility to reconstruct the git metadata (through a MANIFEST file). Probably the best long-term solution here is to build a routine that constructs a MANIFEST file from a tarball and run that prior to building any Python packages. That would address this issue and any others where files are missing due to missing git metadata.

I suspect the solution is similar to that of jaraco.text, which also has package data present and relies on SCM metadata to ensure jaraco.text.Lorem Ipsum.txt is present in the package. What did you do to support that package?

[kloczek]

OK so what now? 🤔

[jaraco]

I found by adding a MANIFEST.in causes all files that were in the tarball to be included:

 pytest-enabler-2.3.1 @ cat >> MANIFEST.in
include **/*

Maybe that's a suitable workaround?

[jaraco]

I presume adding the manifest works around the issue. Feel free to comment if further investigation is needed or if there's something specific this project can do to help.