Package complains about attr.__version__ but the alternative does not work

[timj]

Consider:

>>> import attr
>>> attr.__version__
<stdin>:1: DeprecationWarning: Accessing attr.__version__ is deprecated and will be removed in a future release. Use importlib.metadata directly to query for attrs's packaging metadata.
'23.1.0'

But the code wanting to get the version number from attr can't do this:

>>> import importlib.metadata as M
>>> M.version("attr")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/timj/work/lsstsw/miniconda/envs/lsst-scipipe-6.0.0/lib/python3.10/importlib/metadata/__init__.py", line 996, in version
    return distribution(distribution_name).version
  File "/Users/timj/work/lsstsw/miniconda/envs/lsst-scipipe-6.0.0/lib/python3.10/importlib/metadata/__init__.py", line 969, in distribution
    return Distribution.from_name(distribution_name)
  File "/Users/timj/work/lsstsw/miniconda/envs/lsst-scipipe-6.0.0/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for attr

This is fundamentally because there is no metadata for the attr package because it's all in the attrs package but the attrs package installs code into two separate namespaces and has no way of registering that fact with the installer.

The reason this is a problem for me is that we have some code that gathers version information for our running scientific pipeline code so that we can recreate the environment and have provenance information. The code doing this can't tell that attr is actually part of attrs and realizes that importlib.metadata does not work, so it falls back to __version__. This then results in a warning. I'm not sure what the solution is but thought I should mention the problem.

[hynek]

Hmm, how would your approach work with packages that are fundamentally named differently than the import name? Like the whole generation of packages that prepended their name with python- or py? E.g. python-dateutil? attrs is a bit special because the names are very similar, but in the end the name could be also python-attr…

[timj]

We don't know anything about PyPI distributions. We are scanning sys.modules and then asking the toplevel package names what their version is. We try with __version__ and importlib.metadata.version() but for the case of attr we get a warning saying that __version__ isn't allowed and importlib.metadata doesn't work either because we can't work out that attr has no metadata and should be looking for attrs.

It does sound like it's a fool's errand though. Python shifting to asking people to use importlib.metdata.version rather than __version__ means there is a fundamental disconnect between versioning of a module and versioning of a package. I don't understand how __version__ can be deprecated in this scenario. python-dateutil is a killer argument for importlib.metadata being hard to use unless you know the name of the pypi package -- how can people even work that out programmatically if all they know is that they've imported dateutil?

[timj]

If the problem is not wanting to have to keep __version__ updated, can you consider an alternative? In many of our packages we do something like this in the __init__.py:

from importlib.metadata import PackageNotFoundError, version

__version__: str

try:
    __version__ = version("attrs")
except PacakgeNotFoundError:
    # package is not installed
    __version__ = "0.0.0"

[hynek]

I'm afraid you're a bit on a lost cause here, since looking at __version__ is kinda deprecated.

Whether or not attrs removes it is one thing, but newer projects are less likely to have that attribute increasingly. Also how do you handle name-spaced packages like zope.interface vs zope.sqlalchemy? There's a lot of sharp edges waiting for you.

Have tried coming up with a solution that lists installed packages (i.e. the equivalent of pip list) and then inspect those? This is Python packaging, so there won't be one nice solution, but betting on __version__ that the community is largely moving away from doesn't seem ideal.

[timj]

Okay. I can't find a PEP deprecating __version__ but maybe I'm not looking in the right place. We want to know the versions of the packages we are using, not all the packages that are installed, so pip list isn't going to work for us. Sounds like I'm wasting my time even trying to work it out.

For namespaced packages I look for the version in zope first and if I find one I use that. Then if there is no version I look for the version in zope.interface. This seems to work reasonably well modulo the complete disconnect between PyPI package names and python modules.

[hynek]

I'm sorry if I'm being obnoxious but the thing is: there's no PEP for __version__ either.

As far as I know, it's mentioned en passant in PEP 8 and was part of a PEP that got ultimately rejected: https://peps.python.org/pep-0396/

I'm not trying to be difficult here and postponing the removal is literally no work for me at all. It just seems to me that you're running on borrowed time here.

I'll remove the __version__ deprecation for now, but can't promise it will stay forever.

[timj]

Thanks. I see that pydoc does use __version__ to determine the version number when reporting documentation so there is still usage in python core for it.

[flying-sheep]

importlib.metadata.packages_distributions() gives you a mapping between importable modules and distribution packages.

If you have a import name and want to get the distribution package metadata (i.e. the actually versioned entity), you can use this API or its backport in importlib-metadata:

 [project]
 ...
 dependencies = [
+  'importlib_metadata; python_version < "3.10"',
 ]

try:
    from importlib.metadata import packages_distributions
except ImportError:
    from importlib_metadata import packages_distributions

As said: __version__ is merely a convention and not guaranteed to be there, following a certain format, be of type str, or in any other way usable. Don’t use it.