[BUG] 62.0.0: pkg_resources module discards current sys.path sets it to default

[kloczek]

setuptools version

62.0.0

Python version

3.8.13

OS

Linux/x86_64

Additional environment information

N/A

Description

Looksl like pkg_resources module discards current sys.path sets it to default which makes especially annoing when in sphinx copy.py fiel needs to setup path to module which documentation needs to be generated.

Expected behavior

pkg_resources module should not discard current value of the sys.path.

How to Reproduce

• git clone https://github.com/zopefoundation/transaction
• apply below patch

--- a/docs/conf.py~     2020-12-11 11:41:37.000000000 +0000
+++ b/docs/conf.py      2021-12-25 23:58:01.750032165 +0000
@@ -18,7 +18,7 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #sys.path.insert(0, os.path.abspath('.'))
-sys.path.append(os.path.abspath('../'))
+sys.path.append(os.path.abspath('../src'))
 rqmt = pkg_resources.require('transaction')[0]

 # -- General configuration -----------------------------------------------------

• execute in project tree strace -fe trace=file /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man 2>&1 | grep transaction

Output

[tkloczko@devel-g2v transaction-3.0.1]$ strace -fe trace=file /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man 2>&1 | grep transaction
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
newfstatat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs", {st_mode=S_IFDIR|0755, st_size=340, ...}, 0) = 0
newfstatat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/build/sphinx/man", 0x7fff69d600e0, 0) = -1 ENOENT (No such file or directory)
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
newfstatat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", {st_mode=S_IFREG|0644, st_size=8878, ...}, 0) = 0
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1", 1024) = 48
chdir("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs") = 0
openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", O_RDONLY|O_CLOEXEC) = 3
getcwd("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs", 1024) = 53
newfstatat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", {st_mode=S_IFREG|0644, st_size=8878, ...}, 0) = 0
openat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", O_RDONLY|O_CLOEXEC) = 3
chdir("/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1") = 0
newfstatat(AT_FDCWD, "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", {st_mode=S_IFREG|0644, st_size=8878, ...}, 0) = 0
  File "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", line 22, in <module>
    rqmt = pkg_resources.require('transaction')[0]
pkg_resources.DistributionNotFound: The 'transaction' distribution was not found and is required by the application
  File "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", line 22, in <module>
    rqmt = pkg_resources.require('transaction')[0]
pkg_resources.DistributionNotFound: The 'transaction' distribution was not found and is required by the application
  File "/home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/docs/conf.py", line 22, in <module>
    rqmt = pkg_resources.require('transaction')[0]
pkg_resources.DistributionNotFound: The 'transaction' distribution was not found and is required by the application

On above otput there is no try to opening /home/tkloczko/rpmbuild/BUILD/transaction-3.0.1/src/transaction.{egg|dist}-info.

[abravalheri]

Hi @kloczek, the expectation that require automatically responds to manipulation of sys.path is not guaranteed by setuptools.

The following is what the docs have to say about this topic:

The following methods of WorkingSet objects are also available as module-level functions in pkg_resources that apply to the default working_set instance. Thus, you can use e.g. pkg_resources.require() as an abbreviation for pkg_resources.working_set.require():

It's important to note that the global working_set object is initialized from sys.path when pkg_resources is first imported, but is only updated if you do all future sys.path manipulation via pkg_resources APIs. If you manually modify sys.path, you must invoke the appropriate methods on the working_set instance to keep it in sync.

[kloczek]

I think that sooner or later such problems will gone if more modules will replace use of pkg_resources by importlib.metadata.
What do you think about that direction? 🤔

[abravalheri]

Yes, I believe the eventual migration from pkg_resources to importlib.{metadata,resources} is the direction the community is moving towards.

[kloczek]

Thank you for the confirmation.
I've started raising some RFEs in modules which found that they are using pkg_resources.
Will try to keep below list updated:
python-babel/babel#861
jaraco/cssutils#22
pytest-dev/py#282
pygments/pygments#2116 -> done
pyparsing/pyparsing#391 -> done
executablebooks/sphinx-tabs#159
pypa/wheel#454
pytest-dev/pytest#9884
mcmtroffaes/sphinxcontrib-bibtex#298 -> done
ecmwf/cfgrib#298
Pylons/pastedeploy#31
AnalogJ/lexicon#1240 -> done
Pylons/waitress#380
sphinx-contrib/youtube#35 -> done
cherrypy/cherrypy#1973

[kloczek]

I've updated list of modules which are still using pkg_resources.
Below are sucessfull migrations:
mcmtroffaes/sphinxcontrib-bibtex#299