Replace deprecated imp module with importlib

[hugovk]

This project uses the imp module which has been deprecated since Python 3.4 and removed in 3.12:

• Raised PendingDeprecationWarning since 3.4 (2014)
• Raised DeprecationWarning since 3.5 (2015)
• Updated DeprecationWarning to say removal in 3.12 since 3.10 (2021)
• Removal planned for 3.12 (2023)

Python 3.12 is set for release on 2023-10-02 and this library is one of the top 5,000 most-downloaded from PyPI.

Please could you upgrade to use importlib? The imp docs have suggestions on what to use to replace each function and constant.

[kaos]

Hi @hugovk

Please note that pex still supports Python 2, so it's not as easy to just remove the use of imp as importlib was introduced first in Python 3.1.

https://github.com/pantsbuild/pex/blob/ff220b9e41484450e0c78136f98d6899f2e2325c/pyproject.toml#L19-L28

[hugovk]

Fair enough, some compatibility code will be needed.

Heads up: GitHub Actions is removing Python 2.7 in a couple of weeks: actions/runner-images#7401

[jsirois]

Unfortunately compatibility code will not do the trick. All the imp use is in vendored code:

$ git grep -e "import imp\b" -e "from imp import"
pex/vendor/_vendored/packaging_20_9/packaging/tags.py:    import imp
pex/vendor/_vendored/pip/pip/_internal/utils/compat.py:    import imp
pex/vendor/_vendored/pip/pip/_vendor/distlib/_backport/misc.py:    from imp import cache_from_source
pex/vendor/_vendored/pip/pip/_vendor/distlib/compat.py:        from imp import cache_from_source
pex/vendor/_vendored/pip/pip/_vendor/distlib/wheel.py:import imp
pex/vendor/_vendored/pip/pip/_vendor/packaging/tags.py:    import imp
pex/vendor/_vendored/pip/pip/_vendor/pkg_resources/__init__.py:    import imp as _imp
pex/vendor/_vendored/setuptools/pkg_resources/__init__.py:    import imp as _imp
pex/vendor/_vendored/setuptools/setuptools/_vendor/packaging/tags.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/command/build_ext.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/py27compat.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/py27compat.py:    from imp import PY_COMPILED, PY_FROZEN, PY_SOURCE  # noqa
pex/vendor/_vendored/setuptools/setuptools/site-patch.py:                import imp  # Avoid import loop in Python 3
pex/vendor/_vendored/wheel/wheel/vendored/packaging/tags.py:    import imp
pex/venv/virtualenv_16.7.12_py:        import imp

IOW Pex vendors an old single-file version of virtualenv to support Pythons without -mvenv support and a bootstrap Pip that works with all versions of Python Pex supports. It uses the vendored Pip to upgrade to newer versions of Pip it supports when asked, but the baseline vendored Pip must work with all versions of Python we support.

I had not created a tracking issue yet for the Pex 3.x release, but that fundamentally needs to decide what to do about vendored Pip and legacy support; i.e.: maintain two release branches - one supporting >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<3.12 and one supporting >=3.7,<3.next - or just cut the Pex 2.x series off and our existing Python 2.7 users just have to deal with that.

Thanks for the GHA heads up, I'll switch to pyenv-action where needed. Pex already uses pyenv to maintain a pool of always-available 2.7 and 3.{8,9,10} CPythons and that works well.

[jsirois]

Ah - this threw me for a loop. It's true that pre-installed Python 2.7 in the tool cache on the image goes away, but then actions/setup-python just grabs the release from https://github.com/actions/python-versions/releases via https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json. I have a hard time believing they'll nuke pre-built binary releases, at least I'm not worried about them doing that prior to Pex deciding and implementing a 3.x plan.

[jsirois]

@hugovk as noted in #2170, the vendored Pip code that uses ImpLoader remains and will as long as I can manage to support legacy Pythons without too much trouble - which is the case so far. The relevant Pex code, though, is now fixed to work 1st class with Python 3.12+; thus marking this closed.

[hugovk]

Thank you!