Update pip to 22.0.4, setuptools to 60.10.0 and wheel to 0.37.1

[edmorley]

• Update pip from 21.3.1 to 22.0.4 for Python 3.7+
• Update setuptools from 57.5.0 to:
  • 59.6.0 for Python 3.6 (this is the most recent version that supports the EOL Python 3.6)
  • 60.10.0 for Python 3.7+
• Update wheel from 0.37.0 to 0.37.1 for Python 2.7 and Python 3.5+

The setuptools upgrade should resolve issues Pipenv users are seeing with conflicting setuptools versions causing errors like:

Traceback (most recent call last):
  ...
  File "/app/.heroku/python/lib/python3.10/site-packages/pipenv/environment.py", line 553, in get_distributions
    pkg_resources = self.safe_import("pkg_resources")
  File "/app/.heroku/python/lib/python3.10/site-packages/pipenv/environment.py", line 92, in safe_import
    six.moves.reload_module(module)
  File "/app/.heroku/python/lib/python3.10/importlib/__init__.py", line 169, in reload
    _bootstrap._exec(spec, module)
  File "<frozen importlib._bootstrap>", line 619, in _exec
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/.heroku/python/lib/python3.10/site-packages/pkg_resources/__init__.py", line 74, in <module>
    from pkg_resources.extern.jaraco.text import (
ModuleNotFoundError: No module named 'pkg_resources.extern.jaraco'

As seen in:
https://heroku.support/1079064
https://heroku.support/1085551
https://heroku.support/1088507
https://heroku.support/1093711
https://heroku.support/1094250

However, the setuptools upgrade also drops support for the deprecated use_2to3 feature, which may be used be a small number of legacy dependency versions. This change occurred in setuptools 58.0.0 (released 2021-09-04), which we have deliberately not updated to for as long as possible, in order to give any affected packages time to be fixed. However it's no longer viable to delay updating any longer, due to the pipenv compatibility issues.

For any packages that use this legacy 2to3 feature, package installation will now fail with:

Collecting Flask-OpenID==1.2.5
  Downloading Flask-OpenID-1.2.5.tar.gz (43 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in Flask-OpenID setup command: use_2to3 is invalid.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Affected apps will need to either:

• Update to a newer version of affected dependencies (in the hope that they contain a fix for this issue).
• Use a wheel (.whl) rather than a source distribution (plain source archive) of those dependencies, since wheels don't require the use of setuptools once created. For example by ensuring the Python version in use matches that of any already-published wheels (see the package's "Download files" section on https://pypi.org for which wheels exist), or by asking the package maintainer to upload wheels for an existing older release.
• Or, failing the above, report the issue to the package maintainers (they will need to remove the use_2to3 option from the project's setup.py file), and whilst waiting for a fix, pin your app's buildpack to the previous buildpack release (v209), using:
  https://devcenter.heroku.com/articles/buildpacks#buildpack-references

Note: Pinning buildpack version should be used as a short-term workaround only, and after any affected dependencies are fixed, apps should switch back to the default stable-release buildpack alias of heroku/python.

Release notes:
https://pip.pypa.io/en/stable/news/#v22-0-4
https://setuptools.pypa.io/en/latest/history.html#v60-10-0
https://github.com/pypa/wheel/blob/0.37.1/docs/news.rst

Commits:
pypa/pip@21.3.1...22.0.4
pypa/setuptools@v57.5.0...v60.10.0
pypa/wheel@0.37.0...0.37.1

Note: The --progress off addition is since the new version of pip includes improved progress bars, which have been disabled using in order to reduce build log noise and maintain parity with the existing log verbosity.

The new pip version was uploaded to S3 using:

$ pip download --no-cache pip==22.0.4
Collecting pip==22.0.4
...
Saved ./pip-22.0.4-py3-none-any.whl
Successfully downloaded pip
$ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl" --dryrun
(dryrun) upload: ./pip-22.0.4-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-22.0.4-py3-none-any.whl
$ aws s3 sync . s3://heroku-buildpack-python/common/ --exclude "*" --include "*.whl"
upload: ./pip-22.0.4-py3-none-any.whl to s3://heroku-buildpack-python/common/pip-22.0.4-py3-none-any.whl

GUS-W-10538676.

[edmorley]

https://devcenter.heroku.com/changelog-items/2412

[felipempinto]

I'm facing this issue because of this new release:

error in GDAL setup command: use_2to3 is invalid.

I need to use this buildpack to run GDAL with my Geodjango app.

The problem is, that this pack only allows me to use GDAL versions 2.4.0 and 2.4.2, and I think these versions are the problem (the most recent is 3.4). I could not find any workaround yet, but the last update on the repository was 2 years ago, so I don't have any other option but use setuptools version <=58.

Does someone have any idea about how to fix this?

[edmorley]

@felipempinto Thank you for letting me know about the issue with older GDAL - I'll see if I can build new library versions in https://github.com/heroku/heroku-geo-buildpack so newer versions of the Python packages can be used with them. For now I would pin Python buildpack version, and subscribe to updates on the Geo buildpack repository, so you will know when it has been updated :-)

[edmorley]

@felipempinto I've built/uploaded newer versions of GDAL/GEOS/PROJ for heroku-geo-buildpack.

Could you give them a try (by setting the env vars to select the new versions, since they aren't yet the default for that buildpack) and let me know how that goes?
https://github.com/heroku/heroku-geo-buildpack#default-versions
https://github.com/heroku/heroku-geo-buildpack#available-versions

[felipempinto]

@edmorley Great, thank you, I'll do this as soon as possible and let you know, my hope is to do this by tomorrow or until Sunday.

[felipempinto]

@edmorley Everything seems to be working fine right now, I made a new deployment using Heroku Git and added "GDAL_VERSION=3.5.0" to the environment variables, with the same version in my "requirements.txt", and it worked, thank you!

[edmorley]

@felipempinto Great! Thank you for letting me know, I will now merge heroku/heroku-geo-buildpack/pull/38 :-)