Pin or upgrade pip version before installing requirements

[Gerrit-K]

tl;dr: Is there a way to pin the pip version or force pants to upgrade it before installing any other requirements, e.g. when I run the test or package goals?

I'm developing on a MacBook and recently updated to MacOS 11. Since then I'm experiencing seeminlgy endless requirements builds in pants. My theory to this is as follows:

Unfortunately there was a bug in the pip packaging code which broke backwards compatibility for all binary wheels (in short: pip didn't "know" that MacOS 10.X wheels are compatible with MacOS 11). As a result pip neglected them and instead attempted to compile the wheels from source, which is slow and in many cases even unsuccessful due to missing dependencies. This issue got fixed in pip 20.3. However, the current python distributions don't seem to be shipped with this version (yet), which means that pants has an outdated pip version, too, and thus attempts to compile wheels whenever I run the test or package goals (and probably others, too).

To circumvent this, it would be nice if I could simply tell pants to bootstrap a specific pip version or to call pip install --upgrade pip before attempting to install any further requirements. Is this possible?

[jsirois]

In short - no.

Pants uses Pex for resolving and installing requirements. Pex in turn vendors a fixed version of Pip to do most of the heavy lifting. See here for the latest, but its Pip version 20.0.dev0 with a patch: https://github.com/pantsbuild/pex/tree/master/pex/vendor

There are two remedies coming:

1. Pex will be upgrading its Pip release to bring in support for the new Pip resolver as well as modern macOS support.
2. Pex will be feeling out community support for dropping its Pip patch and the functionality it enabled.

Item 1 is the sole focus of the Pex 2.1.23 release which will be happening next week. Pants should be upgrading to that immediately at that point.

[Gerrit-K]

Thanks for the quick and detailed response. If pants depends on a patched version of pip it makes sense that it cannot be changed that easily. However, it's great to hear that a fix is on its way! For me, that (hopefully) solves the underlying issue and even if it didn't, the question would probably better fit the pex repo, as you said.
/close

[jsirois]

@Gerrit-K if you want to track the Pex Pip upgrade see pex-tool/pex#1133 (this was easier than expected) and the Pex release it will be in see pex-tool/pex#1134.