Poetry 1.2.1 install fails to install packages with pip dependency

[SfinxCZ]

• Poetry version: 1.2.1

• Python version: 3.8.12

• OS version and name: macOS 12.6 but same problem observed in docker built with python:3.8 as base image

• pyproject.toml: https://gist.github.com/SfinxCZ/a8a80b14cc19c6631942adb2fb6ca3ff

• I am on the latest stable Poetry version, installed using a recommended method.

• I have searched the issues of this repo and believe that this is not a duplicate.

• I have consulted the FAQ and blog for any relevant entries or release notes.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

After upgrading to poetry 1.2.1 I have encountered an issue where poetry fails to install project linked above (pyproject.toml). From what I've observed, the problem seems to occur, when the project depends on a library that has pip as its dependency (chalice in this case). Then, during the installation poetry tries to upgrade pip to version listed in the poetry.lock file. However, other libraries that are being installed at the same moment require pip to be installed and installation fails - see the log here install.log.

Reproducing this error is tricky as it seems to be a raise condition that depends on the parallelism used in the install step.

Downgrading to poetry 1.2.0 seems to fix the problem.

[neersighted]

cc @radoering @dimbleby

This is related to walking back the use of the virtualenv seed pip -- I think we may need to add a special case for pip and upgrade it first (with no parallelism) when we are manipulating it to avoid this.

[dimbleby]

If that's right then:

• it has long been broken (albeit briefly not broken at 1.2.0): poetry 1.1.x did not prefer the embedded pip
• it's easy to work around (upgrade pip before running poetry)
• a better fix is getting near (Faster build and installation of packages #6205 so that poetry doesn't rely on pip here at all)

suggest that if someone can think of a cheap fix then great, but in that context I submit that it's not worth a lot of effort

[neersighted]

Poetry 1.1.x will not actually manipulate pip at all -- it is effectively blind to pip (try adding pip to a pyproject.toml, then check the lock file using 1.1.15).

This is a new issue, but arguably not a regression as treating pip as just another dependency is a new capability of Poetry 1.2.

Edit: upgrading pip before running Poetry is not really a solution as Poetry may want to change the version (or reinstall if the direct origin URL is different) based on the lock file -- Poetry would need to upgrade pip first before running the concurrent installer on all other dependencies.

[dimbleby]

re upgrading pip: indeed it would be necessary to update to whatever version was written in the lock file. In this case that is the latest but ack that in principle it might not be

[dimbleby]

maybe that poetry 1.1 behaviour suggests the cheap and good-enough-for-now fix is to reinstate pip among the UNSAFE_PACKAGES

That's still broken in principle - what if the project requires a version of pip that's different than the version that's currently in the environment?

But depending on pip is pretty rare, then it's really quite unlikely that "some recent pip" is not good enough - it's probably a sticking plaster that would hold for some time.

[neersighted]

I'm not sure adding pip back to UNSAFE_PACKAGES is the best call here -- I intend to finish #6458 once the next pip version is released. It will solve this issue for every package but Poetry installed in editable mode installing itself (which to be fair, is very rare and specific, but annoying for us people working on Poetry) -- I'd like to try and solve this if it's not too much of a headache as the existing pip-based install path will remain in the codebase for some time.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.