Dependency resolver in 21.2.1 takes several hours & crashes, whereas 21.1.3 takes 26 to 34 seconds #10230
Closed
1 task done
Labels
type: bug
A confirmed bug or unintended behavior
Comments
makyen
added
S: needs triage
makyen
changed the title
|
I can confirm it is happening as well with python |
|
Combining this into #10201. |
|
FYI I am able to reproduce this issue by merging this in to a single requirements file like so: Using the optimization to backtracking I propose here I am able to install these requirements and I end up these wheels: Hope this helps you figure out some better requirements file in the mean time. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
In the SmokeDetector project we started noticing problems with our CI testing using GitHub Actions not completing within the 6 hour maximum runtime and/or crashing after several hours starting on 2021-07-25: (crash after 5 hours on Python 3.9; canceled after 6 hours on Python 3.6). We resolved the issue, from our point of view, by forcing the use of pip 21.1.3 (i.e. 21.2.1 is completely broken for us and unusable).
Seeing what happens is complicated by the use of dependency caching. pip 21.2.1 can succeed under some conditions when starting from cached dependencies. In order to have a better test case, I created two branches to not use dependency caching.
One branch, Mak-pip3-21.1.3-no-cache, uses pip 21.1.3. The CI testing completed normally, with dependency installation taking 34 seconds in Python 3.6 and 26 seconds in Python 3.9.
The other branch, Mak-pip-latest-no-cache, uses pip 21.2.1. As of this point, the CI testing has been running for about an hour on the dependency installation step for both Python 3.6 and 3.9. Based on past experience, I expect it to continue for at least 5 hours and then either crash or be automatically canceled after 6 hours.
Expected behavior
I expect the dependency installation/resolution to take a time comparable to what it does with 21.1.3, so in the ballpark of 25 to 45 seconds.
pip version
21.2.1
Python version
3.6 and 3.9
OS
Ubuntu 20.04.2 LTS (GitHub Actions virtual env 20210726.1)
How to Reproduce
See CI testing linked in the description for the SmokeDetector repository. I expect the steps to be:
The CI testing executes the following commands:
With a requirements.txt:
And with a user_requirements.txt:
Output
[Note 1: In order to get this output in a timely manner, I manually canceled an identical run to the above linked Python 3.9 GitHub Actions CI testing for SmokeDetector commit b415143.
Note 2: There are some ReadTimeoutErrors near the end of this. While those may appear to by symptomatic, they occur after the resolver has been running for >100 times longer than the 21.1.3 takes to fully complete the process.]
Code of Conduct
The text was updated successfully, but these errors were encountered: