-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dependency resolution is extremely slow #4924
Comments
I'd assume that the issue is numpy here. Poetry has to check too many versions. |
Ok, I'm going to test, but for sure you have a point. I'm not sure if I can pin the exact |
Nope, even without Moreover, using Logs up to stack (same pyproject.toml without numpy)
Next batch, before stuck again
Network plot by btop (gracefully distorted by GitHub mono font...)
|
More updates:
|
Same issue for me base
|
Just tried this myself using 1.1.11 on @item4's pyproject.toml from above: > poetry lock -vvv
...
1: Version solving took 10.174 seconds.
1: Tried 1 solutions. Dunno if it matters, but I have this in my env: Just noticed you had 1.1.12, so I updated and retried: > poetry self update 1.1.12
> rm poetry.lock
> poetry lock -vv
1: Version solving took 0.146 seconds.
1: Tried 1 solutions. |
@jimmywan please, show even the Also, try to rerun after issuing poetry cache clear pypi --all to see the difference with and without cache. |
I've updated my comment above to reflect the fact that I ran that test using @item4 's pyproject.toml.
Below I reran this on a cleared cache using your pyproject.toml. Finished in ~90s.
❯ poetry cache clear pypi --all
Delete 881 entries? (yes/no) [no] yes
> poetry lock -vv
...
0: Duplicate dependencies for typing-extensions
0: Different requirements found for typing-extensions (>=3.10.0.0) with markers * and typing-extensions (!=3.10.0.1) with markers python_version >= "3.10".
1: Version solving took 17.517 seconds.
1: Tried 1 solutions.
0: Retrying dependency resolution with the following overrides ({Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}).
...
1: Version solving took 66.233 seconds.
1: Tried 1 solutions.
0: Retrying dependency resolution with the following overrides ({Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}).
...
1: Version solving took 0.253 seconds.
1: Tried 1 solutions.
0: Complete version solving took 84.019 seconds with 2 overrides Summary
|
Thank you for your data. 90 seconds it's more similar (same scale) but much more reasonable, for a cache-less installation. It's weird that apparently same tool in same conditions has such different results. Of course this means that the conditions are not the same, but it also yields that we are not controlling the very relevant differences. I guess possible differences are:
|
I checked this on the recent master, also works fine. |
@maciejskorski how can i try with master branch? is it just can use with --preview flag installation? |
@item4 For sure you can download the repo and install from the source :) |
Keep taking a lot for me. Still on
take hundreds of seconds, if not thousands. |
In my case, I can fix with change DNS settings. I used poetry in WSL2, and WSL2 change |
Added django to my very first brand new Poetry project and it took almost 1.5 hours to resolve. The "Creating virtualenv" step was also somewhere between 15 and 30 minutes. Finally the installing step took about 20 minutes.
System
|
I don't know much about the PyPI API, but it seems that poetry is querying package versions one-by-one. Isn't it possible to query all versions that satisfy a dependency and store them in a chache? If the API allows that it'd significantly reduce the number of network requests to PyPI which |
This might be relevant, but I wonder if it's really the bottleneck: even |
May it hits some individual API-query limits that forces a wait period? P.S.: If the api docs on rate limits is up-to-date then this doesn't seem to be a candidate for the issue(besides overall networking overhead). |
For those hitting this, I added a couple tips on this issue on how to improve issues with Poetry being slow: #2094 (comment) |
Thank you @john-sandall for your advice, unfortunately I've always been on Ubuntu with a download speed So, the amount of time it's time unexplained for me, but at least I got a reference to an older and still open issue, that I failed to found before (even though has practically my same title). I'm tempted to close this as duplicated. |
Similar or duplicated of:
I believe the best contribution I can give is to close this one, keep tracking the others. I suggest anyone that falls here to look instead to #2094, since it's the oldest one I'm aware of dealing with this issue. |
It will take a thousand years to catch pipenv. Tried almost all workaround and installing in fresh linux with a fresh empty project. Adding fastapi took 4329.2 s. I don't know what poetry does extra for keeping dependency check for me. But I really don't care. All i care is about time. Creating virtual env with cementing lock file for dependencies with pipenv took 0.2 sec. I am out |
Taking long might be frustrating, and I really hope it will improve over time. On the other hand this is not a frequent operation in development. So I just leave my computer doing it for me, while doing something else. In any case, you got it for free @ash1khan, so I see no reason why to complaint with people here, that are only trying to improve. If anything, consider to take part to the effort. Lines 14 to 21 in 9ca39b0
If you're just not interested in, that's it. No need to say anything more :) |
Can poetry consider a pattern install without checking for dependencies |
|
it helped me with Resolving dependencies... (millions seconds) |
This might work for you if you are working on any modern Linux-based distro. It's pretty easy to try. TDLR: Issues as IPv6 resolution is preferred over IPv4.
This solved my issues with Poetry and PIP being extremely slow to install packages. I've written a more extensive post on this workaround. Have a look if you want to get more information. |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
-vvv
option).Specs
OS: Linux 5.13 Ubuntu 21.10 21.10 (Impish Indri)
CPU: (12) x64 Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
1.1.12
pyproject.toml
Internet connection
Issue
Just running
poetry update
took 15 minutes to do exactly nothing:Even more, for other packages with a few more dependencies, I've seen
poetry
running for more than one hour without completing (I killed).In general, this behavior happens for all the sub-commands that potentially have to resolve dependencies, including
add
,update
, but eveninstall
.The text was updated successfully, but these errors were encountered: