You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
I'm changing our virtualenv based solution to poetry. We're running massive amount of tests in batches on multiple machines.
Every time the batch starts we'll need to check if current checkout lock file is in sync with dependencies.
I checked how does poetry install compares with pip install -r requirements for a clean/in-sync repo and had seen big differences:
On my mac m1 pro poetry install takes 1.5s vs 700ms for pip (user time)
On our oldest macs that are running tests it's up to 6.5s vs 2s.
It's quite long for every test batch so I started digging.
I was even more surprised when I found out that poetry uses cleaver hashing trick to speed up such case - so why such a difference between brute-force pip install?
After a bit of profiling I saw that majority of time is spent in parsing toml lock file.
As a proof of concept I did a quick&dirty rewrite on packages/locker.py and used a tomli lib to parse lock.
My results are now 0.56s user time vs 1.5s. Haven't check on older macs yet.
My question is - how would you see using a simpler/faster lib for toml file parsing for time critical parts?
The text was updated successfully, but these errors were encountered:
Issue
Hi,
I'm changing our virtualenv based solution to poetry. We're running massive amount of tests in batches on multiple machines.
Every time the batch starts we'll need to check if current checkout lock file is in sync with dependencies.
I checked how does
poetry install
compares withpip install -r requirements
for a clean/in-sync repo and had seen big differences:On my mac m1 pro
poetry install
takes 1.5s vs 700ms forpip
(user time)On our oldest macs that are running tests it's up to 6.5s vs 2s.
It's quite long for every test batch so I started digging.
I was even more surprised when I found out that poetry uses cleaver hashing trick to speed up such case - so why such a difference between brute-force pip install?
After a bit of profiling I saw that majority of time is spent in parsing toml lock file.
As a proof of concept I did a quick&dirty rewrite on
packages/locker.py
and used atomli
lib to parse lock.My results are now 0.56s user time vs 1.5s. Haven't check on older macs yet.
My question is - how would you see using a simpler/faster lib for toml file parsing for time critical parts?
The text was updated successfully, but these errors were encountered: