Only call pip compile once

[jerbob92]

Avoid unnecessary pip-compile processes, and explain why we need to run an extra one sometimes.

Closes #5903.

[jurre]

Thanks for digging into this, would be interesting to get some data on how this effects real-world workloads!

[jerbob92]

@jurre I have just ran the change on a real-world project of ours. It has 3 .in/.txt combinations (ci, dev and prod). It has a total of 33 direct dependencies and ~150 indirect dependencies. When running the dry run for one dependency where there is a newer version available, these are the run times:

• On current main branch: 4m21.778s
• With my changes: 1m52.702s

As you can see the improvement is quite significant. This is also because it has to run pip-compile for every .in/.txt combination. So you can imagine the total time profit if it's 2.5m faster for every dependency that has a newer version available (it always run this logic, even if the rules don't allow going to this new version, it only doesn't run it when the latest version available is your current version).

In this case the update was not allowed, due to requirements, so the total time profit could be even bigger because it also runs pip-compile twice in the updater itself.

[jurre]

Seems very promising, especially since we see just one test fail. Anything we can do to help from our end?

[deivid-rodriguez]

I think my suggestion from #5903 (comment) should hopefully get things green!

[deivid-rodriguez]

Awesome speed up! 🎉

[Nishnha]

Approving based on the discussion in #5903

[deivid-rodriguez]

Thanks so much for this @jerbob92! 🎉