refactor: Flatten update lookup tasks

[zharinov]

Changes

• Collect all dep fetches for all package files for all managers into the single array
• Process it concurrently (with the limit of simulataneous runs)
• Adjust error handling to behave similarly to current functionality

Context

• Ref: Refactor: separate lookup from filter+decide stages #6575

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[zharinov]

This one should be tested well

[zharinov]

I haven't formalized my thoughts yet, but here we may want the single queue to be cancelable. Because, even we get ExternalHostError, fetching continues to some degree for both current and this new implementations.

[zharinov]

@rarkins Could you provide a hint of how lookup error handling should work? Given we have flat list of lookup tasks performed concurrently, it's not clear how to deal with errors:

• Should some errors stop processing completely? (i.e., for all hosts and for all managers)
• Should some errors stop further requests to particular host, while others are processed as usual?
• Should we log/collect other types of errors?

[rarkins]

Needs rate limiting first

[viceice]

🤷‍♂️ needs extensive testing

[rarkins]

@zharinov needs deconflicting

[viceice]

🤷‍♂️

[renovate-release]

🎉 This issue has been resolved in version 38.41.0 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 38.41.0

Your semantic-release bot 📦🚀

[viceice]

This causes OOM with repo cache and regex managers, so it's getting reverted.

• Multiple customManagers for the same file cause geometric growth in dependency count, leading to memory exhaustion #30906