Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: separate lookup from filter+decide stages #6575

Open
rarkins opened this issue Jun 24, 2020 · 1 comment
Open

Refactor: separate lookup from filter+decide stages #6575

rarkins opened this issue Jun 24, 2020 · 1 comment
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started type:refactor Refactoring or improving of existing code

Comments

@rarkins
Copy link
Collaborator

rarkins commented Jun 24, 2020

Currently Renovate does lookup+filter+decide logic per-dependency. Instead, I'd prefer to:

  • Flatten extracted dependencies into a single-dimensioned array
  • Look up (via datasources) all dependencies simultaneously (but with rate limits)
  • Once lookups are complete, perform the filter+decide step

Separating like this makes more logical sense, and will also make it easier in future in case we want to optimize such as:

  • Caching the entire lookup result and re-using it if running multiple times in a row, or
  • Querying a lookup cache with a single query (list of all packages) instead of one cache query per package
@rarkins rarkins added priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others type:refactor Refactoring or improving of existing code labels Jun 24, 2020
@rarkins rarkins self-assigned this Jun 24, 2020
@rarkins rarkins assigned zharinov and unassigned rarkins Aug 12, 2020
@rarkins
Copy link
Collaborator Author

rarkins commented Aug 12, 2020

Waiting on #6358 - we should have some default rate limit like 10 before we fully parallelise lookups.

@rarkins rarkins added status:blocked Issue is blocked by another issue or external requirement status:in-progress Someone is working on implementation and removed status:in-progress Someone is working on implementation labels Jan 12, 2021
@rarkins rarkins added status:ready and removed status:blocked Issue is blocked by another issue or external requirement labels Apr 21, 2023
@rarkins rarkins added the status:requirements Full requirements are not yet known, so implementation should not be started label Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started type:refactor Refactoring or improving of existing code
Projects
None yet
Development

No branches or pull requests

2 participants