Speed up dealing with non-reachable git repos#5658
Merged
deivid-rodriguez merged 1 commit intomainfrom Sep 8, 2022
Merged
Conversation
54a1e4a to
1b68436
Compare
landongrindheim
approved these changes
Sep 8, 2022
Contributor
landongrindheim
left a comment
There was a problem hiding this comment.
I think this makes a lot of sense! I'm assuming we're okay with not picking up the (likely) small number of cases where a tag is found on retry.
pavera
reviewed
Sep 8, 2022
Contributor
Author
|
My understanding is that However, I'm not sure what the result will be if the update checker gets inconsistent results about the available remote tags 😬. |
pavera
approved these changes
Sep 8, 2022
While working on a spec failure, I observed the failing spec was also
soooooo slow:
```
Top 1 slowest examples (48.37 seconds, 100.0% of total time):
Dependabot::EndToEndJob bundler git dependencies updates dependencies correctly
48.37 seconds ./spec/dependabot/integration_spec.rb:296
```
Further investigation revealed that this spec was trying to do the
following up to 76 times, on a non-existing github repository:
* Try fetch `https://github.com/dependabot/dummy-git-dependency.git/info/refs?service=git-upload-pack`.
* Since the previous fetch failed, try run `git ls-remote https://github.com/dependabot/dummy-git-dependency.git` locally.
This is done every time the update checker needs to fetch remote git
tags, and if this fails, the checker considers that there are no remote
tags, but we'll keep on retrying the same thing and failing once and
again.
This commit changes the commit fetcher to memoize the tags the first
time they are fetched. With this change, the spec is now much faster:
```
Top 1 slowest examples (4.26 seconds, 100.0% of total time):
Dependabot::EndToEndJob bundler git dependencies updates dependencies correctly
4.26 seconds ./spec/dependabot/integration_spec.rb:296
```
I expect this to also improve things in production, of course.
1b68436 to
4257c57
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While working on a spec failure, I observed the failing spec was also soooooo slow:
Further investigation revealed that this spec was trying to do the following up to 76 times, on a non-existing github repository:
https://github.com/dependabot/dummy-git-dependency.git/info/refs?service=git-upload-pack.git ls-remote https://github.com/dependabot/dummy-git-dependency.gitlocally.This is done every time the update checker needs to fetch remote git tags, and if this fails, the checker considers that there are no remote tags, but will keep on retrying the same thing and failing once and again.
This commit changes the commit fetcher to memoize the tags the first time they are fetched. With this change, the spec is now much faster:
I expect this to also improve things in production, of course.