perf/feat: use new generator variants of list_repos in sync_repos #122
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
matt-codecov
requested review from
adrian-codecov,
RulaKhaled,
dana-yaish,
giovanni-guidini and
scott-codecov
Codecov Report
@@ Coverage Diff @@
## main #122 +/- ##
=======================================
Coverage ? 98.42%
=======================================
Files ? 342
Lines ? 26728
Branches ? 0
=======================================
Hits ? 26307
Misses ? 421
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov Report
@@ Coverage Diff @@
## main #122 +/- ##
=======================================
Coverage 98.43% 98.43%
=======================================
Files 347 348 +1
Lines 27287 27375 +88
=======================================
+ Hits 26859 26947 +88
Misses 428 428
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov Report
@@ Coverage Diff @@
## main #122 +/- ##
==========================================
+ Coverage 98.39% 98.46% +0.07%
==========================================
Files 373 374 +1
Lines 27783 27866 +83
==========================================
+ Hits 27336 27438 +102
+ Misses 447 428 -19
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 3 files with indirect coverage changes
|
This was referenced Sep 28, 2023
giovanni-guidini
approved these changes
Oct 6, 2023
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #122 +/- ##
=======================================
Coverage 98.43% 98.43%
=======================================
Files 347 348 +1
Lines 27287 27375 +88
=======================================
+ Hits 26859 26947 +88
Misses 428 428
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
updates shared to pull in codecov/shared#46 (generator versions of list_repos) and codecov/shared#41 (feature flag rollout util)
introduces a usage of the new
Featureutil to control rollout for a speed and UX improvement insync_repos. currently it is rolled out to 0% of users but codecov repos are manually opted into it.the problem:
list_repos()frontloads fetching and doing light processing on every page of repos from the git provider before returning. when syncing hundreds of repos, the UI just has to spin and spin for potentially minutes and a new user can't see any progress that repos are being ingested into the system until they all appear at the very end.the change (in
shared): @adrian-codecov suggested makinglist_repos()a generator so that it can yield a page at a time back tosync_reposandsync_reposcan incrementally feed repos into the database. if the UI refreshes its viewport or whatever, it will begin to see updated repos which probably makes for a better experience.additionally, github syncs with this feature should go faster overall. github lets us fetch the total number of repos which we can use to precompute the range of pages, and therefore the list of page URLs, that we need to fetch. knowing the list of urls in advance lets us initiate all the fetches concurrently.
Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.