fix: use original PR base to compute patch coverage #199
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
matt-codecov
requested review from
adrian-codecov,
giovanni-guidini and
scott-codecov
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #199 +/- ##
==========================================
- Coverage 98.35% 98.34% -0.01%
==========================================
Files 353 354 +1
Lines 27844 27934 +90
==========================================
+ Hits 27386 27473 +87
- Misses 458 461 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov Report
Changes have been made to critical files, which contain lines commonly executed in production. Learn more Additional details and impacted files@@ Coverage Diff @@
## main #199 +/- ##
==========================================
- Coverage 98.31% 98.31% -0.01%
==========================================
Files 382 383 +1
Lines 28489 28579 +90
==========================================
+ Hits 28010 28097 +87
- Misses 479 482 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
|
giovanni-guidini
approved these changes
Dec 4, 2023
| # To see how a patch changes project coverage, we compare the branch head's | ||
| # report against the base's report, or if the base isn't in our database, | ||
| # the next-oldest commit that is. Be aware that this base commit may not be | ||
| # the true base that, for example, a PR is based on. | ||
| base: FullCommit |
There was a problem hiding this comment.
I wonder if using project_coverage_base and patch_coverage_base would be better names from an understanding perspective.
Although the comments do a good job of explaining the difference.
There was a problem hiding this comment.
oh you're right those are much better names. i tried renaming base to adjusted_base and it was a very messy rename so i backed out since merging this was a priority, but i assigned myself a GH issue to rename the fields with your suggested names when i get a spare minute #203
scott-codecov
added a commit
that referenced
this pull request
Dec 6, 2023
* main: Only trigger AI PR review if pull is open Add log line when triggering AI PR review task fix: use original PR base to compute patch coverage (#199) Prepare release 23.12.4 Update workflows chore(deps): Update codecov-shared dependency (#194) Prep the terrain for reports with label compression. (#188) allow staging deploy when pushing to staging (#192)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
codecov/engineering-team#814
this code could use some cleanup one day but this is a minimal change to remove unnecessary inaccuracy from patch coverage
currently for both patch coverage and project coverage deltas we compare the PR head's report against the base, or if the base isn't in our database, the next-oldest commit (the "adjusted base"). that's necessary for project coverage, it's unnecessary (and problematic) for patch coverage
this PR:
original_base_commitidin theComparisonobjectuse_original_baseswitch toComparisonProxy.get_diff()use_original_base=Trueto calls toget_diff()in the notification serviceservices/notification/notifiers/mixins/status.py: context is explicitly patch-coverage-relatedservices/notification/notifiers/checks/patch.py: explicitly a patch-coverage checkservices/notification/notifiers/mixins/message/__init__.py: the diff computed here is plumbed through all of the various comment section implementations. afaict the only usages of it are patch-coverage-related, so changing the top of the funnel shouldn't impact anything elseextra notes
tasks/save_report_results.pyis dead codetasks/sync_pull.pycomputes patch coverage against the correct base and saves it topull.diffin the database! but then afaict nothing ever reads from it (link)comparison.enriched_pull.database_pull.diffand not have to recompute it every time, but thesync_pulltask runs after thenotifytask and changing that is Too Much for right nowpull.difffor patch coverage instead of recomputing it all over engineering-team#884 documents this opportunity for improvementtasks/upload_processor.pycomputes patch coverage for just the current commit (not compared to the PR base) but doesn't do anything with itReportmodel will use it to updateReportFileSummary.diff_totalsfor each file in the report, and that gets saved innetwork_file_json()or whatever it's called and then rehydrated. but after a quick scan i couldn't find anything that tries to read this fieldtasks/compute_comparison.pyalonepull.baseis the real base andpull.compared_tois the adjusted base, but everywhere else they usebaseto refer to the adjusted base so i had to call the real baseoriginal_baseLegal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.