Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notify user of errors with uploading in comment #570

Merged
merged 7 commits into from
Jul 31, 2024

Conversation

joseph-sentry
Copy link
Contributor

The BaseNotifier class will be used by the TestResultsNotifier and the
ErrorCommentNotifier. It provides a basic structure and utility for
sending the comment to Github.

The build_message method is not implemented and actual concrete
notifier classes are meant to extend this class and implement the
build_message method to generate the contents of the comment that will
be sent to Github when notify() is called.
This commit changes the TestResultsNotifier to extend the BaseNotifier
class from helpers. A change that was made as a side effect is that
the BaseNotifier does not accept any arguments in the build_message or
notify methods so the class constructor was extended to accept the
payload that is used in the build_message method.
This commit changes the functionality of the upload finisher to call
the notify task only when all uploads are successful, and when
failing uploads are detected to call the NotifyError task instead that
will make a comment on the relevant PR notifying the user that
upload processing failed.
@codecov-notifications
Copy link

codecov-notifications bot commented Jul 22, 2024

Codecov Report

Attention: Patch coverage is 99.24528% with 2 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files Patch % Lines
helpers/notifier.py 97.67% 1 Missing ⚠️
services/test_results.py 92.85% 1 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #570      +/-   ##
==========================================
+ Coverage   97.49%   97.52%   +0.03%     
==========================================
  Files         414      417       +3     
  Lines       34982    35152     +170     
==========================================
+ Hits        34105    34282     +177     
+ Misses        877      870       -7     
Flag Coverage Δ
integration 97.52% <99.24%> (+0.03%) ⬆️
latest-uploader-overall 97.52% <99.24%> (+0.03%) ⬆️
unit 97.52% <99.24%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.62% <98.76%> (+0.08%) ⬆️
OutsideTasks 97.74% <97.75%> (+0.03%) ⬆️
Files Coverage Δ
celery_config.py 70.51% <100.00%> (+0.38%) ⬆️
helpers/checkpoint_logger/flows.py 100.00% <100.00%> (ø)
services/tests/test_test_results.py 100.00% <100.00%> (ø)
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/notify_error.py 100.00% <100.00%> (ø)
tasks/test_results_finisher.py 97.14% <100.00%> (+0.15%) ⬆️
tasks/tests/unit/test_notify_error_task.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_test_results_finisher.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_upload_finisher_task.py 100.00% <100.00%> (ø)
tasks/upload_finisher.py 72.77% <100.00%> (+2.18%) ⬆️
... and 2 more

... and 7 files with indirect coverage changes

@codecov-qa
Copy link

codecov-qa bot commented Jul 22, 2024

Codecov Report

Attention: Patch coverage is 99.24528% with 2 lines in your changes missing coverage. Please review.

Project coverage is 97.52%. Comparing base (555a769) to head (a73ff80).
Report is 5 commits behind head on main.

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #570      +/-   ##
==========================================
+ Coverage   97.49%   97.52%   +0.03%     
==========================================
  Files         414      417       +3     
  Lines       34982    35152     +170     
==========================================
+ Hits        34105    34282     +177     
+ Misses        877      870       -7     
Flag Coverage Δ
integration 97.52% <99.24%> (+0.03%) ⬆️
latest-uploader-overall 97.52% <99.24%> (+0.03%) ⬆️
unit 97.52% <99.24%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.62% <98.76%> (+0.08%) ⬆️
OutsideTasks 97.74% <97.75%> (+0.03%) ⬆️
Files Coverage Δ
celery_config.py 70.51% <100.00%> (+0.38%) ⬆️
helpers/checkpoint_logger/flows.py 100.00% <100.00%> (ø)
services/tests/test_test_results.py 100.00% <100.00%> (ø)
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/notify_error.py 100.00% <100.00%> (ø)
tasks/test_results_finisher.py 97.14% <100.00%> (+0.15%) ⬆️
tasks/tests/unit/test_notify_error_task.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_test_results_finisher.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_upload_finisher_task.py 100.00% <100.00%> (ø)
tasks/upload_finisher.py 72.77% <100.00%> (+2.18%) ⬆️
... and 2 more

... and 7 files with indirect coverage changes

Copy link

codecov-public-qa bot commented Jul 22, 2024

Codecov Report

Attention: Patch coverage is 99.24528% with 2 lines in your changes missing coverage. Please review.

Project coverage is 97.52%. Comparing base (555a769) to head (a73ff80).
Report is 5 commits behind head on main.

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #570      +/-   ##
==========================================
+ Coverage   97.49%   97.52%   +0.03%     
==========================================
  Files         414      417       +3     
  Lines       34982    35152     +170     
==========================================
+ Hits        34105    34282     +177     
+ Misses        877      870       -7     
Flag Coverage Δ
integration 97.52% <99.24%> (+0.03%) ⬆️
latest-uploader-overall 97.52% <99.24%> (+0.03%) ⬆️
unit 97.52% <99.24%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.62% <98.76%> (+0.08%) ⬆️
OutsideTasks 97.74% <97.75%> (+0.03%) ⬆️
Files Coverage Δ
celery_config.py 70.51% <100.00%> (+0.38%) ⬆️
helpers/checkpoint_logger/flows.py 100.00% <100.00%> (ø)
services/tests/test_test_results.py 100.00% <100.00%> (ø)
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/notify_error.py 100.00% <100.00%> (ø)
tasks/test_results_finisher.py 97.14% <100.00%> (+0.15%) ⬆️
tasks/tests/unit/test_notify_error_task.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_test_results_finisher.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_upload_finisher_task.py 100.00% <100.00%> (ø)
tasks/upload_finisher.py 72.77% <100.00%> (+2.18%) ⬆️
... and 2 more

... and 7 files with indirect coverage changes

Copy link

codecov bot commented Jul 22, 2024

Codecov Report

Attention: Patch coverage is 99.24528% with 2 lines in your changes missing coverage. Please review.

Project coverage is 97.61%. Comparing base (555a769) to head (a73ff80).
Report is 5 commits behind head on main.

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

✅ All tests successful. No failed tests found.

Files Patch % Lines
helpers/notifier.py 97.67% 1 Missing ⚠️
services/test_results.py 92.85% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #570      +/-   ##
==========================================
+ Coverage   97.54%   97.61%   +0.06%     
==========================================
  Files         449      452       +3     
  Lines       36188    37160     +972     
==========================================
+ Hits        35299    36272     +973     
+ Misses        889      888       -1     
Flag Coverage Δ
integration 97.52% <99.24%> (+0.03%) ⬆️
latest-uploader-overall 97.52% <99.24%> (+0.03%) ⬆️
unit 97.52% <99.24%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.69% <98.76%> (+0.05%) ⬆️
OutsideTasks 97.74% <97.75%> (+0.03%) ⬆️
Files Coverage Δ
celery_config.py 70.51% <100.00%> (+0.38%) ⬆️
helpers/checkpoint_logger/flows.py 100.00% <100.00%> (ø)
services/tests/test_test_results.py 100.00% <100.00%> (ø)
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/notify_error.py 100.00% <100.00%> (ø)
tasks/test_results_finisher.py 97.85% <100.00%> (+0.11%) ⬆️
tasks/tests/unit/test_notify_error_task.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_test_results_finisher.py 100.00% <100.00%> (ø)
tasks/tests/unit/test_upload_finisher_task.py 100.00% <100.00%> (ø)
tasks/upload_finisher.py 72.90% <100.00%> (+2.16%) ⬆️
... and 2 more

... and 7 files with indirect coverage changes

Related Entrypoints
run/app.tasks.upload.UploadFinisher
run/app.tasks.test_results.TestResultsFinisherTask

@joseph-sentry joseph-sentry marked this pull request as ready for review July 25, 2024 18:28
@joseph-sentry joseph-sentry requested a review from a team July 25, 2024 18:28
Copy link
Contributor

@nora-codecov nora-codecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice job with BaseNotifier - helpful to have cleanups like this. Overall looks good but should be thoroughly tested in Staging since you're updating a handful of pieces - wanted to check that you did that testing 👍


return (True, "comment_posted")

def insert_breaks(self, table_value):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this in the code but not being used? 💀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes 😅

def build_message(
self,
) -> str:
error_message = f"❗️ We couldn't process [{self.failed_upload}] out of [{self.total_upload}] uploads. Codecov cannot generate a coverage report with partially processed data. Please review the upload errors on the commit page."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this emoji reference work? When I changed notifiers with emojis before I had to use something like :tada: (see _create_welcome_message)

)

# get all upload errors for this commit
#
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove unused line

)

commit: Commit = commits_query.first()
assert commit, "Commit not found in database."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain this line? are you making sure that the commit does not exist? why does the .first() function return as that message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's the opposite, we're making sure that commit exists, and if it doesn't we throw the assertion error with that message

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aaahhhh, i was reading it as though the assert was asserting that element 1 and element 2 were equal, rather than that element 1 is truthy else throw element 2

num_failed_upload=num_failed_upload,
num_total_upload=num_total_upload,
),
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this throw any errors if any of the extras are null or not found in the transformations above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if they're null it'll display null, but if they are undefined it will throw an error, but i don't think it's possible for them to be undefined (or else my ide would probably complain)

Copy link
Contributor

@nora-codecov nora-codecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@joseph-sentry joseph-sentry added this pull request to the merge queue Jul 31, 2024
Merged via the queue into main with commit e8dd77d Jul 31, 2024
29 of 30 checks passed
@joseph-sentry joseph-sentry deleted the joseph/notify-error branch July 31, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants