Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundle analysis add to sizes to timeseries #493

Merged
merged 12 commits into from
Jun 12, 2024

Conversation

JerrySentry
Copy link
Contributor

@JerrySentry JerrySentry commented Jun 6, 2024

Insert bundle analysis related data points to the timeseries measurements table. Inserts six rows for each bundle,

  • javascript_type -> the sum of all javascript typed assets for this bundle
  • image_type -> the sum of all javascript typed assets for this bundle
  • font_type -> the sum of all javascript typed assets for this bundle
  • stylesheet_type -> the sum of all javascript typed assets for this bundle
  • asset_type -> specific js assets using its asset UUID as the identifier (ie measurable_id) link with past commits
  • bundle_type -> the sum of all assets for this bundle

This change adds a new task (bundle_analysis_save_measurements) that is called during the last step of the bundle_analysis_processor task. The new task will retrieve the bundle analysis report and insert the appropriate measurements to timeseries.

Creating timeseries dataset entries will be done in the API during handling of uploading the bundle file.

closes codecov/engineering-team#1769
closes codecov/engineering-team#1770

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Copy link

codecov bot commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 98.75000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 97.29%. Comparing base (099911a) to head (011aaa0).

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##             main     #493    +/-   ##
========================================
  Coverage   97.29%   97.29%            
========================================
  Files         445      447     +2     
  Lines       35208    35512   +304     
========================================
+ Hits        34255    34551   +296     
- Misses        953      961     +8     
Flag Coverage Δ
integration 97.26% <98.75%> (+<0.01%) ⬆️
latest-uploader-overall 97.26% <98.75%> (+<0.01%) ⬆️
unit 97.26% <98.75%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.47% <100.00%> (-0.01%) ⬇️
OutsideTasks 97.55% <98.16%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.61% <100.00%> (+0.35%) ⬆️
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/bundle_analysis_processor.py 98.30% <100.00%> (+0.05%) ⬆️
tasks/bundle_analysis_save_measurements.py 100.00% <100.00%> (ø)
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)
...nit/test_bundle_analysis_save_measurements_task.py 100.00% <100.00%> (ø)
services/tests/test_bundle_analysis.py 98.94% <97.76%> (-1.06%) ⬇️

... and 1 file with indirect coverage changes

This change has been scanned for critical changes. Learn more

@codecov-qa
Copy link

codecov-qa bot commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 98.75000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 97.26%. Comparing base (099911a) to head (011aaa0).

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@           Coverage Diff            @@
##             main     #493    +/-   ##
========================================
  Coverage   97.26%   97.26%            
========================================
  Files         414      416     +2     
  Lines       34479    34783   +304     
========================================
+ Hits        33536    33832   +296     
- Misses        943      951     +8     
Flag Coverage Δ
integration 97.26% <98.75%> (+<0.01%) ⬆️
latest-uploader-overall 97.26% <98.75%> (+<0.01%) ⬆️
unit 97.26% <98.75%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.42% <100.00%> (-0.01%) ⬇️
OutsideTasks 97.55% <98.16%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.61% <100.00%> (+0.35%) ⬆️
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/bundle_analysis_processor.py 98.30% <100.00%> (+0.05%) ⬆️
tasks/bundle_analysis_save_measurements.py 100.00% <100.00%> (ø)
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)
...nit/test_bundle_analysis_save_measurements_task.py 100.00% <100.00%> (ø)
services/tests/test_bundle_analysis.py 98.94% <97.76%> (-1.06%) ⬇️

... and 1 file with indirect coverage changes

@codecov-notifications
Copy link

codecov-notifications bot commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 98.75000% with 4 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@           Coverage Diff            @@
##             main     #493    +/-   ##
========================================
  Coverage   97.26%   97.26%            
========================================
  Files         414      416     +2     
  Lines       34479    34783   +304     
========================================
+ Hits        33536    33832   +296     
- Misses        943      951     +8     
Flag Coverage Δ
integration 97.26% <98.75%> (+<0.01%) ⬆️
latest-uploader-overall 97.26% <98.75%> (+<0.01%) ⬆️
unit 97.26% <98.75%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.42% <100.00%> (-0.01%) ⬇️
OutsideTasks 97.55% <98.16%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.61% <100.00%> (+0.35%) ⬆️
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/bundle_analysis_processor.py 98.30% <100.00%> (+0.05%) ⬆️
tasks/bundle_analysis_save_measurements.py 100.00% <100.00%> (ø)
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)
...nit/test_bundle_analysis_save_measurements_task.py 100.00% <100.00%> (ø)
services/tests/test_bundle_analysis.py 98.94% <97.76%> (-1.06%) ⬇️

... and 1 file with indirect coverage changes

Copy link

codecov-public-qa bot commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 98.75000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 97.26%. Comparing base (099911a) to head (011aaa0).

✅ All tests successful. No failed tests found ☺️

Impacted file tree graph

@@           Coverage Diff            @@
##             main     #493    +/-   ##
========================================
  Coverage   97.26%   97.26%            
========================================
  Files         414      416     +2     
  Lines       34479    34783   +304     
========================================
+ Hits        33536    33832   +296     
- Misses        943      951     +8     
Flag Coverage Δ
integration 97.26% <98.75%> (+<0.01%) ⬆️
latest-uploader-overall 97.26% <98.75%> (+<0.01%) ⬆️
unit 97.26% <98.75%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 94.42% <100.00%> (-0.01%) ⬇️
OutsideTasks 97.55% <98.16%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis.py 97.61% <100.00%> (+0.35%) ⬆️
tasks/__init__.py 100.00% <100.00%> (ø)
tasks/bundle_analysis_processor.py 98.30% <100.00%> (+0.05%) ⬆️
tasks/bundle_analysis_save_measurements.py 100.00% <100.00%> (ø)
.../tests/unit/test_bundle_analysis_processor_task.py 100.00% <100.00%> (ø)
...nit/test_bundle_analysis_save_measurements_task.py 100.00% <100.00%> (ø)
services/tests/test_bundle_analysis.py 98.94% <97.76%> (-1.06%) ⬇️

... and 1 file with indirect coverage changes

# For individual javascript associated assets using UUID
if MeasurementName.bundle_analysis_asset_size.value in dataset_names:
for asset in bundle_report.asset_reports():
if asset.asset_type == AssetType.JAVASCRIPT:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this going to be == to AssetType.JAVASCRIPT or == to AssetType.JAVASCRIPT.value? Same for the other enums

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no .value here because the asset_type is a SQLAlchemyEnum type which does the comparison without getting the actual values

commit_yaml: dict,
previous_result: Any,
):
repoid = int(repoid)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps redundant if you're expecting an int?

Copy link
Contributor

@adrian-codecov adrian-codecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, small comment left cause I'm not sure what enum value these end up with based on the tests (maybe worth adding?)

Did you also test this locally to ensure this works?

@JerrySentry
Copy link
Contributor Author

Did you also test this locally to ensure this works?

Yeah, the local timescale DB I showed you with the data was inserted with this PR.

@JerrySentry JerrySentry added this pull request to the merge queue Jun 12, 2024
Merged via the queue into main with commit 2bb8eb7 Jun 12, 2024
29 checks passed
@JerrySentry JerrySentry deleted the jun_05_ba_save_measurements branch June 12, 2024 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants