Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add timeseries models and django scaffold #168

Merged
merged 12 commits into from
Dec 14, 2023

Conversation

matt-codecov
Copy link
Contributor

codecov/engineering-team#547
codecov/shared#72

this PR creates a django scaffold in worker which:

  • imports database settings from shared (which were copied from codecov-api)
  • installs pg_telemetry and ts_telemetry apps to gain access to those models
  • calls django.setup() during worker startup
  • uses no routing, asgi/wsgi, or other parts of django at all

it also introduces a helper MetricContext which wraps pg_telemetry and ts_telemetry so we can log to both until we pick one. and a little TimeseriesTimer timer utility as well. it uses TimeseriesTimer and MetricContext to log runtimes and success/failure/retry metrics in CodecovBaseTask. this gives us, for instance, per-repo success rates for upload_task, or per-owner runtimes for sync_repos_task

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

@matt-codecov matt-codecov force-pushed the matt/add-telemetry-django-models branch from 8a12a1f to 9907464 Compare November 2, 2023 06:43
Copy link

codecov bot commented Nov 2, 2023

Codecov Report

Merging #168 (fb098d8) into main (b71cc41) will decrease coverage by 0.06%.
The diff coverage is 88.88%.

❗ Current head fb098d8 differs from pull request most recent head 270ae94. Consider uploading reports for the commit 270ae94 to get more accurate results

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #168      +/-   ##
==========================================
- Coverage   98.29%   98.23%   -0.06%     
==========================================
  Files         389      386       -3     
  Lines       29620    28721     -899     
==========================================
- Hits        29114    28214     -900     
- Misses        506      507       +1     
Flag Coverage Δ
integration 98.26% <88.88%> (-0.06%) ⬇️
latest-uploader-overall 98.26% <88.88%> (-0.06%) ⬇️
unit 98.26% <88.88%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.47% <77.23%> (-0.11%) ⬇️
OutsideTasks 98.03% <86.79%> (-0.07%) ⬇️
Files Coverage Δ
helpers/tests/unit/test_telemetry.py 100.00% <100.00%> (ø)
main.py 94.28% <100.00%> (+0.25%) ⬆️
tasks/base.py Critical 98.65% <100.00%> (+0.09%) ⬆️
tasks/tests/unit/test_base.py 98.03% <100.00%> (+0.06%) ⬆️
helpers/telemetry.py 98.50% <98.50%> (ø)
manage.py 0.00% <0.00%> (ø)
django_scaffold/settings.py 0.00% <0.00%> (ø)

... and 46 files with indirect coverage changes

Related Entrypoints
run/app.tasks.compute_comparison.ComputeComparison
run/app.tasks.pulls.Sync
run/app.tasks.upload.Upload
run/app.tasks.notify.Notify
run/app.tasks.profiling.normalizer
run/app.tasks.upload.UploadProcessor
run/app.tasks.status.SetError
run/app.tasks.upload.UploadFinisher
run/app.tasks.upload.PreProcessUpload
run/app.tasks.commit_update.CommitUpdate
run/app.tasks.sync_teams.SyncTeams
run/app.cron.profiling.findinguncollected
run/app.cron.hourly_check.HourlyCheckTask
run/app.tasks.sync_repos.SyncRepos
run/app.cron.daily.GitHubAppWebhooksCheckTask
run/app.tasks.profiling.collection
run/app.tasks.static_analysis.check_suite
run/app.tasks.profiling.summarization
run/app.tasks.label_analysis.process_request
run/app.tasks.new_user_activated.NewUserActivated
run/app.tasks.ai_pr_review.AiPrReview
run/app.cron.daily.BrollyStatsRollupTask

@codecov-qa
Copy link

codecov-qa bot commented Nov 21, 2023

Codecov Report

Merging #168 (270ae94) into main (b71cc41) will decrease coverage by 0.09%.
The diff coverage is 88.13%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #168      +/-   ##
==========================================
- Coverage   98.32%   98.23%   -0.09%     
==========================================
  Files         358      362       +4     
  Lines       28923    29139     +216     
==========================================
+ Hits        28438    28626     +188     
- Misses        485      513      +28     
Flag Coverage Δ
integration ?
latest-uploader-overall 98.23% <88.13%> (-0.09%) ⬇️
unit 98.23% <88.13%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.47% <77.04%> (-0.22%) ⬇️
OutsideTasks 97.97% <85.85%> (-0.13%) ⬇️
Files Coverage Δ
helpers/tests/unit/test_telemetry.py 100.00% <100.00%> (ø)
main.py 94.28% <100.00%> (+0.25%) ⬆️
tasks/base.py 98.65% <100.00%> (+0.09%) ⬆️
tasks/tests/unit/test_base.py 98.01% <100.00%> (+0.05%) ⬆️
helpers/telemetry.py 98.48% <98.48%> (ø)
manage.py 0.00% <0.00%> (ø)
django_scaffold/settings.py 0.00% <0.00%> (ø)

Copy link
Contributor

@joseph-sentry joseph-sentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it doesn't make sense to test settings.py and manage.py, the rest LGTM

main.py Outdated Show resolved Hide resolved
@codecov-staging
Copy link

codecov-staging bot commented Nov 29, 2023

Codecov Report

Merging #168 (fb098d8) into main (f751090) will decrease coverage by 0.09%.
The diff coverage is 88.88%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #168      +/-   ##
==========================================
- Coverage   98.35%   98.26%   -0.09%     
==========================================
  Files         353      357       +4     
  Lines       27844    28076     +232     
==========================================
+ Hits        27386    27590     +204     
- Misses        458      486      +28     
Flag Coverage Δ
integration 98.26% <88.88%> (-0.09%) ⬇️
latest-uploader-overall 98.26% <88.88%> (-0.09%) ⬇️
unit 98.26% <88.88%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.59% <77.23%> (-0.23%) ⬇️
OutsideTasks 98.03% <86.79%> (-0.13%) ⬇️
Files Coverage Δ
helpers/tests/unit/test_telemetry.py 100.00% <100.00%> (ø)
main.py 94.28% <100.00%> (+0.25%) ⬆️
tasks/base.py 98.65% <100.00%> (+0.09%) ⬆️
tasks/tests/unit/test_base.py 98.02% <100.00%> (+0.06%) ⬆️
helpers/telemetry.py 98.50% <98.50%> (ø)
manage.py 0.00% <0.00%> (ø)
django_scaffold/settings.py 0.00% <0.00%> (ø)

Copy link

codecov-public-qa bot commented Dec 14, 2023

Codecov Report

Merging #168 (270ae94) into main (b71cc41) will decrease coverage by 0.09%.
Report is 1 commits behind head on main.
The diff coverage is 88.13%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #168      +/-   ##
==========================================
- Coverage   98.32%   98.23%   -0.09%     
==========================================
  Files         358      362       +4     
  Lines       28923    29139     +216     
==========================================
+ Hits        28438    28626     +188     
- Misses        485      513      +28     
Flag Coverage Δ
integration 98.23% <88.13%> (-0.09%) ⬇️
latest-uploader-overall 98.23% <88.13%> (-0.09%) ⬇️
unit 98.23% <88.13%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.47% <77.04%> (-0.22%) ⬇️
OutsideTasks 97.97% <85.85%> (-0.13%) ⬇️
Files Coverage Δ
helpers/tests/unit/test_telemetry.py 100.00% <100.00%> (ø)
main.py 94.28% <100.00%> (+0.25%) ⬆️
tasks/base.py 98.65% <100.00%> (+0.09%) ⬆️
tasks/tests/unit/test_base.py 98.01% <100.00%> (+0.05%) ⬆️
helpers/telemetry.py 98.48% <98.48%> (ø)
manage.py 0.00% <0.00%> (ø)
django_scaffold/settings.py 0.00% <0.00%> (ø)

@matt-codecov matt-codecov merged commit ed06839 into main Dec 14, 2023
12 of 19 checks passed
@matt-codecov matt-codecov deleted the matt/add-telemetry-django-models branch December 14, 2023 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants