Complete the new TA pipeline implementations #1033

joseph-sentry · 2025-01-24T22:06:26Z

creates the new cache analytics task
creates the new ta process flakes task
TA finisher uses new TADriver interface and ta_utils
TA finisher queues up new tasks

- generalize the way bigquery accepts query parameters - change type of params arg in bigquery_service to sequence - feat: add upload_id field to ta_testrun protobuf - add flags_hash field to ta_testrun protobuf - create new testid generation function - add test_id to ta_testrun proto - add flaky_failure to testrun protobuf - handle flaky failures in ta_storage.bq - create sql queries for reading from bq - write tests for ta_storage.bq aggregate queries

Improve TADriver interface - add some more methods to the TADriver - implement a base constructor - modify the write_testruns interface - implement all methods in BQ and PG - improve BQ and PG tests - modify use of TADriver interface in processor and finishers - update django settings to include new settings - TODO: modify requirements to suitable shared version - create ta_utils to replace test_results in the future - the reason for this is that we want a slightly different implementation of the test results notifier for the new TA pipeline

codecov · 2025-01-24T22:10:31Z

❌ 12 Tests Failed:

Tests completed	Failed	Passed	Skipped
12	12	0	0

View the top 3 failed tests by shortest run time

tasks/tests/integration/test_ghm_sync_plans.py::tasks.tests.integration.test_ghm_sync_plans

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_ghm_sync_plans.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_http_request_task.py::tasks.tests.integration.test_http_request_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_http_request_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_send_email_task.py::tasks.tests.integration.test_send_email_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_send_email_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

codecov-staging · 2025-01-24T22:10:41Z

❌ 12 Tests Failed:

Tests completed	Failed	Passed	Skipped
12	12	0	0

View the top 3 failed tests by shortest run time

tasks/tests/integration/test_http_request_task.py::tasks.tests.integration.test_http_request_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_http_request_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_notify_task.py::tasks.tests.integration.test_notify_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_notify_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_status_set_pending_task.py::tasks.tests.integration.test_status_set_pending_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_status_set_pending_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

codecov-qa · 2025-01-24T22:10:50Z

❌ 12 Tests Failed:

Tests completed	Failed	Passed	Skipped
12	12	0	0

View the top 3 failed tests by shortest run time

tasks/tests/integration/test_ghm_sync_plans.py::tasks.tests.integration.test_ghm_sync_plans

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_ghm_sync_plans.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_status_set_pending_task.py::tasks.tests.integration.test_status_set_pending_task

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_status_set_pending_task.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

tasks/tests/integration/test_timeseries_backfill.py::tasks.tests.integration.test_timeseries_backfill

Stack Traces | 0s run time

ImportError while importing test module '.../tests/integration/test_timeseries_backfill.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../local/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/__init__.py:52: in <module>
    from tasks.ta_finisher import ta_finisher_task
tasks/ta_finisher.py:47: in <module>
    from tasks.ta_cache_analytics import ta_cache_analytics_task
tasks/ta_cache_analytics.py:5: in <module>
    from shared.django_apps.test_analytics.models import LastRollupDate
E   ImportError: cannot import name 'LastRollupDate' from 'shared.django_apps.test_analytics.models' (.../local/lib/python3.13.../django_apps/test_analytics/models.py)

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

codecov-public-qa · 2025-01-24T22:11:00Z

❌ 12 Tests Failed:

Tests completed	Failed	Passed	Skipped
12	12	0	0

View the top 3 failed tests by shortest run time

tasks/tests/integration/test_ai_pr_review.py::::tasks.tests.integration.test_ai_pr_review

Stack Traces | 0s run time

No failure message available

tasks/tests/integration/test_sync_pull.py::::tasks.tests.integration.test_sync_pull

Stack Traces | 0s run time

No failure message available

tasks/tests/integration/test_upload_e2e.py::::tasks.tests.integration.test_upload_e2e

Stack Traces | 0s run time

No failure message available

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

github-actions · 2025-01-24T22:11:12Z

❌ 24 Tests Failed:

Tests completed	Failed	Passed	Skipped
24	24	0	0

View the top 3 failed tests by shortest run time

services.tests.test_repository_service

Stack Traces | 0.000s run time

No failure message available

tasks.tests.integration.test_ai_pr_review

Stack Traces | 0.000s run time

No failure message available

tasks.tests.integration.test_ghm_sync_plans

Stack Traces | 0.000s run time

No failure message available

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

Swatinem · 2025-01-28T10:53:20Z

tasks/ta_finisher.py

-            Flake.count != (Flake.recent_passes_count + Flake.fail_count),
+
+def get_postgres_test_data(
+    db_session: Session, repo: Repository, commit_sha: str, commit_yaml: UserYaml


the repo and commit_yaml are only used to initialize the driver, and for the should_do_flaky_detection check.
instead, you could pass the driver and that flag from the outside. That way you only need a single function instead of two.

Swatinem · 2025-01-28T10:55:39Z

tasks/ta_process_flakes.py

+    # get all uploads pending process flakes in the entire repo? why stop at a given commit :D
+    uploads_to_process = ReportSession.objects.filter(
+        report__report_type=CommitReport.ReportType.TEST_RESULTS.value,
+        report__commit__repository__repoid=repo_id,


Suggested change

report__commit__repository__repoid=repo_id,

report__commit__repository=repo_id,

Not sure if the ORM is smart enough to do this automatically, but this could potentially avoid a join on repository.

Swatinem · 2025-01-28T10:57:20Z

tasks/ta_process_flakes.py

+
+    if settings.BIGQUERY_WRITE_ENABLED:
+        bq = BQDriver(repo_id)
+        bq.write_flakes([upload for upload in uploads_to_process])


Suggested change

bq.write_flakes([upload for upload in uploads_to_process])

bq.write_flakes(list(uploads_to_process))

joseph-sentry added 9 commits January 24, 2025 16:11

fix: address feedback

45f6463

fix lint

95f393a

address feedback i missed

30b426f

create new cache analytics task

c40a3ae

create new process flakes task

4f1273f

consume new TADriver and flake, cache tasks in finisher

6130eca

quick fix in ta processor

7bcdb96

Swatinem reviewed Jan 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Complete the new TA pipeline implementations #1033

Complete the new TA pipeline implementations #1033

joseph-sentry commented Jan 24, 2025

codecov bot commented Jan 24, 2025

codecov-staging bot commented Jan 24, 2025

codecov-qa bot commented Jan 24, 2025

codecov-public-qa bot commented Jan 24, 2025

github-actions bot commented Jan 24, 2025

Swatinem Jan 28, 2025

Swatinem Jan 28, 2025

Swatinem Jan 28, 2025

	report__commit__repository__repoid=repo_id,
	report__commit__repository=repo_id,

	bq.write_flakes([upload for upload in uploads_to_process])
	bq.write_flakes(list(uploads_to_process))

Complete the new TA pipeline implementations #1033

Are you sure you want to change the base?

Complete the new TA pipeline implementations #1033

Conversation

joseph-sentry commented Jan 24, 2025

codecov bot commented Jan 24, 2025

❌ 12 Tests Failed:

codecov-staging bot commented Jan 24, 2025

❌ 12 Tests Failed:

codecov-qa bot commented Jan 24, 2025

❌ 12 Tests Failed:

codecov-public-qa bot commented Jan 24, 2025

❌ 12 Tests Failed:

github-actions bot commented Jan 24, 2025

❌ 24 Tests Failed:

Swatinem Jan 28, 2025

Choose a reason for hiding this comment

Swatinem Jan 28, 2025

Choose a reason for hiding this comment

Swatinem Jan 28, 2025

Choose a reason for hiding this comment