Skip to content

Set retries to 0 in example DAGs#1782

Merged
pankajkoti merged 1 commit into
mainfrom
retries-0-example-dags
May 27, 2025
Merged

Set retries to 0 in example DAGs#1782
pankajkoti merged 1 commit into
mainfrom
retries-0-example-dags

Conversation

@pankajkoti
Copy link
Copy Markdown
Contributor

@pankajkoti pankajkoti commented May 27, 2025

This PR removes default retries (previously set to 2) from example DAGs to speed up CI and surface failures faster. While we can add retries to specific DAGs if needed, having them enabled by default adds unnecessary delays to our test suite.

related: https://github.com/astronomer/oss-integrations-private/issues/124
related: #1778

@netlify
Copy link
Copy Markdown

netlify Bot commented May 27, 2025

Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name Link
🔨 Latest commit c32cd58
🔍 Latest deploy log https://app.netlify.com/projects/sunny-pastelito-5ecb04/deploys/6835657a72e6a10008d5e2e0

@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.03%. Comparing base (62e9b40) to head (c32cd58).
Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1782   +/-   ##
=======================================
  Coverage   98.03%   98.03%           
=======================================
  Files          85       85           
  Lines        5247     5247           
=======================================
  Hits         5144     5144           
  Misses        103      103           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pankajkoti pankajkoti marked this pull request as ready for review May 27, 2025 07:29
Copilot AI review requested due to automatic review settings May 27, 2025 07:29
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. area:ci Related to CI, Github Actions, or other continuous integration tools labels May 27, 2025
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes the default retry behavior from example DAGs to reduce CI runtime and surface failures immediately.

  • Sets default_args={'retries': 0} across all example DAG files
  • Speeds up test feedback loops in CI

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
dev/dags/user_defined_profile.py Changed retries from 2 to 0
dev/dags/example_tasks_map.py Changed retries from 2 to 0
dev/dags/example_source_rendering.py Changed retries from 2 to 0
dev/dags/example_model_version.py Changed retries from 2 to 0
dev/dags/example_dbt_deps.py Changed retries from 2 to 0
dev/dags/example_cosmos_python_models.py Changed retries from 2 to 0
dev/dags/example_cosmos_dbt_build.py Changed retries from 2 to 0
dev/dags/dbt_docs.py Changed retries from 2 to 0
dev/dags/cosmos_seed_dag.py Changed retries from 2 to 0
dev/dags/cosmos_profile_mapping.py Changed retries from 2 to 0
dev/dags/cosmos_manifest_example.py Changed retries from 2 to 0
dev/dags/cosmos_callback_dag.py Changed retries from 2 to 0
dev/dags/basic_cosmos_task_group.py Changed retries from 2 to 0
dev/dags/basic_cosmos_dag_full_module_path_imports.py Changed retries from 2 to 0
dev/dags/basic_cosmos_dag.py Changed retries from 2 to 0
Comments suppressed due to low confidence (1)

dev/dags/user_defined_profile.py:45

  • [nitpick] Consider extracting the repeated default_args={'retries': 0} into a shared constant imported across example DAGs to reduce duplication and simplify future updates.
default_args={"retries": 0},

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label May 27, 2025
@pankajkoti pankajkoti merged commit 9519324 into main May 27, 2025
93 checks passed
@pankajkoti pankajkoti deleted the retries-0-example-dags branch May 27, 2025 09:52
@pankajkoti pankajkoti added this to the Cosmos 1.10.2 milestone Jun 13, 2025
@tatiana tatiana mentioned this pull request Aug 7, 2025
tatiana pushed a commit that referenced this pull request Aug 7, 2025
This PR removes default retries (previously set to 2) from example DAGs
to speed up CI and surface failures faster. While we can add retries to
specific DAGs if needed, having them enabled by default adds unnecessary
delays to our test suite.

related:
astronomer/oss-integrations-private#124
related: #1778
(cherry picked from commit 9519324)
tatiana added a commit that referenced this pull request Aug 8, 2025
**Bug Fixes**

* Fix task instance ``try_number`` attribute for Airflow 3 compatibility
by @pankajkoti in #1781
* Fix rendered template override logic when
``should_store_compiled_sql=False`` to restore pre-refactor behaviour by
@pankajkoti in #1777
* Fix ``ProfileConfig`` in GCP Cloud Run job execution mode by
@ramonvermeulen in #1783
* Fix dbt Docs page height by @1cadumagalhaes in #1793
* Add support to base64 encoded pem in Snowflake profiles by
@brunocmartins in #1801
* Allow to disable owner inheritance from dbt into airflow DAG owners by
@CorsettiS in #1787
* Fix Kubernetes Pod Operator conversion of ``container_resources`` to
``resources`` by @johnhoran in #1821
* Fix ``dbt deps`` with project level variables by @AlexandrKhabarov in
#1822
* Fix source freshness warnings in kubernetes execution mode by
@Pawel-Drabczyk in #1859
* Fix: Harden DbtNode against null config/meta by @pankajkoti in #1877
* Fix cache behaviour when DAG name contains "." by @tatiana in #1908

**Documentation**

* Fix ``contributing.rst`` docs by @tatiana in #1785
* Fix docs rendering in Airflow 3 Compatibility by @pankajastro in #1790
* Fix typo in ``selecting-excluding.rst`` by @msshroff in #1814
* Update testing behavior file with ``ExecutionMode.KUBERNETES`` by
@LuigiCerone in #1813
* Add step to fork repo in contributing guide by @pankajastro in #1808
* Fix ``depends_on`` attribute by @benedikt-buchert in #1837
* Fix character name by @ThePsyjo in #1860
* Update suggested MWAA startup script by @jaklan in #1884
* Make implementation & docs consistent regarding
``use_dataset_airflow3_uri_standard`` by @Anti0ff in #1878

**Others**

* Set retries to 0 in example DAGs by @pankajkoti in #1782
* Fix ``test_async_example_dag_without_setup_task`` tests by
@pankajastro in #1788
* Fix test hash value for Darwin when using Py 3.12.10 by @tatiana in
#1786
* Upgrade Python and Airflow used to run MyPy checks by @tatiana in
#1796
* Assert example DAGs' ``DagRunState`` and fix issues by @pankajkoti and
@tatiana in #1778
* Update the conflict matrix to include AF 2.10, 2.11 & 3.0 and dbt 1.9
& 1.10 by @tatiana in #1820
* Fix broken CI due to Pydantic conflicts by @tatiana in #1809
* Drop Python 3.8 Support by @pankajastro in #1852
* Add Airflow 2.11 to the test matrix by @tatiana in #1807
* Require Authorize for all jobs on pull requests from external
contributors in CI by @pankajkoti in #1861
* Leverage Trusted Publisher Management when publishing PyPI package by
@tatiana in #1862
* CI: Add back accidentally deleted python-version matrix for running
unit tests by @pankajkoti in #1872
* Remove commented code and fix mypy failures by @pankajkoti in #1876
* Add Zizmor analysis GitHub action by @pankajkoti in #1870
* Catch FlushError on Datasets for Airflow 2.11 dags test by @pankajkoti
in #1880
* Add deprecation warning for ``LoadMode.CUSTOM`` parser by
@duongphannamhung in #1885
* CI: Add GitHub CodeQL analysis workflow (codeql.yml) by @pankajkoti in
#1871
* Resolve 'credential persistence through GitHub Actions artifacts'
warnings from Zizmor by @pankajkoti in #1890
* Resolve 'overly broad permissions' warnings from Zizmor by @pankajkoti
in #1889
* Resolve Zizmor error alerts for unpinned image references; mark alert
for pull_request_target ignored by @pankajkoti in #1888
* Fix broken CI ``tests.py3.11-2.8-1.9:test-integration-setup`` by
@tatiana in #1902
* Add dbt-core 1.10 to test matrix by @tatiana in #1767
* Pin package dbt-databricks by @pankajastro in #1909
* Enable matrix test entry for dbt-1.9, python-3.9 and airflow-3.0 tests
in CI by @pankajastro in #1900
* Pre-commit updates: #1779, #1795, #1800, #1857, #1863, #1869, #1892,
#1901
* Dependabot updates: #1904

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
@pankajkoti pankajkoti mentioned this pull request Aug 8, 2025
tatiana added a commit that referenced this pull request Aug 8, 2025
**Bug Fixes**

* Fix task instance ``try_number`` attribute for Airflow 3 compatibility
by @pankajkoti in #1781
* Fix rendered template override logic when
``should_store_compiled_sql=False`` to restore pre-refactor behaviour by
@pankajkoti in #1777
* Fix ``ProfileConfig`` in GCP Cloud Run job execution mode by
@ramonvermeulen in #1783
* Fix dbt Docs page height by @1cadumagalhaes in #1793
* Add support to base64 encoded pem in Snowflake profiles by
@brunocmartins in #1801
* Allow to disable owner inheritance from dbt into airflow DAG owners by
@CorsettiS in #1787
* Fix Kubernetes Pod Operator conversion of ``container_resources`` to
``resources`` by @johnhoran in #1821
* Fix ``dbt deps`` with project level variables by @AlexandrKhabarov in
#1822
* Fix source freshness warnings in kubernetes execution mode by
@Pawel-Drabczyk in #1859
* Fix: Harden DbtNode against null config/meta by @pankajkoti in #1877
* Fix cache behaviour when DAG name contains "." by @tatiana in #1908

**Documentation**

* Fix ``contributing.rst`` docs by @tatiana in #1785
* Fix docs rendering in Airflow 3 Compatibility by @pankajastro in #1790
* Fix typo in ``selecting-excluding.rst`` by @msshroff in #1814
* Update testing behavior file with ``ExecutionMode.KUBERNETES`` by
@LuigiCerone in #1813
* Add step to fork repo in contributing guide by @pankajastro in #1808
* Fix ``depends_on`` attribute by @benedikt-buchert in #1837
* Fix character name by @ThePsyjo in #1860
* Update suggested MWAA startup script by @jaklan in #1884
* Make implementation & docs consistent regarding
``use_dataset_airflow3_uri_standard`` by @Anti0ff in #1878

**Others**

* Set retries to 0 in example DAGs by @pankajkoti in #1782
* Fix ``test_async_example_dag_without_setup_task`` tests by
@pankajastro in #1788
* Fix test hash value for Darwin when using Py 3.12.10 by @tatiana in
#1786
* Upgrade Python and Airflow used to run MyPy checks by @tatiana in
#1796
* Assert example DAGs' ``DagRunState`` and fix issues by @pankajkoti and
@tatiana in #1778
* Update the conflict matrix to include AF 2.10, 2.11 & 3.0 and dbt 1.9
& 1.10 by @tatiana in #1820
* Fix broken CI due to Pydantic conflicts by @tatiana in #1809
* Drop Python 3.8 Support by @pankajastro in #1852
* Add Airflow 2.11 to the test matrix by @tatiana in #1807
* Require Authorize for all jobs on pull requests from external
contributors in CI by @pankajkoti in #1861
* Leverage Trusted Publisher Management when publishing PyPI package by
@tatiana in #1862
* CI: Add back accidentally deleted python-version matrix for running
unit tests by @pankajkoti in #1872
* Remove commented code and fix mypy failures by @pankajkoti in #1876
* Add Zizmor analysis GitHub action by @pankajkoti in #1870
* Catch FlushError on Datasets for Airflow 2.11 dags test by @pankajkoti
in #1880
* Add deprecation warning for ``LoadMode.CUSTOM`` parser by
@duongphannamhung in #1885
* CI: Add GitHub CodeQL analysis workflow (codeql.yml) by @pankajkoti in
#1871
* Resolve 'credential persistence through GitHub Actions artifacts'
warnings from Zizmor by @pankajkoti in #1890
* Resolve 'overly broad permissions' warnings from Zizmor by @pankajkoti
in #1889
* Resolve Zizmor error alerts for unpinned image references; mark alert
for pull_request_target ignored by @pankajkoti in #1888
* Fix broken CI ``tests.py3.11-2.8-1.9:test-integration-setup`` by
@tatiana in #1902
* Add dbt-core 1.10 to test matrix by @tatiana in #1767
* Pin package dbt-databricks by @pankajastro in #1909
* Enable matrix test entry for dbt-1.9, python-3.9 and airflow-3.0 tests
in CI by @pankajastro in #1900
* Pre-commit updates: #1779, #1795, #1800, #1857, #1863, #1869, #1892,
#1901
* Dependabot updates: #1904

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ci Related to CI, Github Actions, or other continuous integration tools lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants