Handle Param Validation errors#2358
Conversation
✅ Deploy Preview for astronomer-cosmos canceled.
|
There was a problem hiding this comment.
Pull request overview
This PR hardens Cosmos DAG conversion by handling Airflow param validation failures when persisting compressed telemetry metadata into dag.params, so DAG parsing/conversion can continue even if the param write is rejected.
Changes:
- Import
ParamValidationErrorfrom Airflow exceptions. - Wrap storing
__cosmos_telemetry_metadata__indag.paramswith atry/except ParamValidationErrorand emit a warning on failure.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2358 +/- ##
=======================================
Coverage 97.90% 97.90%
=======================================
Files 102 102
Lines 6833 6840 +7
=======================================
+ Hits 6690 6697 +7
Misses 143 143 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Attempt to fix the issue that PRs are not having automatic reviewers. In practice, @astronomer/astro-cosmos-admins exists only for organisational reasons, and the people in that group have not been contributing to the project - except me (but I'm part of @astronomer/oss-integrations)
Co-authored-by: Phani Kumar <phani.kumar@astronomer.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
While trying to run some DAGs locally with Airflow 3.1.7, I observed this behaviour intermitently: <img width="825" height="755" alt="Screenshot 2026-02-13 at 09 45 06" src="https://github.com/user-attachments/assets/b5305a77-e07b-4dcd-84d3-9668f300013c" /> Although I was unable to create a reproducible sequence of steps to reproduce it, I believe we should handle this exception so it is not disruptive to end users.
**Enhancements** * Change Snowflake profile mappings to default to four threads by @tatiana in #2374 * Refactor to avoid potential future ``UnboundLocalError`` for ``producer_task`` in ``calculate_tasks_map`` by @rin in #2309 **Bug Fixes** * Fix graph selector when using + selector with ``dbt-loom`` by @award1230 in #2389 * Populate ``compiled_sql`` for ``InvocationMode.SUBPROCESS`` in ``ExecutionMode.WATCHER`` by @pankajkoti in #2319 * Preserve ``extra_context`` for watcher consumer task instances by @pankajkoti in #2381 * Fix watcher: respect ``deferrable=False`` from ``operator_args`` on consumer sensor by @pankajkoti in #2384 * Error handle invalid YAML with ``LoadMode.DBT_MANIFEST`` and ``RenderConfig.selector`` by @jonbillings in #2316 * Fix selecting model when it has the same name as folder by @pankajastro in #2328 * Handle Param Validation errors by @tatiana in #2358 * Fix cache swap issue by @jonbillings in #2332 * Fix leaked semaphore warnings in Airflow 3 by resetting dbt adapters by @pankajkoti in #2335 **Docs** * Document ``ExecutionMode.KUBERNETES`` limitations by @tatiana in #2326 **Others** * Add .airflow-registry.yaml for Airflow Provider Registry by @kaxil in #2387 * Improve test coverage for PR #2307 by @tatiana in #2308 * Address feedback from code review #2389 by @evanvolgas in #2394 Closes: astronomer/oss-integrations-private#333
Enhancements * Change Snowflake profile mappings to default to four threads by @tatiana in #2374 * Refactor to avoid potential future ``UnboundLocalError`` for ``producer_task`` in ``calculate_tasks_map`` by @rin in #2309 Bug Fixes * Fix graph selector when using + selector with ``dbt-loom`` by @award1230 in #2389 * Populate ``compiled_sql`` for ``InvocationMode.SUBPROCESS`` in ``ExecutionMode.WATCHER`` by @pankajkoti in #2319 * Preserve ``extra_context`` for watcher consumer task instances by @pankajkoti in #2381 * Fix watcher: respect ``deferrable=False`` from ``operator_args`` on consumer sensor by @pankajkoti in #2384 * Error handle invalid YAML with ``LoadMode.DBT_MANIFEST`` and ``RenderConfig.selector`` by @YourRoyalLinus in #2316 * Fix selecting model when it has the same name as folder by @pankajastro in #2328 * Handle Param Validation errors by @tatiana in #2358 * Fix cache swap issue by @YourRoyalLinus in #2332 * Fix leaked semaphore warnings in Airflow 3 by resetting dbt adapters by @pankajkoti in #2335 Docs * Document ``ExecutionMode.KUBERNETES`` limitations by @tatiana in #2326 Others * Add .airflow-registry.yaml for Airflow Provider Registry by @kaxil in #2387 * Improve test coverage for PR #2307 by @tatiana in #2308 * Address feedback from code review #2389 by @evanvolgas in #2394 Additional details on this change: - The actual release was done from branch 1.13: https://github.com/astronomer/astronomer-cosmos/releases/tag/astronomer-cosmos-v1.13.1 - This PR aims to update our CHANGELOG, and allow us to give credit to the username that was being questioned by codespell
While trying to run some DAGs locally with Airflow 3.1.7, I observed this behaviour intermitently:

Although I was unable to create a reproducible sequence of steps to reproduce it, I believe we should handle this exception so it is not disruptive to end users.