Conversation
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR fixes an issue where SQL templated fields were not properly rendered in the Airflow UI for dynamically mapped tasks when using DbtRunLocalOperator.partial in Airflow 2. The root cause was that the _override_rtif method was deleting RenderedTaskInstanceFields records for all mapped task instances instead of filtering by the specific map_index.
Key Changes:
- Added
map_indexfilter to the delete query in_override_rtif_airflow_2_xmethod - Added comprehensive test coverage to verify the fix works correctly for dynamically mapped tasks
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| cosmos/operators/local.py | Added map_index filter to prevent deletion of other mapped task instances' records |
| tests/operators/test_local.py | Added test to verify map_index filtering in the _override_rtif method for Airflow 2 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pankajkoti
approved these changes
Nov 12, 2025
Contributor
pankajkoti
left a comment
pankajkoti
left a comment
There was a problem hiding this comment.
Nice, great fix. Thanks!
1 task
tatiana
added a commit
that referenced
this pull request
Nov 12, 2025
…#2119) Fixes SQL not rendering in Airflow UI when using `DbtRunLocalOperator.partial` in Airflow 2. The problem was that when using dynamically mapped tasks in Airflow 2, the `_override_rtif method` was not filtering by map_index when deleting old RenderedTaskInstanceFields records. This meant it could delete records for other mapped task instances, not just the current one. I added a unittest to confirm the implementation takes into account the mapped tasks index, and I also validated by running `airflow standalone` with the example DAG shared by @hkc-8010 in the original bug report. The screenshots below show the SQL being rendered in all the tasks of the Dynamic Task Group, instead of only rendering for the last task: <img width="4064" height="2334" alt="Screenshot 2025-11-12 at 16 32 55" src="https://github.com/user-attachments/assets/75ce3a91-a6b5-4c2e-b37e-6369db7d15bc" /> <img width="4064" height="2334" alt="Screenshot 2025-11-12 at 16 34 33" src="https://github.com/user-attachments/assets/04b8e17a-c9f7-45e4-b8ac-d3a5e6ddb9a1" /> <img width="4064" height="2334" alt="Screenshot 2025-11-12 at 16 34 23" src="https://github.com/user-attachments/assets/21116ec7-d211-4afc-b0c3-8fc9ed69c28a" /> Closes: #2018 (cherry picked from commit 4994474)
Merged
tatiana
added a commit
that referenced
this pull request
Nov 12, 2025
Bug fixes * Fix ``ExecutionMode.WATCHER`` deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087 * Fix ``ExecutionMode.AIRFLOW_ASYNC`` ``TaskGroup`` XCom issue by @tatiana in #2088 * Guard watcher callback exceptions to avoid hanging producer tasks by @pankajkoti in #2101 * Fix SQL templated field rendering for dynamically mapped tasks in Airflow 2 by @tatiana in #2119 Enhancements * Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073 Docs * Improve ``ExecutionMode.AIRFLOW_ASYNC`` docs by @tatiana in #2103 * Add note about experimenting threads count for the Watcher Execution mode by @pankajkoti in #2083 * Fix minor documentation formatting issue by @dnskrv in #2098 * Correct example YAML key from ``operator_args`` to ``operator_kwargs`` by @jx2lee in #2091 Others * Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076 * pre-commit autoupdate in #2078, #2104 related: astronomer/oss-integrations-private#272
Merged
tatiana
added a commit
that referenced
this pull request
Nov 13, 2025
Bug fixes * Fix ``ExecutionMode.WATCHER`` deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087 * Fix ``ExecutionMode.AIRFLOW_ASYNC`` ``TaskGroup`` XCom issue by @tatiana in #2088 * Guard watcher callback exceptions to avoid hanging producer tasks by @pankajkoti in #2101 * Fix SQL templated field rendering for dynamically mapped tasks in Airflow 2 by @tatiana in #2119 Enhancements * Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073 Docs * Improve ``ExecutionMode.AIRFLOW_ASYNC`` docs by @tatiana in #2103 * Add note about experimenting threads count for the Watcher Execution mode by @pankajkoti in #2083 * Fix minor documentation formatting issue by @dnskrv in #2098 * Correct example YAML key from ``operator_args`` to ``operator_kwargs`` by @jx2lee in #2091 Others * Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076 * pre-commit autoupdate in #2078, #2104 Closes: astronomer/oss-integrations-private#272
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes SQL not rendering in Airflow UI when using
DbtRunLocalOperator.partialin Airflow 2.The problem was that when using dynamically mapped tasks in Airflow 2, the
_override_rtif methodwas not filtering by map_index when deleting old RenderedTaskInstanceFields records. This meant it could delete records for other mapped task instances, not just the current one.I added a unittest to confirm the implementation takes into account the mapped tasks index, and I also validated by running
airflow standalonewith the example DAG shared by @hkc-8010 in the original bug report.The screenshots below show the SQL being rendered in all the tasks of the Dynamic Task Group, instead of only rendering for the last task:
Closes: #2018