Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2#2073
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2073 +/- ##
=======================================
Coverage 97.77% 97.78%
=======================================
Files 91 91
Lines 5848 5862 +14
=======================================
+ Hits 5718 5732 +14
Misses 130 130 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds explicit Airflow version gating (>= 3.1) and resilient connection handling for the Cosmos Airflow 3 FastAPI plugin. The implementation includes runtime version checks, conditional use of the connection_env context manager (only for Airflow 3.1.0-3.1.1), and comprehensive test coverage.
Key changes:
- Enforces Airflow >= 3.1 requirement through runtime checks in plugin initialization and app creation
- Conditionally uses
connection_envcontext manager only for Airflow 3.1.0-3.1.1 (direct connection access for 3.1.2+) - Moves FastAPI app initialization from class-level to constructor to enable version validation before initialization
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
cosmos/plugin/airflow3.py |
Adds version gating logic, conditional connection handling based on Airflow version, and moves FastAPI app initialization to constructor |
tests/plugin/test_plugin_af3.py |
Adds comprehensive tests for version gating, plugin initialization, conditional connection handling, and marks existing functional tests to skip on Airflow < 3.1 |
docs/configuration/hosting-docs.rst |
Documents the Airflow >= 3.1 requirement for the plugin |
docs/configuration/cosmos-conf.rst |
Updates dbt_docs_projects description to clarify Airflow >= 3.1 requirement |
docs/airflow3_compatibility/index.rst |
Adds prominent note about Airflow >= 3.1 requirement for the FastAPI plugin |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tatiana
left a comment
There was a problem hiding this comment.
Looks excellent, @pankajkoti , thank you very much!
Minor comments inline.
92a926b to
4c2ff88
Compare
4c2ff88 to
0e5b407
Compare
tatiana
left a comment
There was a problem hiding this comment.
Looks great, @pankajkoti , thank you!
…n Airflow >= 3.1.2 (#2073) This PR adds explicit Airflow version gating and resilient connection handling for the Airflow 3 FastAPI plugin, along with tests and documentation updates. We now support the plugin only on Airflow >= 3.1 (enforced via a runtime check), move FastAPI app initialisation into the plugin’s constructor to avoid import-time failures, and conditionally use the connection_env context manager only on Airflow 3.1.0–3.1.1 (direct connection access on 3.1.2+). Tests cover the version gate, plugin initialisation, conditional connection handling, and functional routes (skipped on Airflow < 3.1). The documentation is updated to clearly state the Airflow >= 3.1 requirement and configuration keys for dbt_docs_projects. closes: #2061
Bug fixes * Fix ``ExecutionMode.WATCHER`` deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087 * Fix ``ExecutionMode.AIRFLOW_ASYNC`` ``TaskGroup`` XCom issue by @tatiana in #2088 * Guard watcher callback exceptions to avoid hanging producer tasks by @pankajkoti in #2101 * Fix SQL templated field rendering for dynamically mapped tasks in Airflow 2 by @tatiana in #2119 Enhancements * Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073 Docs * Improve ``ExecutionMode.AIRFLOW_ASYNC`` docs by @tatiana in #2103 * Add note about experimenting threads count for the Watcher Execution mode by @pankajkoti in #2083 * Fix minor documentation formatting issue by @dnskrv in #2098 * Correct example YAML key from ``operator_args`` to ``operator_kwargs`` by @jx2lee in #2091 Others * Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076 * pre-commit autoupdate in #2078, #2104 related: astronomer/oss-integrations-private#272
Bug fixes * Fix ``ExecutionMode.WATCHER`` deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087 * Fix ``ExecutionMode.AIRFLOW_ASYNC`` ``TaskGroup`` XCom issue by @tatiana in #2088 * Guard watcher callback exceptions to avoid hanging producer tasks by @pankajkoti in #2101 * Fix SQL templated field rendering for dynamically mapped tasks in Airflow 2 by @tatiana in #2119 Enhancements * Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073 Docs * Improve ``ExecutionMode.AIRFLOW_ASYNC`` docs by @tatiana in #2103 * Add note about experimenting threads count for the Watcher Execution mode by @pankajkoti in #2083 * Fix minor documentation formatting issue by @dnskrv in #2098 * Correct example YAML key from ``operator_args`` to ``operator_kwargs`` by @jx2lee in #2091 Others * Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076 * pre-commit autoupdate in #2078, #2104 Closes: astronomer/oss-integrations-private#272
This PR adds explicit Airflow version gating and resilient connection handling for the Airflow 3 FastAPI plugin, along with tests and documentation updates. We now support the plugin only on Airflow >= 3.1 (enforced via a runtime check), move FastAPI app initialisation into the plugin’s constructor to avoid import-time failures, and conditionally use the connection_env context manager only on Airflow 3.1.0–3.1.1 (direct connection access on 3.1.2+). Tests cover the version gate, plugin initialisation, conditional connection handling, and functional routes (skipped on Airflow < 3.1). The documentation is updated to clearly state the Airflow >= 3.1 requirement and configuration keys for dbt_docs_projects.
closes: #2061