Release 1.11.0

[tatiana]

Features

• Introduce ExecutionMode.WATCHER to reduce DAG run time by 1/5 in several PRs. Learn more about it here. This feature was implemented via multiple PRs, including:

  • Expose new execution mode by @tatiana @pankajastro @pankajkoti in Introduce ExecutionMode.WATCHER to reduce DAG run time by 1/5 #1999
  • Add DbtProducerWatcherOperator for the proposed ExecutionMode.WATCHER by @pankajkoti in Add DbtProducerWatcherOperator for the proposed ExecutionMode.WATCHER #1982
  • Add DbtConsumerWatcherSensor for the proposed ExecutionMode.WATCHER by @pankajastro in Add DbtConsumerWatcherSensor for the proposed ExecutionMode.WATCHER #1998
  • Push producer's task completion status to XCOM by @pankajkoti in Push producer's task completion status to XCOM #2000
  • Add default priority_weight for DbtProducerWatcherOperator by @pankajkoti in Add default priority_weight for DbtProducerWatcherOperator #1995
  • Add sample dbt events for the dbt watcher execution mode by @pankajkoti in Add sample dbt events for the dbt watcher execution mode #1952
  • Add compiled_sql as a template fields on ExecutionMode.WATCHER when using run_results.json by @pankajastro in Add compiled_sql as a template fields on ExecutionMode.WATCHER when using run_results.json #2070
  • Set push_run_results_to_xcom kwargs correctly for invocation mode subprocess and Watcher mode by @pankajastro in Set push_run_results_to_xcom kwargs correctly for invocation mode subprocess and Watcher mode #2067
  • Store compiled SQL as template field for dbt callback events in ExecutionMode.WATCHER by @pankajkoti in Store compiled SQL as template field for dbt callback events in ExecutionMode.WATCHER #2068
  • Add initial documentation for ExecutionMode.WATCHER by @tatiana in Add initial documentation for ExecutionMode.WATCHER #2046
  • Support running State.UPSTREAM_FAILED tasks when WATCHER consumer upstream tasks fail by @tatiana in Support running State.UPSTREAM_FAILED tasks when WATCHER consumer upstream tasks fail #2062
  • Fail sensor tasks immediately if the ExecutionMode.WATCHER producer task fails by @pankajastro in Fail sensor tasks immediately if the ExecutionMode.WATCHER producer task fails #2040
  • Add ``WATCHER``` to GitHub issue template by @tatiana in Add WATCHER to GitHub issue template #2056
  • Add support for TestBehavior.AFTER_ALL with ExecutionMode.WATCHER by @pankajastro in Add support for TestBehavior.AFTER_ALL with ExecutionMode.WATCHER #2049
  • Add support for TestBehavior.NONE with ExecutionMode.WATCHER by @pankajastro in Add support for TestBehavior.NONE with ExecutionMode.WATCHER #2047
  • Fix ExecutionMode.WATCHER behaviour with DbtTaskGroup by @tatiana in Fix ExecutionMode.WATCHER behaviour with DbtTaskGroup #2044
  • Fix Cosmos behaviour when using watcher with InvocationMode.DBT_RUNNER by @tatiana in Fix Cosmos behaviour when using watcher with InvocationMode.DBT_RUNNER #2048

• Add Airflow 3 plugin for dbt docs with multiple dbt projects support by @pankajkoti in Add Airflow 3 plugin for dbt docs with multiple dbt projects support #2009, check the documentation.

• Initial support to dbt Fusion by @tatiana in Add experimental support to dbt Fusion #1803. More details here.

• Support to prune sources without downstream references in dbt projects by @corsettigyg in Support to prune sources without downstream references in dbt projects #1988

• Allow to set task display name as a user-defined function by @corsettigyg in Allow to set task display name as a user-defined function #1761

• Add dbt project's hash to dag docs to support dag versioning in Airflow 3 by @pankajkoti in Add dbt project's hash to dag docs to support dag versioning in Airflow 3 #1907

• feat: Add Jinja templating support for dbt_cmd_flags by @skillicinski in feat: Add Jinja templating support for dbt_cmd_flags #1899

• Add Scarf metric to collect the execution mode uses by @pankajastro in Add Scarf metric to collect the execution mode uses #1981

• Support Airflow 3.1 by @tatiana in Support Airflow 3.1 #1980

• Add MySQL profile mapping by @Lee2532 in Add MySQL profile mapping #1977

• Add sqlserver profile mapping by @pankajastro in Add sqlserver profile mapping #1737

Enhancement

• Improved the runtime speed of ExecutionMode.AIRFLOW_ASYNC by 36% , via the following tickets:
  • Use XCom to store sql when using ExecutionMode.AIRFLOW_ASYNC by @pankajastro in Use XCom to store sql when using ExecutionMode.AIRFLOW_ASYNC #1934
  • Refactor AIRFLOW_ASYNC teardown so it doesn't install the virtualenv by @pankajastro in Refactor AIRFLOW_ASYNC teardown so it doesn't install the virtualenv #1938
  • Reuse the virtual env for AIRFLOW_ASYNC setup task by @pankajastro in Reuse the virtual env for AIRFLOW_ASYNC setup task #1939
• Improve dataset/asset experience in Cosmos by @tatiana in Improve dataset/asset experience in Cosmos #2030
• Add downstreams to DbtNode by @wornjs in Add downstreams to DbtNode #2028

Bug fixes

• Fix tags extraction by @ms32035 in Fix tags extraction #1915
• Fix task flow operator args by @anyapriya in Fix task flow operator args #2024

Documentation

• Add documentation for Airflow 3 Plugin supporting dbt docs for multiple dbt projects by @pankajkoti in Add documentation for Airflow 3 Plugin supporting dbt docs for multiple dbt projects #2063
• Add Cosmos Deferrable Operator Guide by @pankajastro in [DOC]: Add Cosmos Deferrable Operator Guide #1922
• Add dbt Fusion documentation by @tatiana in Add dbt Fusion documentation #1824 Fix dbt Fusion documentation #1830
• Update dbt-fusion.rst to explicitly highlight it is in alpha by @tatiana in Update dbt-fusion.rst to explicitly highlight it is in alpha #1838
• Fix a bunch of docs build errors and warnings by @pankajkoti in Fix a bunch of docs build errors and warnings #1886
• Add docs note for param virtualenv_dir for async execution mode by @pankajastro in Add docs note for param virtualenv_dir for async execution mode #1969
• Use pepy.tech downloads badge in README by @pankajkoti in Use pepy.tech downloads badge in README #1920
• Correct the default value of cache_dir by @seokyun.ha in docs: correct the default value of cache_dir #2027

Others

• Promote @corsettigyg to committer by @tatiana in Promote @corsettigyg to committer #1985
• Add @pankajkoti and @pankajastro to contributors.rst by @tatiana in Add @pankajkoti and @pankajastro to contributors.rst #1983
• Update setup script for airflow3 script by @dwreeves in Update setup script for airflow3 script #2023
• Prevent pytest from trying to test classes that aren't actually tests by @anyapriya in Prevent pytest from trying to test classes that aren't actually tests #2032
• Fix dag.test() for Airflow 3.1+ by syncing DAG to database bby @kaxil in Fix dag.test() for Airflow 3.1+ by syncing DAG to database #2037
• Disable Scarf in CI by @pankajastro in Disable Scarf in CI #2016
• Fix failing dbt Fusion tests when run in parallel in CI by @pankajkoti in Fix failing dbt-fusion tests when run in parallel in CI #1896
• Fix MyPy issues related to ObjectStoragePath in main branch by @tatiana in Fix mypy issues related to ObjectStoragePath in main branch #2012
• Cleanup example dbt event JSON dictionaries kept for XCOM referencby @pankajkoti in Cleanup example dbt event JSON dictionaries kept for XCOM reference #1997
• Bump min hatch version that includes fixes for click>=8.3.0 by @pankajkoti in Bump min hatch version that includes fixes for click>=8.3.0 #1996
• Use official postgres image from Docker hub for kubernetes setup by @pankajkoti in Use official postgres image from Docker hub for kubernetes setup #1986
• Use click<8.3.0 for hatch as click 8.3 breaks hatch by @pankajkoti in Use click<8.3.0 for hatch as click 8.3 breaks hatch #1987
• Pin Airflow version in type check CI job by @pankajastro in Pin Airflow version in type check CI job #2003
• Improve comments after feedback on Fix running tests with dbt Fusion 2.0.0 preview versions #1948 by @tatiana in Improve comments after feedback on #1948 #1963
• Fix running tests with dbt Fusion 2.0.0 preview versions by @tatiana in Fix running tests with dbt Fusion 2.0.0 preview versions #1948
• Test hardening of dbt node having tags as unset or missing by @pankajkoti in Test hardening of dbt node having tags as unset or missing #1918
• Fix Sphinx issue in the main branch by @tatiana in Fix Sphinx issue in the main branch #2064
• pre-commit autoupdate in ⬆ [pre-commit.ci] pre-commit autoupdate #2065, ⬆ [pre-commit.ci] pre-commit autoupdate #2043, ⬆ [pre-commit.ci] pre-commit autoupdate #2033, ⬆ [pre-commit.ci] pre-commit autoupdate #2019, ⬆ [pre-commit.ci] pre-commit autoupdate #1990, ⬆ [pre-commit.ci] pre-commit autoupdate #2019, ⬆ [pre-commit.ci] pre-commit autoupdate #2008, ⬆ [pre-commit.ci] pre-commit autoupdate #1941, ⬆ [pre-commit.ci] pre-commit autoupdate #1935, ⬆ [pre-commit.ci] pre-commit autoupdate #1924
• GitHub dependabot update in Bump actions/download-artifact from 5 to 6 #2051, Bump actions/upload-artifact from 4 to 5 #2050, Bump sphinx-notes/pages from 3.4 to 3.5 #2038, Bump github/codeql-action from 3 to 4 #2022, Bump codecov/codecov-action from 5.5.0 to 5.5.1 #1947, Bump zizmorcore/zizmor-action from 0.1.2 to 0.2.0 #1955, Bump actions/stale from 9 to 10 #1946, Bump pypa/gh-action-pypi-publish from 1.12.4 to 1.13.0 #1944, Bump actions/setup-python from 5 to 6 #1945, Bump codecov/codecov-action from 5.4.3 to 5.5.0 #1928, Bump zizmorcore/zizmor-action from 0.1.1 to 0.1.2 #1921, Bump actions/checkout from 4 to 5 #1917

Co-authored-by: Pankaj Koti pankaj.koti@astronomer.io
Co-authored-by: Pankaj Singh pankaj.singh@astronomer.io

[netlify]

✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name	Link
🔨 Latest commit	7bfb83e
🔍 Latest deploy log	https://app.netlify.com/projects/sunny-pastelito-5ecb04/deploys/68e48746d5e929000840dc22

[tatiana]

I just cut 1.11.0a6 out of this branch (commit 7bfb83e)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.77%. Comparing base (65332c5) to head (1aac6c4).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2020   +/-   ##
=======================================
  Coverage   97.77%   97.77%           
=======================================
  Files          91       91           
  Lines        5848     5848           
=======================================
  Hits         5718     5718           
  Misses        130      130           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull Request Overview

This PR prepares the 1.11.0 release candidate (rc1) by updating the version number and consolidating the comprehensive changelog with all features, enhancements, bug fixes, and documentation updates that have been developed across numerous pull requests.

Key Changes:

• Version bump from 1.11.0a13 (alpha) to 1.11.0rc1 (release candidate)
• Complete changelog with detailed feature descriptions, primarily focusing on the new WATCHER execution mode
• Documentation of all bug fixes, enhancements, and other changes since version 1.10.2

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
cosmos/init.py	Updates package version from alpha (1.11.0a13) to release candidate (1.11.0rc1)
CHANGELOG.rst	Replaces placeholder alpha changelog with comprehensive release notes including features, enhancements, bug fixes, documentation updates, and other changes

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.