From a7e26f525b240b9b74239a91c0522bb948444c86 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:39:47 +0000 Subject: [PATCH 1/6] Improve the watcher documentation on dbt threads Highlight in the docs that the watcher performance improvement is dependent upon the number of dbt threads. --- docs/getting_started/watcher-execution-mode.rst | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index fb00522f32..6e587084c8 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -5,7 +5,10 @@ Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Executi With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging ``dbt threading ``_, the performance gains will depend on two major factors: + +- The amount of ``threads`` set either via the dbt profile configuration or as a flag +- The topology of the dbt pipeline ------------------------------------------------------------------------------- @@ -407,6 +410,14 @@ We have a follow up ticket to `further investigate this use-case ``_ for more information on how to set the number of threads. + + + Summary ------- From 01f222ae992c55119c25dc2ead3f91159f364d38 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:40:42 +0000 Subject: [PATCH 2/6] Apply suggestion from @tatiana --- docs/getting_started/watcher-execution-mode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index 6e587084c8..33655fa3e2 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -5,7 +5,7 @@ Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Executi With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging ``dbt threading ``_, the performance gains will depend on two major factors: +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_, the performance gains will depend on two major factors: - The amount of ``threads`` set either via the dbt profile configuration or as a flag - The topology of the dbt pipeline From dc1960be61553da6a5cf5450e553b6f23b91c1a5 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:41:03 +0000 Subject: [PATCH 3/6] Apply suggestion from @tatiana --- docs/getting_started/watcher-execution-mode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index 33655fa3e2..d9ec1e8f32 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -414,7 +414,7 @@ Troubleshooting --------------- Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." -Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. ``Check the dbt docs ``_ for more information on how to set the number of threads. +Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. =`Check the dbt docs `_ for more information on how to set the number of threads. From b317609592ebba4cc95b8fb3f52dc0c19b281721 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:41:41 +0000 Subject: [PATCH 4/6] Apply suggestion from @tatiana --- docs/getting_started/watcher-execution-mode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index d9ec1e8f32..1b6d160159 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -7,7 +7,7 @@ With the release of **Cosmos 1.11.0**, we are introducing a powerful new experim Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_, the performance gains will depend on two major factors: -- The amount of ``threads`` set either via the dbt profile configuration or as a flag +- The amount of ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag - The topology of the dbt pipeline ------------------------------------------------------------------------------- From 1fa6648359b39e45a7619047f887ff76ac176f3d Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:41:53 +0000 Subject: [PATCH 5/6] Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/getting_started/watcher-execution-mode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index 1b6d160159..ec254ebb97 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -5,7 +5,7 @@ Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Executi With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_, the performance gains will depend on two major factors: +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging ``dbt threading ``_, the performance gains will depend on two major factors: - The amount of ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag - The topology of the dbt pipeline From 7da9ba953b258b448b842ce4472f2826c8668ba5 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Mon, 19 Jan 2026 15:43:19 +0000 Subject: [PATCH 6/6] Apply suggestions from code review --- docs/getting_started/watcher-execution-mode.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst index ec254ebb97..6623504d46 100644 --- a/docs/getting_started/watcher-execution-mode.rst +++ b/docs/getting_started/watcher-execution-mode.rst @@ -5,7 +5,7 @@ Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Executi With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging ``dbt threading ``_, the performance gains will depend on two major factors: +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_, the performance gains will depend on two major factors: - The amount of ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag - The topology of the dbt pipeline @@ -414,7 +414,7 @@ Troubleshooting --------------- Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." -Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. =`Check the dbt docs `_ for more information on how to set the number of threads. +Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. `Check the dbt docs `_ for more information on how to set the number of threads.