From 95907bbd01dcd37a36a8416c99aaf42758321889 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:52:27 -0500 Subject: [PATCH 01/48] Create config subfolder structure --- .../{ => callbacks}/callbacks.rst | 0 .../testing-behavior.rst | 0 .../{ => dbt-docs}/generating-docs.rst | 0 .../{ => dbt-docs}/hosting-docs.rst | 0 .../execution-modes-local-conflicts.rst | 133 +++++ docs/configuration/index.rst | 89 +++- .../airflow-worker/async-execution-mode.rst | 247 +++++++++ .../run-dbt/airflow-worker/index.rst | 9 + .../airflow-worker/watcher-execution-mode.rst | 480 ++++++++++++++++++ .../container/aws-container-run-job.rst | 191 +++++++ .../container/azure-container-instance.rst | 138 +++++ .../run-dbt/container/docker.rst | 111 ++++ .../run-dbt/container/gcp-cloud-run-job.rst | 265 ++++++++++ .../configuration/run-dbt/container/index.rst | 13 + .../run-dbt/container/kubernetes.rst | 167 ++++++ .../watcher-kubernetes-execution-mode.rst | 214 ++++++++ .../configuration/run-dbt/execution-modes.rst | 387 ++++++++++++++ 17 files changed, 2428 insertions(+), 16 deletions(-) rename docs/configuration/{ => callbacks}/callbacks.rst (100%) rename docs/configuration/{ => configure-tests}/testing-behavior.rst (100%) rename docs/configuration/{ => dbt-docs}/generating-docs.rst (100%) rename docs/configuration/{ => dbt-docs}/hosting-docs.rst (100%) create mode 100644 docs/configuration/execution-modes-local-conflicts.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/index.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst create mode 100644 docs/configuration/run-dbt/container/aws-container-run-job.rst create mode 100644 docs/configuration/run-dbt/container/azure-container-instance.rst create mode 100644 docs/configuration/run-dbt/container/docker.rst create mode 100644 docs/configuration/run-dbt/container/gcp-cloud-run-job.rst create mode 100644 docs/configuration/run-dbt/container/index.rst create mode 100644 docs/configuration/run-dbt/container/kubernetes.rst create mode 100644 docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst create mode 100644 docs/configuration/run-dbt/execution-modes.rst diff --git a/docs/configuration/callbacks.rst b/docs/configuration/callbacks/callbacks.rst similarity index 100% rename from docs/configuration/callbacks.rst rename to docs/configuration/callbacks/callbacks.rst diff --git a/docs/configuration/testing-behavior.rst b/docs/configuration/configure-tests/testing-behavior.rst similarity index 100% rename from docs/configuration/testing-behavior.rst rename to docs/configuration/configure-tests/testing-behavior.rst diff --git a/docs/configuration/generating-docs.rst b/docs/configuration/dbt-docs/generating-docs.rst similarity index 100% rename from docs/configuration/generating-docs.rst rename to docs/configuration/dbt-docs/generating-docs.rst diff --git a/docs/configuration/hosting-docs.rst b/docs/configuration/dbt-docs/hosting-docs.rst similarity index 100% rename from docs/configuration/hosting-docs.rst rename to docs/configuration/dbt-docs/hosting-docs.rst diff --git a/docs/configuration/execution-modes-local-conflicts.rst b/docs/configuration/execution-modes-local-conflicts.rst new file mode 100644 index 0000000000..9fec173751 --- /dev/null +++ b/docs/configuration/execution-modes-local-conflicts.rst @@ -0,0 +1,133 @@ +:orphan: + +.. _execution-modes-local-conflicts: + +Airflow and dbt dependencies conflicts +====================================== + +When using the `Local Execution Mode `__, users may face dependency conflicts between +`Apache Airflow® `_ and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used. + +If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. +With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate +Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and +`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. + +The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. + +In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): + ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| Airflow / DBT | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | ++===============+=====+=====+=====+=====+=====+=====+=====+=====+=====+=====+======+ +| 2.2 | | | | x | x | x | x | x | x | x | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.3 | x | x | | x | x | x | x | x | x | x | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.4 | x | x | x | | | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.5 | x | x | x | | | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.6 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.7 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.8 | x | x | x | x | x | | x | | | | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.9 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.10 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.11 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 3.0 | x | x | x | x | x | x | x | x | | | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ + +Examples of errors +----------------------------------- + +.. code-block:: bash + + The conflict is caused by: + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10 + + +.. code-block:: bash + + ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies. + The conflict is caused by: + apache-airflow 2.2.4 depends on jinja2<3.1 and >=2.10.1 + dbt-core 1.5.0 depends on Jinja2==3.1.2 + +.. code-block:: bash + + ERROR: Cannot install apache-airflow==2.6.0 and dbt-core because these package versions have conflicting dependencies. + The conflict is caused by: + apache-airflow 2.6.0 depends on importlib-metadata<5.0.0 and >=1.7; python_version < "3.9" + dbt-semantic-interfaces 0.1.0.dev7 depends on importlib-metadata==6.6.0 + +.. code-block:: bash + + ERROR: Cannot install apache-airflow, apache-airflow==2.7.0 and dbt-core==1.4.0 because these package versions have conflicting dependencies. + + The conflict is caused by: + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.12.0 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.2 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.1 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.0 depends on PyYAML<6 and >=5.1 + apache-airflow 2.7.0 depends on jsonschema>=4.18.0 + flask-appbuilder 4.3.3 depends on jsonschema<5 and >=3 + connexion 2.10.0 depends on jsonschema<4 and >=2.5.1 + +.. code-block:: bash + +ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. + +The conflict is caused by: + dbt-core 1.10.0 depends on pydantic<2 + apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 + + + +How to reproduce +---------------- + +The table was created by running `nox `__ with the following ``noxfile.py``: + +.. code-block:: python + + import nox + + nox.options.sessions = ["compatibility"] + nox.options.reuse_existing_virtualenvs = True + + + @nox.session(python=["3.10"]) + @nox.parametrize( + "dbt_version", + ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9", "1.10"], + ) + @nox.parametrize( + "airflow_version", + ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "3.0"], + ) + def compatibility(session: nox.Session, airflow_version, dbt_version) -> None: + """Run both unit and integration tests.""" + session.run( + "pip3", + "install", + "--pre", + f"apache-airflow=={airflow_version}", + f"dbt-core=={dbt_version}", + ) diff --git a/docs/configuration/index.rst b/docs/configuration/index.rst index a6042327b0..d699e6189e 100644 --- a/docs/configuration/index.rst +++ b/docs/configuration/index.rst @@ -6,31 +6,88 @@ Configuration Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. .. toctree:: - :caption: Contents: + :maxdepth: 1 + :hidden: + :caption: Translating dbt into Airflow + + Source Nodes Rendering + Post-rendering DAG customization + +.. toctree:: + :maxdepth: 3 + :hidden: + :caption: How Cosmos runs dbt + + execution-modes-local-conflicts + run-dbt/execution-modes + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configure tests + + configure-tests/testing-behavior + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Callbacks + + callbacks/callbacks + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Multi-project Setups - dbt Fusion Multi-Project Setups +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Operators + + Operator Args + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Documentation + + dbt-docs/generating-docs + dbt-docs/hosting-docs + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Optimizing Performance + + Memory Optimization + dbt Fusion + Selecting & Excluding + Parsing Methods + Partial Parsing + Caching + Render Config + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + Project Config Profile Config Execution Config - Render Config - Parsing Methods + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Customizing Airflow + Configuring in Airflow Configuring Lineage - Generating Docs - Hosting Docs Scheduling - Testing Behavior - Selecting & Excluding - Partial Parsing - Source Nodes Rendering - Post-rendering DAG customization - Operator Args Compiled SQL Logging - Caching - Task display name - Callbacks - Memory Optimization + Task display name \ No newline at end of file diff --git a/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst b/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst new file mode 100644 index 0000000000..55d6778abc --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst @@ -0,0 +1,247 @@ +.. _async-execution-mode: + +Airflow Async Execution Mode +============================ + +This execution mode can reduce the runtime by 35% in comparison to Cosmos LOCAL execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. + +It can be particularly useful for long-running transformations, since it leverages Airflow's `deferrable operators `__. + +In this mode, there is a ``SetupAsyncOperator`` that will pre-generate the SQL files for the dbt project and upload them to Airflow XCom or a remote location. A remote location will only be used if users set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator is run before the remaining pipeline. +All the pipeline dbt model transformations will be run using ``DbtRunAirflowAsyncOperator`` which, instead of running the ``dbt run`` command for each model. They will download the SQL files from the Airflow XCom or remote location and execute them directly leveraging the Airflow ``BigQueryInsertJobOperator``. + +Users can leverage other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. + + +Advantages of Airflow Async Mode +++++++++++++++++++++++++++++++++ + +- **Improved Task Throughput:** Async tasks free up Airflow workers by leveraging the Airflow Trigger framework. While long-running SQL transformations are executing in the data warehouse, the worker is released and can handle other tasks, increasing overall task throughput. +- **Better Resource Utilization:** By minimizing idle time on Airflow workers, async tasks allow more efficient use of compute resources. Workers aren't blocked waiting for external systems and can be reused for other work while waiting on async operations. +- **Faster Task Execution:** With Cosmos ``SetupAsyncOperator``, the SQL transformations are precompiled and uploaded to XCom (default behaviour) or a remote location. Instead of invoking a full dbt run during each dbt model task, the SQL files are downloaded from this XCom or remote path and executed directly. This eliminates unnecessary overhead from running the full dbt command, resulting in faster and more efficient task execution. + +We have `observed `_ the following performance improvements by running a dbt project with 129 models: + ++----------------------------------------------+--------------------------+ +| How the dbt pipeline was executed | Execution Time (seconds) | ++==============================================+==========================+ +| ``dbt run`` with dbt Core 1.10 | 13 | ++----------------------------------------------+--------------------------+ +| Cosmos 1.11 with ExecutionMode.LOCAL | 11 | ++----------------------------------------------+--------------------------+ +| Cosmos 1.11 with ExecutionMode.AIRFLOW_ASYNC | 7 | ++----------------------------------------------+--------------------------+ + + +Getting Started with Airflow Async Mode ++++++++++++++++++++++++++++++++++++++++ + +This guide walks you through setting up an Astro CLI project and running a Cosmos-based DAG with a deferrable operator, enabling asynchronous task execution in Apache Airflow. + +Prerequisites ++++++++++++++ + +- `Astro CLI `_ +- Airflow>=2.9 + +1. Create Astro-CLI Project ++++++++++++++++++++++++++++ + +Run the following command in your terminal: + +.. code-block:: bash + + astro dev init + +This will create an Astro project with the following structure: + +.. code-block:: bash + + . + ├── Dockerfile + ├── README.md + ├── airflow_settings.yaml + ├── dags/ + ├── include/ + ├── packages.txt + ├── plugins/ + ├── requirements.txt + └── tests/ + + +2. Update Dockerfile +++++++++++++++++++++ + +Edit your Dockerfile to ensure all necessary requirements are included. + +.. code-block:: bash + + FROM astrocrpublic.azurecr.io/runtime:3.0-2 + + +3. Add astronomer-cosmos Dependency ++++++++++++++++++++++++++++++++++++ + +In your ``requirements.txt``, add: + +.. code-block:: bash + + astronomer-cosmos[dbt-bigquery, google]>=1.9 + + +4. Create Airflow DAG ++++++++++++++++++++++ + +1. Create a new DAG file: ``dags/cosmos_async_dag.py`` + +- Update the ``dataset`` and ``project`` + +.. code-block:: python + + import os + from datetime import datetime + from pathlib import Path + + from cosmos import ( + DbtDag, + ExecutionConfig, + ExecutionMode, + ProfileConfig, + ProjectConfig, + ) + from cosmos.constants import TestBehavior + from cosmos.profiles import GoogleCloudServiceAccountDictProfileMapping + + DEFAULT_DBT_ROOT_PATH = Path(__file__).resolve().parent / "dbt" + DBT_ROOT_PATH = Path(os.getenv("DBT_ROOT_PATH", DEFAULT_DBT_ROOT_PATH)) + DBT_ADAPTER_VERSION = os.getenv("DBT_ADAPTER_VERSION", "1.9") + + cosmos_async_dag = DbtDag( + project_config=ProjectConfig( + DBT_ROOT_PATH / "jaffle_shop", + ), + profile_config=ProfileConfig( + profile_name="default", + target_name="dev", + profile_mapping=GoogleCloudServiceAccountDictProfileMapping( + conn_id="gcp_conn", + profile_args={ + "dataset": "cosmos_async_demo", + "project": "astronomer-**", + }, + ), + ), + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AIRFLOW_ASYNC, + async_py_requirements=[f"dbt-bigquery=={DBT_ADAPTER_VERSION}"], + ), + schedule=None, + start_date=datetime(2025, 1, 1), + catchup=False, + dag_id="cosmos_async_dag", + operator_args={ + "location": "US", + "install_deps": True, + "full_refresh": True, + "virtualenv_dir": "dbt_venv", + }, + ) + +2. Folder structure for dbt project + +- Add a valid dbt project inside your Airflow project under ``dags/dbt/``. + + +5. Start the Project +++++++++++++++++++++ + +Launch the Airflow project locally: + +.. code-block:: bash + + astro dev start + +This will: + +- Spin up the scheduler, webserver, and triggerer (needed for deferrable operators) +- Expose Airflow UI at http://localhost:8080 + +6. Create Airflow Connection +++++++++++++++++++++++++++++ + +Create an Airflow connection with following configurations + +- Connection ID: gcp_conn +- Connection Type: google_cloud_platform +- Extra Fields JSON: + +.. code-block:: bash + + { + "project": "astronomer-**", + "keyfile_dict": { + "type": "***", + "project_id": "***", + "private_key_id": "***", + "private_key": "***", + "client_email": "***", + "client_id": "***", + "auth_uri": "***", + "token_uri": "***", + "auth_provider_x509_cert_url": "***", + "client_x509_cert_url": "***", + "universe_domain": "***" + } + } + + +7. Execute the DAG +++++++++++++++++++ + +1. Visit the Airflow UI at ``http://localhost:8080`` +2. Enable the DAG: ``cosmos_async_dag`` +3. Trigger the DAG manually + +.. image:: /_static/jaffle_shop_async_execution_mode.png + :alt: Cosmos dbt Async DAG + :align: center + +The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks. + + +Control of where to upload the SQL files +++++++++++++++++++++++++++++++++++++++++ + +For optimal performance we encourage to keep Cosmos standard behaviour (introduced in 1.11), which is to upload the SQL files to XCom, instead of a remote object location. + +For the benchmark example described in a previous section, there was an overhead of ~500 seconds with remote SQL file upload/download, but only ~2 seconds using XCom, which can outweigh the performance improvements introduced by using deferrable operators. + +However, if you want to upload the SQL files to a remote object location instead of XCom, you can set the following environment variables: + +.. code-block:: bash + + AIRFLOW__COSMOS__REMOTE_TARGET_PATH=gs://cosmos_remote_target_demo + AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID=gcp_conn + + +Limitations ++++++++++++ + + +1. **Limited to dbt models**: Only dbt resource type models are run asynchronously using Airflow deferrable operators. Other resource types are executed synchronously, similar to the local execution mode. + +2. **BigQuery support only**: This mode only supports BigQuery as the target database. If a different target is specified, Cosmos will throw an error indicating the target database is unsupported in this mode. Adding support for other adapters is on the roadmap. + +3. **ProfileMapping parameter required**: You need to specify the ``ProfileMapping`` parameter in the ``ProfileConfig`` for your DAG. Refer to the example DAG below for details on setting this parameter. + +4. **Location parameter required**: You must specify the location of the BigQuery dataset in the ``operator_args`` of the ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. + +5. **async_py_requirements parameter required**: If you're using the default approach of having a setup task, you must specify the necessary dbt adapter Python requirements based on your profile type for the async execution mode in the ``ExecutionConfig`` of your ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. + +6. **Creation of new isolated virtual environment for each task run**: By default, the ``SetupAsyncOperator`` creates and executes within a new isolated virtual environment for each task run, which can cause performance issues. To reuse an existing virtual environment, use the ``virtualenv_dir`` parameter within the ``operator_args`` of the ``DbtDag``. We have observed that for ``dbt-bigquery``, the ``SetupAsyncOperator`` executes approximately 30% faster when reusing an existing virtual environment, particularly for transformations that take around 10 minutes to complete. + +7. **Performance degradation when uploading to remote object location**: Even though it is possible to upload the SQL files to a remote object location by setting environment variables, it is slow. We observed that this introduces a significant overhead in the execution time (500s for 129 models). + +8. **TeardownAsyncOperator limitation**: When using a remote object location, in addition to the ``SetupAsyncOperator``, a ``TeardownAsyncOperator`` is also added to the DAG. This task will delete the SQL files from the remote location by the end of the DAG Run. This is can lead to a limitation from a retry perspective, as described in the issue `#2066 `_. This can be avoided by setting the ``enable_teardown_async_task`` configuration to ``False``, as described in the :ref:`enable_teardown_async_task` section. + +For a comparison between different Cosmos execution modes, please, check the :ref:`execution-modes-comparison` section. diff --git a/docs/configuration/run-dbt/airflow-worker/index.rst b/docs/configuration/run-dbt/airflow-worker/index.rst new file mode 100644 index 0000000000..00cb281bc8 --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/index.rst @@ -0,0 +1,9 @@ +Run dbt in an Airflow worker +============================ + +.. toctree:: + :maxdepth: 1 + :caption: Run dbt in an Airflow worker + + async-execution-mode + watcher-execution-mode \ No newline at end of file diff --git a/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst new file mode 100644 index 0000000000..af7589650c --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst @@ -0,0 +1,480 @@ +.. _watcher-execution-mode: + +Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Execution in Cosmos +============================================================================================ + +With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. + +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_ and Airflow deferrable sensors, the performance gains will depend on three major factors: + +- The amount of dbt ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag +- The topology of the dbt pipeline +- The ``poke_interval`` and ``timeout`` settings of the ``DbtConsumerWatcherSensor`` operator, which determine the frequency and duration of the sensor's polling. + +------------------------------------------------------------------------------- + +Background: The Problem with the Local Execution Mode in Cosmos +--------------------------------------------------------------- + +When running dbt via Cosmos using the default ``ExecutionMode.LOCAL``, each dbt model is executed as a separate Airflow task. + +This provides strong observability and task-level retry control — but it comes at a cost. Each model runs a new dbt process, which introduces significant overhead. + +Consider the `google/fhir-dbt-analytics `_ project: + ++-------------------------------------------------------------+-----------------------------------+------------------+ +| Run Type | Description | Total Runtime | ++=============================================================+===================================+==================+ +| Single ``dbt run`` (dbt CLI) | Runs the whole DAG in one command | ~5m 30s | ++-------------------------------------------------------------+-----------------------------------+------------------+ +| One ``dbt run`` per model, totalling 184 commands (dbt CLI) | Each model is its own task | ~32m | ++-------------------------------------------------------------+-----------------------------------+------------------+ + +This difference motivated a rethinking of how Cosmos interacts with dbt. + +------------------------------------------------------------------------------- + +Concept: ``ExecutionMode.WATCHER`` +---------------------------------- + +``ExecutionMode.WATCHER`` combines the **speed of a single dbt run** with the **observability and task management of Airflow**. + +It is built on two operator types: + +* ``DbtProducerWatcherOperator`` (`#1982 `_) + Runs dbt **once** across the entire pipeline, register to `dbt event callbacks `_ and sends model progress updates via Airflow **XComs**. + +* ``DbtConsumerWatcherSensor`` (`#1998 `_) + Watches those XComs and marks individual Airflow tasks as complete when their corresponding dbt models finish. + +Together, these operators let you: + +* Run dbt as a single command (for speed) +* Retain model-level observability (for clarity) +* Retry specific models (for resilience) + +------------------------------------------------------------------------------- + +Performance Gains +----------------- + +We used a dbt project developed by Google, the `google/fhir-dbt-analytics `_ project, that interfaces with BigQuery. It contains: +* 2 seeds +* 52 sources +* 185 models + +Initial benchmarks, using illustrate significant improvements: + ++-----------------------------------------------+-----------+--------------------+ +| Environment | Threads | Execution Time (s) | ++===============================================+===========+====================+ +| dbt build (dbt CLI) | 4 | 6–7 | ++-----------------------------------------------+-----------+--------------------+ +| dbt run per model (dbt CLI) | — | 30 | +| similar to the Cosmos ``ExecutionMode.LOCAL`` | | | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.LOCAL`` (Astro CLI) | — | 10–15 | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.WATCHER`` (Astro CLI) | 1 | 26 | +| | 2 | 14 | +| | 4 | 7 | +| | 8 | 4 | +| | 16 | 2 | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.WATCHER`` (Astro Cloud | 8 | ≈5 | +| Standard Deployment with A10 workers | | | ++-----------------------------------------------+-----------+--------------------+ + +The last line represents the performance improvement in a real-world Airflow deployment, using `Astro Cloud `_. + +Depending on the dbt workflow topology, if your dbt DAG previously took 5 minutes with ``ExecutionMode.LOCAL``, you can expect it to complete in roughly **1 minute** with ``ExecutionMode.WATCHER``. + +We plan to repeat these benchmarks and share the code with the community in the future. + + +.. note:: + ``ExecutionMode.WATCHER`` relies on the ``threads`` value defined in your dbt profile. Start with a conservative value that matches the CPU capacity of your Airflow workers, then gradually increase it to find the sweet spot between faster runs and acceptable memory/CPU usage. + +When we ran the `astronomer/cosmos-benchmark `_ project with ``ExecutionMode.WATCHER``, that same ``threads`` setting directly affected runtime: moving from 1 to 8 threads reduced the end-to-end ``dbt build`` duration from roughly 26 seconds to about 4 seconds (see table above), while 16 threads squeezed it to around 2 seconds at the cost of higher CPU usage. Use those numbers as a reference point when evaluating how thread counts scale in your own environment. + +To increase the number of threads, edit your dbt ``profiles.yml`` (or Helm values if you manage the profile there) and update the ``threads`` key for the target you use with Cosmos: + +.. code-block:: yaml + + your_dbt_project: + target: prod + outputs: + prod: + type: postgres + host: your-host + user: your-user + password: your-password + schema: analytics + threads: 8 # increase or decrease to match available resources + + +If you prefer to manage threads through Cosmos profile mappings instead of editing ``profiles.yml`` directly, pass ``profile_args={"threads": }`` to your ``ProfileConfig``. For example, using the built-in ``PostgresUserPasswordProfileMapping``: + +.. code-block:: python + + from cosmos.config import ProfileConfig + from cosmos.profiles import PostgresUserPasswordProfileMapping + + profile_config = ProfileConfig( + profile_name="jaffle_shop", + target_name="prod", + profile_mapping=PostgresUserPasswordProfileMapping( + conn_id="postgres_connection", + profile_args={"threads": 8}, + ), + ) + + +------------------------------------------------------------------------------- + +Example Usage of ``ExecutionMode.WATCHER`` +------------------------------------------ + +There are two main ways to use the new execution mode in Cosmos — directly within a ``DbtDag``, or embedded as part of a ``DbtTaskGroup`` inside a larger DAG. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can enable WATCHER mode directly in your ``DbtDag`` configuration. +This approach is best when your Airflow DAG is fully dedicated to a dbt project. + +.. literalinclude:: ../../dev/dags/example_watcher.py + :language: python + :start-after: [START example_watcher] + :end-before: [END example_watcher] + +As it can be observed, the only difference with the default ``ExecutionMode.LOCAL`` is the addition of the ``execution_config`` parameter with the ``execution_mode`` set to ``ExecutionMode.WATCHER``. The ``ExecutionMode`` enum can be imported from ``cosmos.constants``. For more information on the ``ExecutionMode.LOCAL``, please, check the `dedicated page `__ + +**How it works:** + +* Cosmos executes your dbt project once via a producer task. +* Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. +* The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). + +**How it looks like:** + +.. image:: /_static/jaffle_shop_watcher_dbt_dag_dag_run.png + :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` + :align: center + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Example 2 — Using ``DbtTaskGroup`` with ``ExecutionMode.WATCHER`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If your Airflow DAG includes multiple stages or integrations (e.g., data ingestion → dbt → reporting), use ``DbtTaskGroup`` to embed your dbt project into a larger DAG — still benefiting from WATCHER performance. + +.. code-block:: python + :caption: example_watcher_taskgroup.py + :name: example_watcher_taskgroup + + from airflow.models import DAG + from airflow.operators.empty import EmptyOperator + from cosmos import DbtTaskGroup + + with DAG( + dag_id="example_watcher_taskgroup", + schedule="@daily", + start_date=datetime(2023, 1, 1), + catchup=False, + ): + """ + The simplest example of using Cosmos to render a dbt project as a TaskGroup. + """ + pre_dbt = EmptyOperator(task_id="pre_dbt") + + first_dbt_task_group = DbtTaskGroup( + group_id="first_dbt_task_group", + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER, + ), + project_config=ProjectConfig(DBT_PROJECT_PATH), + profile_config=profile_config, + operator_args=operator_args, + ) + + pre_dbt >> first_dbt_task_group + +**Key advantages:** + +* Integrates seamlessly into complex Airflow DAGs. +* Uses the same high-performance producer/consumer execution model. +* Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. + +.. image:: /_static/jaffle_shop_watcher_dbt_taskgroup_dag_run.png + :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` + :align: center + +------------------------------------------------------------------------------- + +Additional details +------------------- + +~~~~~~~~~~~~~~~~ +How retries work +~~~~~~~~~~~~~~~~ + +When the ``dbt build`` command run by ``DbtProducerWatcherOperator`` fails, it will notify all the ``DbtConsumerWatcherSensor``. + +The individual watcher tasks that subclass ``DbtConsumerWatcherSensor`` can retry the dbt command themselves, using the same behavior as ``ExecutionMode.LOCAL``. + +If a branch of the DAG fails, users can clear the status of a failed consumer task, including its downstream tasks, via the Airflow UI, and each of them will run in ``ExecutionMode.LOCAL``. + +**Producer retry behavior** + +.. versionadded:: 1.12.2 + +When the ``DbtProducerWatcherOperator`` is triggered for a retry (try_number > 1), it will not re-run the dbt build command and will succeed. In previous versions of Cosmos, the producer task would fail during retries. +This behavior is designed to support TaskGroup-level retries, as reported in `#2282 `_. + +**Why this matters:** + +- In earlier versions, attempting to retry the producer task would raise an ``AirflowException``, causing the retry to fail immediately. +- Now, the producer gracefully skips execution on retries, logging an informational message explaining that the retry was skipped to avoid running a second ``dbt build``. +- This allows users to retry entire TaskGroups and/or DAGs without the producer task blocking the retry flow. + +**Important considerations:** + +- The producer task should still be configured with ``retries=0`` (which Cosmos enforces by default) to avoid unintended duplicate ``dbt build`` runs. + +- By default, Cosmos sets ``retries`` to ``0`` in``DbtProducerWatcherOperator``. Users can retry manually by clearing the status of the producer task and all its downstream tasks, keeping in mind that the producer task will not re-run the ``dbt build`` command and will succeed. + +The overall retry behavior will be further improved once `#1978 `_ is implemented. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Watcher dbt Execution Queue +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. versionadded:: 1.14.0 + +In watcher execution mode, by default, consumer sensor tasks are lightweight sensors that wait for the producer task to complete. On their first attempt, they require minimal CPU and memory resources. However, when these tasks retry, they execute the dbt command for the node, which may require significantly more resources. + +The ``watcher_dbt_execution_queue`` configuration allows you to specify a different worker queue for retry attempts. This enables you to: + +- **Optimize resource allocation** — Use lightweight workers for initial sensor execution and high-resource workers for retries +- **Improve scheduling efficiency** — Prevent resource contention between initial sensor tasks and retry executions +- **Scale independently** — Scale retry queues separately based on retry workload patterns + +**Configuration:** + +Set the ``watcher_dbt_execution_queue`` in your Airflow configuration: + +.. code-block:: ini + + [cosmos] + watcher_dbt_execution_queue = high_memory_queue + +Or via environment variable: + +.. code-block:: bash + + export AIRFLOW__COSMOS__WATCHER_DBT_EXECUTION_QUEUE=high_memory_queue + +**How it works:** + +- For watcher producer tasks (``DbtProducerWatcherOperator``), the configured queue is used during their first execution +- For watcher consumer tasks (``DbtConsumerWatcherSensor``), from their first retry onwards, if ``watcher_dbt_execution_queue`` is configured, the task is automatically assigned to the specified queue +- This behavior is enforced by Cosmos via an `Airflow cluster policy `_ (``task_instance_mutation_hook``) that mutates ``task_instance.queue`` at runtime for retry attempts + +.. note:: + + For producer task execution, we encourage users to set the ``watcher_dbt_execution_queue`` configuration. If, for any reason, users prefer to use a different node pool for producer tasks without setting an Airflow Cluster Policy, they can set the ``queue`` argument via ``setup_operator_args``. This, however, would not solve the problem of assigning consumer retries to nodes that may have more memory and CPU available. + + The effective precedence is: + + ``watcher_dbt_execution_queue`` > explicit ``queue`` on the producer (from ``setup_operator_args``) > ``operator_args`` > your Airflow deployment’s default queue. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Installation of Airflow and dbt +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since Cosmos 1.12.0, ``ExecutionMode.WATCHER`` works well regardless of whether dbt and Airflow are installed in the same Python virtual environment. + +When dbt and Airflow are installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` uses dbt `callback features `_. + +When dbt and Airflow are not installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` consumes the dbt `structured logging `_ to update the consumer tasks. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Synchronous versus Asynchronous sensor execution +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In Cosmos 1.11.0, the ``DbtConsumerWatcherSensor`` operator is implemented as a synchronous XCom sensor, which continuously occupies the worker slot - even if they're just sleeping and checking periodically. + +Starting with Cosmos 1.12.0, the ``DbtConsumerWatcherSensor`` supports +`deferrable (asynchronous) execution `_. Deferrable execution frees up the Airflow worker slot, while task status monitoring is handled by the Airflow triggerer component, +which increases overall task throughput. By default, the sensor now runs in deferrable mode. + +------------------------------------------------------------------------------- + +Known Limitations +------------------- + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Producer task implementation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The producer task is implemented as a ``DbtProducerWatcherOperator`` and currently relies on dbt being installed alongside the Airflow deployment, as in the ``ExecutionMode.LOCAL`` implementation. + +The alternative to this implementation is to use ``ExecutionMode.WATCHER_KUBERNETES``, which is built on top of ``ExecutionMode.KUBERNETES``. Check :ref:`watcher-kubernetes-execution-mode` for more information. + +~~~~~~~~~~~~~~~~~~~~~~~~ +Individual dbt Operators +~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``ExecutionMode.WATCHER`` efficiently implements the following operators: +* ``DbtSeedWatcherOperator`` +* ``DbtSnapshotWatcherOperator`` +* ``DbtRunWatcherOperator`` + +However, other operators that are available in the ``ExecutionMode.LOCAL`` mode are not implemented. + +The ``DbtBuildWatcherOperator`` is not implemented, since the build command is executed by the producer ``DbtProducerWatcherOperator`` operator. + +Additionally, since the ``dbt build`` command does not run ``source`` nodes, the operator ``DbtSourceWatcherOperator`` is equivalent to the ``DbtSourceLocalOperator`` operator, from ``ExecutionMode.LOCAL``. + +Finally, the following features are not implemented as operators under ``ExecutionMode.WATCHER``: + +* ``dbt ls`` +* ``dbt run-operation`` +* ``dbt docs`` +* ``dbt clone`` + +You can still invoke these operators using the default ``ExecutionMode.LOCAL`` mode. + +~~~~~~~~~~~~~ +Test behavior +~~~~~~~~~~~~~ + +By default, the watcher mode runs tests alongside models via the ``dbt build`` command being executed by the producer ``DbtProducerWatcherOperator`` operator. + +As a starting point, this execution mode does not support the ``TestBehavior.AFTER_EACH`` behavior, since the tests are not run as individual tasks. Since this is the default ``TestBehavior`` in Cosmos, we are injecting ``EmptyOperator`` as a starting point to ensure a seamless transition to the new mode. + +The ``TestBehavior.BUILD`` behavior is embedded in the producer ``DbtProducerWatcherOperator`` operator. + +The ``TestBehavior.NONE`` and ``TestBehavior.AFTER_ALL`` behave similarly to ``ExecutionMode.LOCAL``. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Airflow Datasets and Assets +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +While the ``ExecutionMode.WATCHER`` supports the ``emit_datasets`` parameter, the Airflow Datasets and Assets are emitted from the ``DbtProducerWatcherOperator`` task instead of the consumer tasks, as done for other Cosmos' execution modes. + +~~~~~~~~~~~~~~~~~~~~~~ +Source freshness nodes +~~~~~~~~~~~~~~~~~~~~~~ + +Since Cosmos 1.6, it `supports the rendering of source nodes `_. + +We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. + +This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. + +We have a follow-up ticket to `further investigate this use case `_. + + +Advanced config +------------------- + +~~~~~~~~~~~~~~~~ +Callback support +~~~~~~~~~~~~~~~~ + +The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` will use the user-defined callback function similar to ``ExecutionMode.LOCAL`` mode. + +You can define different ``callback`` behaviors for producer and consumer nodes by using ``operator_args`` to configure the consumer callback and ``setup_operator_args`` to override the callback for the producer, as described below. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Overriding ``operator_args`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` operators handle ``operator_args`` similar to the ``ExecutionMode.LOCAL`` mode. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Using Custom Args for the Producer and Watcher +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. versionadded:: 1.12.0 + +If you need to override ``operator_args`` for the ``DbtProducerWatcherOperator``, you can do so using ``setup_operator_args``. + +When using ``ExecutionMode.WATCHER``, you may want to configure specific properties, such as ``retries`` specifically for the ``DbtProducerWatcherOperator`` task. This can be useful for several reasons: +- Improved resilience - transient issues (e.g., temporary database or network failures) can be automatically retried. +- Reduced manual intervention - failed producer runs can recover without requiring operator restarts. +- Better reliability - retry behavior can be tuned independently from sensor tasks. + +Example: Configure the producer task with custom retry settings. + +.. code-block:: python + + from datetime import timedelta + from cosmos.config import ExecutionConfig + from cosmos.constants import ExecutionMode + + execution_config = ExecutionConfig( + execution_mode=ExecutionMode.WATCHER, + setup_operator_args={ + "retries": 0, + "retry_delay": timedelta(minutes=5), + }, + ) + +This allows you to customize ``DbtProducerWatcherOperator`` retry behavior without affecting the arguments used by the other sensor tasks. + +If configuring queues, we suggest using the previously mentioned ``watcher_dbt_execution_queue`` configuration instead of the ``setup_operator_args``. + +.. note:: + Please note that ``setup_operator_args`` is specific to Cosmos and is not related to Airflow setup or teardown task. + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Sensor slot allocation and polling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each ``DbtDag`` or ``DbtTaskGroup`` root node will startup during DAG runs at - potentially - the same time as the DAG Run. This may not happen, since it is dependent on the +concurrency settings and available task slots in the Airflow deployment. + +The consequence is that tasks may take longer to be updated if they are not sensing at the moment that the transformation happens. + +We plan to review this behaviour and alternative approaches in the future. + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Asynchronous sensor execution +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Deferrable execution is currently supported only for dbt models, seeds and snapshots. +- Deferrable execution applies only to the first task attempt (try number 1). For subsequent retries, the sensor falls back to synchronous execution. + +To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. + +.. literalinclude:: ../../dev/dags/example_watcher.py + :language: python + :start-after: [START example_watcher_synchronous] + :end-before: [END example_watcher_synchronous] + +------------------------------------------------------------------------------- + +Troubleshooting +--------------- + +Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." +Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. Check the `dbt threading docs `_ for more information on how to set the number of threads. + + +Summary +------- + +``ExecutionMode.WATCHER`` represents a significant leap forward for running dbt in Airflow via Cosmos: + +* ✅ Up to **5× faster** dbt DAG runs +* ✅ Maintains **model-level visibility** in Airflow +* ✅ Enables **smarter resource allocation** +* ✅ Built on proven Cosmos rendering techniques + +This is an experimental feature, and we are looking for feedback from the community. + +Stay tuned for further documentation and base image support for the ``ExecutionMode.WATCHER`` in upcoming releases. diff --git a/docs/configuration/run-dbt/container/aws-container-run-job.rst b/docs/configuration/run-dbt/container/aws-container-run-job.rst new file mode 100644 index 0000000000..4321c8f346 --- /dev/null +++ b/docs/configuration/run-dbt/container/aws-container-run-job.rst @@ -0,0 +1,191 @@ +.. _aws-container-run-job: + +Getting Started with Astronomer Cosmos on AWS ECS +================================================== + +Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. +Schematically, the guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aws_ecs_schematic.png + :width: 800 + +Prerequisites ++++++++++++++ + +Before you begin, ensure you have the following: + +- An active **AWS Account** with permissions to create ECS clusters, register task definitions, and run tasks. +- The **AWS CLI** installed and configured with the proper credentials. +- **Docker** installed for building your container image. +- Access to your container registry (for example, **Amazon ECR**) where your job image is stored. +- Basic familiarity with AWS ECS concepts (clusters, task definitions, services, and Fargate). +- An existing installation of **Astronomer Cosmos** (refer to the `Cosmos documentation `_ for more details). + + + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: + +.. code-block:: bash + + python3 -m venv venv + source venv/bin/activate + python3 -m pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[amazon]" + pip install "aiobotocore[boto3]" +.. note:: + The package aiobotocore[boto3] is optional; you will need it if you plan to use **deferred tasks**. + +**Set up your ECR** + +1. **Set your secrets** + On the `cosmos-examples `_ repository, you can find a ready-to-use Docker image for the AWS ECS service. Just replace your secrets, or you can create your own. + +2. **AWS CLI login** + Before building and pushing your image, you first need to log in to the AWS service using the AWS CLI tool. + Use the following command: + + .. code-block:: bash + + aws ecr-public get-login-password --region | docker login --username AWS --password-stdin + +3. **Build and tag your image** + Once you have your image ready, run the following commands: + + .. code-block:: bash + + docker build -f Dockerfile.aws_ecs . --platform=linux/amd64 -t + docker tag + +4. **Push your image** + + .. code-block:: bash + + docker push + +**Configure Your AWS Environment** + +1. **Create an ECS Cluster** + + Create an ECS cluster to host your Cosmos jobs. You can do this from the AWS Console or using the AWS CLI: + + .. code-block:: bash + + aws ecs create-cluster --cluster-name my-cosmos-cluster + +2. **Set Up an IAM Role for ECS Tasks** + + Ensure you have an IAM role that your ECS tasks can assume. This role should include permissions for ECS, ECR, and CloudWatch (for logs). For example, you might create a role named ``ecsTaskExecutionRole`` with the managed policies: + + - ``AmazonECSTaskExecutionRolePolicy`` + - (Optional) Additional policies for custom resource access + +3. **Configure Networking** + + For Fargate tasks, make sure you have at least one subnet (preferably in multiple Availability Zones) and a security group that permits outbound internet access if needed. Note the subnet IDs for later use. + +**Prepare Your Cosmos Job Definition** + +Cosmos jobs are defined as container tasks. Create a task definition file (e.g., ``cosmos-task-definition.json``) with the configuration for your job. + +For example: + +.. code-block:: json + + { + "family": "cosmos-job", + "networkMode": "awsvpc", + "requiresCompatibilities": [ + "FARGATE" + ], + "cpu": "512", + "memory": "1024", + "executionRoleArn": "arn:aws:iam:::role/ecsTaskExecutionRole", + "containerDefinitions": [ + { + "name": "cosmos-job", + "image": "/your_image:latest", + "essential": true, + "environment": [ + { "name": "VAR1", "value": "value1" }, + { "name": "VAR2", "value": "value2" } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/cosmos-job", + "awslogs-region": "us-east-1", + "awslogs-stream-prefix": "ecs" + } + } + } + ] + } + +.. note:: + + Replace ````, ````, and adjust the CPU, memory, and environment variables as needed. + +**Deploy Your Cosmos Job on AWS ECS** + +1. **Register the Task Definition** + + Use the AWS CLI to register your task definition: + + .. code-block:: bash + + aws ecs register-task-definition --cli-input-json file://cosmos-task-definition.json + +2. **Run the Task** + + Run a test task on your ECS cluster. Specify the subnets and security groups in your network configuration. For example: + + .. code-block:: bash + + aws ecs run-task \ + --cluster my-cosmos-cluster \ + --launch-type FARGATE \ + --task-definition cosmos-job \ + --network-configuration "awsvpcConfiguration={subnets=[subnet-12345678,subnet-87654321],securityGroups=[sg-abcdef12],assignPublicIp=ENABLED}" + + Once the test is ok, we are able to run the dbt commands in our Cosmos DAG: + + .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run.png + :width: 800 + + .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run_logs.png + :width: 800 + + Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! + + +**Monitor and Debug Your Job** + +1. **Check Task Status** + + You can view the status of your task from the AWS Console under your ECS cluster or via the CLI: + + .. code-block:: bash + + aws ecs describe-tasks --cluster my-cosmos-cluster --tasks + +2. **View Logs** + + Since the task definition configures AWS CloudWatch Logs, you can view your job’s output in the CloudWatch Logs console. Look for log streams with the prefix you set (e.g., ``ecs/cosmos-job``). + +**Conclusion** + + +By following this guide, you can deploy Astronomer Cosmos jobs on AWS ECS using Fargate. This integration enables you to leverage the scalability and managed infrastructure of ECS while maintaining a consistent container orchestration experience with Cosmos. + +For more detailed information on AWS ECS, please refer to the `AWS ECS Developer Guide `_. + +Happy deploying! :rocket: + + +Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! diff --git a/docs/configuration/run-dbt/container/azure-container-instance.rst b/docs/configuration/run-dbt/container/azure-container-instance.rst new file mode 100644 index 0000000000..86ce3ab9ef --- /dev/null +++ b/docs/configuration/run-dbt/container/azure-container-instance.rst @@ -0,0 +1,138 @@ +.. _azure-container-instance: + +Azure Container Instance Execution Mode +======================================= +.. versionadded:: 1.4 + +This tutorial will guide you through the steps required to use Azure Container Instance as the Execution Mode for your dbt code with Astronomer Cosmos. Schematically, the guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aci_schematic.png + :width: 800 + +Prerequisites ++++++++++++++ +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Azure CLI (install guide here: `Azure CLI `_) +4. Astronomer-cosmos package containing the dbt Azure Container Instance operators +5. Azure account with: + 1. A resource group + 2. A service principal with `Contributor` permissions on the resource group + 3. A Container Registry + 4. A Postgres instance accessible from Azure. (we use an Azure Postgres instance in the example) +6. Docker image built with required dbt project and dbt DAG +7. dbt DAG with dbt Azure Container Instance operators in the Airflow DAGs directory to run in Airflow + +More information on how to achieve 2-6 is detailed below. + +Note that the steps below will walk you through an example, for which the code can be found HERE + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install apache airflow & astronomer-postgres + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-postgres,azure-container-instance]" + +**Setup Postgres database** + +You will need a postgres database running to be used as the database for the dbt project. In order to have it accessible from Azure Container Instance, the easiest way is to create an Azure Postgres instance. For this, run the following (assuming you are logged into your Azure account) + +.. code-block:: bash + + az postgres server create -l westeurope -g <<>> -n <<>> -u dbadmin -p <<>> --sku-name B_Gen5_1 --ssl-enforcement Enabled + + +**Setup Azure Container Registry** +In order to run a container in Azure Container Instance, it needs access to the container image. In our setup, we will use Azure Container Registry for this. To set an Azure Container Registry up, you can use the following bash command: + +.. code-block:: bash + + az acr create --name <<>> --resource-group <<>> --sku Basic --admin-enabled + +**Build the dbt Docker image** + +For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. + +Clone the `cosmos-example `_ repo + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. + +.. code-block:: bash + + docker build -t <<>:1.0.0 -f Dockerfile.azure_container_instance . + +After this, the image needs to be pushed to the registry of your choice. Note that your image name should contain the name of your registry: +.. code-block:: bash + + docker push <<>>:1.0.0 + +.. note:: + + You may need to ensure docker knows to use the right credentials. If using Azure Container Registry, this can be done by running the following command: + .. code-block:: bash + + az acr login + +.. note:: + + If running on M1, you may need to set the following envvars for running the docker build command in case it fails + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The `dbt profile `_ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Setup Airflow Connections** +Now you have the required Azure infrastructure, you still need to add configuration to Airflow to ensure the infrastructure can be used. You'll need 3 connections: + +1. ``aci_db``: a Postgres connection to your Azure Postgres instance. +2. ``aci``: an Azure Container Instance connection configured with a Service Principal with sufficient permissions (i.e. ``Contributor`` on the resource group in which you will use Azure Container Instances). +3. ``acr``: an Azure Container Registry connection configured for your Azure Container Registry. + +Check out the ``airflow-settings.yml`` file `here `_ for an example. If you are using Astro CLI, filling in the right values here will be enough for this to work. + +**Setup and Trigger the DAG with Airflow** + +Copy the dags directory from cosmos-example repo to your Airflow home + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_azure_container_instance `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_azure_container_instance.png + :width: 800 diff --git a/docs/configuration/run-dbt/container/docker.rst b/docs/configuration/run-dbt/container/docker.rst new file mode 100644 index 0000000000..0005914886 --- /dev/null +++ b/docs/configuration/run-dbt/container/docker.rst @@ -0,0 +1,111 @@ +.. _docker: + +Docker Execution Mode +======================================== + +The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. + +Requirements +++++++++++++ + +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Astronomer-cosmos package containing the dbt Docker operators +4. Postgres docker container +5. Docker image built with required dbt project and dbt DAG +6. dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow + +More information on how to achieve 2-6 is detailed below. + +Step-by-step instructions ++++++++++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install `Apache Airflow® `_ & astronomer-postgres + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-postgres]" + +**Setup Postgres database** + +You will need a postgres database running to be used as the database for the dbt project. Run the following command to run and expose a postgres database + +.. code-block:: bash + + docker run --name some-postgres -e POSTGRES_PASSWORD="" -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p5432:5432 -d postgres + +**Build the dbt Docker image** + +For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. + +Clone the `cosmos-example `_ repo + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. + +.. code-block:: bash + + docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . + +.. note:: + + If running on M1, you may need to set the following envvars for running the docker build command in case it fails + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The `dbt profile `_ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Setup and Trigger the DAG with Airflow** + +Copy the dags directory from cosmos-example repo to your Airflow home + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_docker `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_docker_dag_run.png + :width: 800 + + +Specifying ProfileConfig ++++++++++++++++++++++++++ + +Starting with Cosmos 1.8.0, you can use the ``profile_config`` argument in your Dbt DAG Docker operators to reference +profiles for your dbt project defined in a profiles.yml file. To do so, provide the file’s path via the +``profiles_yml_path`` parameter in ``profile_config``. + +Note that in ``ExecutionMode.DOCKER``, the ``profile_config`` is only compatible with the ``profiles_yml_path`` +approach. The ``profile_mapping`` method will not work because the required Airflow connections cannot be accessed +within the Docker container to map them to the dbt profile. diff --git a/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst b/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst new file mode 100644 index 0000000000..fa4d0c60c4 --- /dev/null +++ b/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst @@ -0,0 +1,265 @@ +.. _gcp-cloud-run-job: + +GCP Cloud Run Job Execution Mode +======================================= +.. versionadded:: 1.7 + +This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_gcp_crj_schematic.png + :width: 600 + +Prerequisites ++++++++++++++ +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Google Cloud SDK (`install guide `_) +4. Astronomer-cosmos package containing the dbt Cloud Run Job operators +5. GCP account with: + 1. A GCP project (`setup guide `_) + 2. IAM roles: + * Basic Role: `Owner `_ (control over whole project) or + * Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) + 3. Enabled service APIs: + * Artifact Registry API + * Cloud Run Admin API + * BigQuery API + 4. A service account with BigQuery roles: `JobUser `_ and `DataEditor `_ +6. Docker image built with required dbt project and dbt DAG +7. dbt DAG with Cloud Run Job operators in the Airflow DAGs directory to run in Airflow + +.. note:: + + Google Cloud Platform provides free tier on many resources, as well as Free Trial with $300 in credit. Learn more `here `_. + +More information on how to achieve 2-6 is detailed below. + + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: + +.. code-block:: bash + + python3 -m venv venv + source venv/bin/activate + python3 -m pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-bigquery,gcp-cloud-run-job]" + +**Setup gcloud and environment variables** + +Set environment variables that will be used to create cloud infrastructure. Replace placeholders with your unique GCP ``project id`` and ``region`` of the project: + +.. code-block:: bash + + export PROJECT_ID=<<>> + export REGION=<<>> + export REPO_NAME="astronomer-cosmos-dbt" + export IMAGE_NAME="$REGION-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/cosmos-example" + export SERVICE_ACCOUNT_NAME="cloud-run-job-sa" + export DATASET_NAME="astronomer_cosmos_example" + export CLOUD_RUN_JOB_NAME="astronomer-cosmos-example" + +Before we do anything in the GCP project, we first need to authorize gcloud to access the Cloud Platform with Google user credentials: + +.. code-block:: bash + + gcloud auth login + +You'll receive a link to sign into Google Cloud SDK using a Google Account. + +Next, set default ``project id`` using below command: + +.. code-block:: bash + + gcloud config set project $PROJECT_ID + +In case BigQuery has never been used before in the project, run below command to enable BigQuery API: + +.. code-block:: bash + + gcloud services enable bigquery.googleapis.com + +**Setup Artifact Registry** + +In order to run a container in Cloud Run Job, it needs access to the container image. In our setup, we will use Artifact Registry repository that stores images. +To use Artifact Registry, you need to enable the API first: + +.. code-block:: bash + + gcloud services enable artifactregistry.googleapis.com + +To set an Artifact Registry repository up, you can use the following bash command: + +.. code-block:: bash + + gcloud artifacts repositories create $REPO_NAME \ + --repository-format=docker \ + --location=$REGION \ + --project $PROJECT_ID + +**Setup Service Account** + +In order to use dbt and make transformations in BigQuery, Cloud Run Job needs some BigQuery permissions. One way to achieve that is to set up a separate ``Service Account`` with needed permissions: + +.. code-block:: bash + + # create a service account + gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME + +.. code-block:: bash + + # grant JobUser role + gcloud projects add-iam-policy-binding $PROJECT_ID \ + --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ + --role="roles/bigquery.jobUser" + +.. code-block:: bash + + # grant DataEditor role + gcloud projects add-iam-policy-binding $PROJECT_ID \ + --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ + --role="roles/bigquery.dataEditor" + +**Build the dbt Docker image** + +Now, we are going to download an example dbt project and build a Docker image with it. + +.. important:: + + You need to ensure Docker is using the right credentials to push images. For Artifact Registry, this can be done by running the following command: + + .. code-block:: bash + + gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://$REGION-docker.pkg.dev + + The token will be valid for 1 hour. After that, you need to create another one, if still needed. + +Clone the `cosmos-example `_ repo: + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Open `Dockerfile `_ located in ``gcp_cloud_run_job_example`` folder and change environments variables ``GCP_PROJECT_ID`` and ``GCP_REGION`` to your GCP project id and project region. + +Build a Docker image using previously modified ``Dockerfile``, which will be used by Cloud Run Job: + +.. code-block:: bash + + docker build -t $IMAGE_NAME -f gcp_cloud_run_job_example/Dockerfile.gcp_cloud_run_job . + +.. important:: + + Make sure to stay in ``cosmos-example`` directory when running ``docker build`` command. + +After this, the image needs to be pushed to the Artifact Registry: + +.. code-block:: bash + + docker push $IMAGE_NAME + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The `bigquery dbt profile `_ file is added to the image + - The dbt_project.yml is replaced with `bigquery_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Create Cloud Run Job instance** + +When the image is pushed to Artifact Registry, you can finally create Cloud Run Job with the image and previously created service account. + +First, enable Cloud Run Admin API using below command: + +.. code-block:: bash + + gcloud services enable run.googleapis.com + + +Next, set default Cloud Run region to your GCP region: + +.. code-block:: bash + + gcloud config set run/region $REGION + +Then, run below command to create Cloud Run Job instance: + +.. code-block:: bash + + gcloud run jobs create $CLOUD_RUN_JOB_NAME \ + --image=$IMAGE_NAME \ + --task-timeout=180s \ + --max-retries=0 \ + --cpu=1 \ + --memory=512Mi \ + --service-account=$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com + +**Setup Airflow Connections** + +Now, when you have the required Google Cloud infrastructure, you still need to check Airflow configuration to ensure the infrastructure can be used. You'll need a ``google_cloud_default`` connection in order to work on GCP resources. + +Check out an `example `_ of the ``airflow-settings.yml`` file. If you are using Astro CLI, filling in the right values here will be enough for this to work. + +**Setup and Trigger the DAG with Airflow** + +Open `jaffle_shop_gcp_cloud_run_job `_ DAG file and update ``GCP_PROJECT_ID`` and ``GCP_LOCATION`` constants with your GCP project id and project region. + +When the DAG is configured, copy the ``dags`` directory from ``cosmos-example`` repo to your Airflow home: + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow: + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_gcp_cloud_run_job `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_gcp_cloud_run_job.png + :width: 800 + + +You can also verify the tables that were created using dbt in BigQuery Studio: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_big_query.png + :width: 800 + + +**Delete resources** + +After the successful tests, don't forget to delete Google Cloud resources to save up costs: + +.. code-block:: bash + + # Delete Cloud Run Job instance + + gcloud run jobs delete $CLOUD_RUN_JOB_NAME + +.. code-block:: bash + + # Delete BigQuery main and custom dataset specified in dbt schema.yml with all tables included + + bq rm -r -f -d $PROJECT_ID:$DATASET_NAME + + bq rm -r -f -d $PROJECT_ID:dbt_dev + +.. code-block:: bash + + # Delete Artifact Registry repository with all images included + + gcloud artifacts repositories delete $REPO_NAME \ + --location=$REGION diff --git a/docs/configuration/run-dbt/container/index.rst b/docs/configuration/run-dbt/container/index.rst new file mode 100644 index 0000000000..634e9e8eb4 --- /dev/null +++ b/docs/configuration/run-dbt/container/index.rst @@ -0,0 +1,13 @@ +Run dbt in a container +====================== + +.. toctree:: + :maxdepth: 1 + :caption: Run dbt in a container + + aws-container-run-job + azure-container-instance + docker + gcp-cloud-run-job + kubernetes + watcher-kubernetes-execution-mode \ No newline at end of file diff --git a/docs/configuration/run-dbt/container/kubernetes.rst b/docs/configuration/run-dbt/container/kubernetes.rst new file mode 100644 index 0000000000..607ba07bd7 --- /dev/null +++ b/docs/configuration/run-dbt/container/kubernetes.rst @@ -0,0 +1,167 @@ +.. _kubernetes: + +Kubernetes Execution Mode +============================================== + +The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: + +- Postgres is run in the Kubernetes (K8s) cluster as a container +- Airflow is run locally, and it triggers a K8s Pod which runs dbt + +Requirements +++++++++++++ + +To test the DbtKubernetesOperators locally, we encourage you to install the following: + +- Local Airflow (either standalone or using Astro CLI) +- `Kind `_ to run K8s locally +- `Helm `_ to install Postgres in K8s +- `Docker `_ to create the dbt container image, which will allow Airflow to create a K8s pod which will run dbt + +At the moment, the user is expected to add to the Docker image both: + +- The dbt project files +- The dbt Profile, which contains the information for dbt to access the database while parsing the project from Apache Airflow nodes +- Handle secrets + +Additional KubernetesPodOperator parameters can be added to the ``operator_args`` parameter of the ``DbtKubernetesOperator``. + +For instance, + +.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py + :language: python + :start-after: [START kubernetes_tg_example] + :end-before: [END kubernetes_tg_example] + +Step-by-step instructions ++++++++++++++++++++++++++ + +Using installed `Kind `_, you can setup a local kubernetes cluster + +.. code-block:: bash + + kind create cluster + +Deploy a Postgres pod to Kind using `Helm `_ + +.. code-block:: bash + + helm repo add bitnami https://charts.bitnami.com/bitnami + helm repo update + helm install postgres bitnami/postgresql + +Retrieve the Postgres password and set it as an environment variable. + +.. code-block:: bash + + export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgres-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d) + +Check that the environment variable was set and that it is not empty + +.. code-block:: bash + + echo $POSTGRES_PASSWORD + +Expose the Postgres to the host running Docker/Kind. + +.. code-block:: bash + + kubectl port-forward --namespace default postgres-postgresql-0 5432:5432 + +Check that you're able to connect to the exposed pod. + +.. code-block:: bash + + PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432 + + postgres=# \dt + \q + +Create a K8s secret which contains the credentials to access Postgres. + +.. code-block:: bash + + kubectl create secret generic postgres-secrets --from-literal=host=postgres-postgresql.default.svc.cluster.local --from-literal=password=$POSTGRES_PASSWORD + +Clone the example repo that contains the Airflow DAG and dbt project files. + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example/ + +Create a Docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be run in K8s. + +.. code-block:: bash + + docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . + +.. note:: + + If running on M1, you may need to set the following environment variables to run the Docker build command in case it fails. + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a look at the Dockerfile to understand its purpose so that you can use it as a reference in your project. + + - The `dbt profile `__ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +Make the build image available in the Kind K8s cluster. + +.. code-block:: bash + + kind load docker-image dbt-jaffle-shop:1.0.0 + +Create a Python virtual environment and install the latest version of Astronomer Cosmos, which contains the K8s Operator. + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install "astronomer-cosmos[dbt-postgres]" apache-airflow-providers-cncf-kubernetes + +Make the `jaffle_shop_kubernetes.py `__ file at your Airflow DAG home: + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You may need to run Airflow standalone with ``sudo`` if your Airflow user is unable to access the Docker socket URL or pull images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_k8s `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_k8s_dag_run.png + :width: 800 + +.. _kubernetes-known-limitations: + +Known Limitations ++++++++++++++++++ + +The Kubernetes execution mode has the following limitations: + +- Does not emit OpenLineage events (there is an `open ticket #496 `__ to address this) +- Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) +- Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) +- Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) +- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) +- Does not expose Compiled SQL as a `templated field `_ +- Does not benefit from `Cosmos caching mechanisms `_ +- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst new file mode 100644 index 0000000000..16dbbffd0a --- /dev/null +++ b/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst @@ -0,0 +1,214 @@ +.. _watcher-kubernetes-execution-mode: + +``ExecutionMode.WATCHER_KUBERNETES``: High-Performance dbt Execution in Kubernetes +=================================================================================== + +.. versionadded:: 1.13.0 + +The ``ExecutionMode.WATCHER_KUBERNETES`` combines the **speed of the** :ref:`watcher-execution-mode` **with the isolation of** :ref:`kubernetes`. + +This execution mode is ideal for users who: + +* Want to leverage the performance benefits of the watcher execution mode +* Need to run dbt in isolated Kubernetes pods +* Prefer not to install dbt in their Airflow deployment + +------------------------------------------------------------------------------- + +Background +---------- + +The :ref:`watcher-execution-mode` introduced in Cosmos 1.11.0 significantly reduces dbt pipeline run times by running dbt as a single command while maintaining model-level observability in Airflow. + +However, the original ``ExecutionMode.WATCHER`` requires dbt to be installed alongside Airflow. The ``ExecutionMode.WATCHER_KUBERNETES`` removes this limitation by running the dbt command inside Kubernetes pods, similar to ``ExecutionMode.KUBERNETES``. + +For more details on the watcher concept and how it works, please refer to the :ref:`watcher-execution-mode` documentation. + +------------------------------------------------------------------------------- + +How to Use +---------- + +Users previously using ``ExecutionMode.KUBERNETES`` can simply replace the ``execution_mode`` to use ``ExecutionMode.WATCHER_KUBERNETES``. + +The following example shows how to configure a ``DbtDag`` with ``ExecutionMode.WATCHER_KUBERNETES``: + +.. code-block:: python + + from cosmos import DbtDag + from cosmos.config import ExecutionConfig + from cosmos.constants import ExecutionMode + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + "image": DBT_IMAGE, + "get_logs": True, + "log_events_on_failure": True, + }, + ) + +**Key differences from** ``ExecutionMode.KUBERNETES``: + +* The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` +* The producer task runs the entire ``dbt build`` command in a single Kubernetes pod +* Consumer tasks (sensors) watch for the completion of their corresponding dbt models + +For the complete setup including Kubernetes secrets, Docker image configuration, and profile setup, refer to the :ref:`kubernetes` documentation. + +------------------------------------------------------------------------------- + +Performance Gains +----------------- + +Early benchmarks using the ``jaffle_shop_watcher_kubernetes`` DAG show significant improvements: + ++-----------------------------------------------+------------------+ +| Execution Mode | Total Runtime | ++===============================================+==================+ +| ``ExecutionMode.KUBERNETES`` | 00:00:32.155 | ++-----------------------------------------------+------------------+ +| ``ExecutionMode.WATCHER_KUBERNETES`` | 00:00:11.783 | ++-----------------------------------------------+------------------+ + +This represents approximately a **63% reduction** in total DAG runtime. + +The performance improvement comes from: + +* Running dbt as a single command (reducing Kubernetes pod startup overhead) +* Leveraging dbt's native threading capabilities +* Eliminating repeated dbt initialization for each model + +------------------------------------------------------------------------------- + +Known Limitations +----------------- + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Kubernetes Provider Version Compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``ExecutionMode.WATCHER_KUBERNETES`` does not work with older versions of the ``apache-airflow-providers-cncf-kubernetes`` provider (<=10.7.0). + +Please ensure you have a compatible version installed: + +.. code-block:: bash + + pip install "apache-airflow-providers-cncf-kubernetes>10.7.0" + +We successfully tested against the most recent release of the provider (`10.12.2 `_). + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Support for KPO deferrable mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The producer node created by the ``ExecutionMode.WATCHER_KUBERNETES`` producer task can be set to deferrable mode as long as: + +- The correct version of Airflow Kubernetes is installed (``>=10.12.2``). This version fixed a bug (`PR `_) that prevented setting callbacks and parsing the logs when the Kubernetes Operator run using ``deferrable``. The experience should be further improved once `this other PR is merged `_. + +.. code-block:: bash + + pip install "apache-airflow-providers-cncf-kubernetes>=10.12.2" + +- The arguments ``deferrable=True`` and ``is_delete_operator_pod=True`` are set: + +.. code-block:: python + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + "deferrable": True, + "is_delete_operator_pod": True, + "image": DBT_IMAGE, + "get_logs": True, + "log_events_on_failure": True, + }, + ) + +Conversely, the consumer tasks that subclass ``DbtConsumerWatcherKubernetesSensor`` run in deferrable mode by default when operating as a sensor. They can also operate in deferrable mode if they are running dbt themselves upon retry. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Mandatory ``operator_args`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``operator_args`` must define ``get_logs`` and ``log_events_on_failure``: + +.. code-block: python + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + # ... other KPO mandatory args ... + "get_logs": True, + "log_events_on_failure": True, + }, + ) + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Other Inherited Limitations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following limitations from ``ExecutionMode.WATCHER`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``: + +* **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. + +* **Test behavior**: The ``TestBehavior.AFTER_EACH`` is not supported. Tests are run as part of the ``dbt build`` command by the producer task. + +* **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. + +For more details on these limitations, refer to the :ref:`watcher-execution-mode` documentation. + +Additionally, the limitations from ``ExecutionMode.KUBERNETES`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``. For details, refer to the :ref:`kubernetes-known-limitations` documentation. + +------------------------------------------------------------------------------- + +Example DAG +----------- + +Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: + +.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py + :language: python + +------------------------------------------------------------------------------- + +Prerequisites +------------- + +Before using ``ExecutionMode.WATCHER_KUBERNETES``, ensure you have: + +1. A Kubernetes cluster configured and accessible from your Airflow deployment +2. A Docker image containing your dbt project and profile +3. The ``apache-airflow-providers-cncf-kubernetes`` provider installed (version >10.7.0) + +For detailed setup instructions, refer to the :ref:`kubernetes` documentation. + +------------------------------------------------------------------------------- + +Summary +------- + +``ExecutionMode.WATCHER_KUBERNETES`` provides: + +* ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` +* ✅ **Isolation** between dbt and Airflow dependencies +* ✅ **Model-level visibility** in Airflow +* ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` + +This execution mode is ideal for teams who want the performance benefits of the watcher mode while maintaining the isolation provided by Kubernetes execution. diff --git a/docs/configuration/run-dbt/execution-modes.rst b/docs/configuration/run-dbt/execution-modes.rst new file mode 100644 index 0000000000..a9bd3f1e2b --- /dev/null +++ b/docs/configuration/run-dbt/execution-modes.rst @@ -0,0 +1,387 @@ +.. _execution-modes: + +Execution Modes +=============== + +.. toctree:: + :maxdepth: 3 + :caption: Run dbt in the Airflow worker + + airflow-worker/index + +.. toctree:: + :maxdepth: 3 + :caption: Run dbt in a container + + container/index + + +Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: + +1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) +2. **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos +3. **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) +4. **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) +5. **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) +6. **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) +7. **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) +8. **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) +9. **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ +10. **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. +11. **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. + +The choice of the ``execution mode`` can vary based on each user's needs and concerns. For more details, check each execution mode described below. + +.. _execution-modes-comparison: + +.. list-table:: Execution Modes Comparison + :widths: 25 25 25 25 + :header-rows: 1 + + * - Execution Mode + - Task Duration + - Environment Isolation + - Cosmos Profile Management + * - Local + - Fast + - None + - Yes + * - Virtualenv + - Medium + - Lightweight + - Yes + * - Docker + - Slow + - Medium + - No + * - Kubernetes + - Slow + - High + - No + * - AWS_EKS + - Slow + - High + - No + * - Azure Container Instance + - Slow + - High + - No + * - GCP Cloud Run Job Instance + - Slow + - High + - No + * - AWS ECS + - Slow + - High + - No + * - Airflow Async + - Very Fast + - Medium + - Yes + * - Watcher + - Very Fast + - None + - Yes + * - Watcher Kubernetes + - Fast + - High + - No + +Local +----- + +By default, Cosmos uses the ``local`` execution mode. + +The ``local`` execution mode is the fastest way to run Cosmos operators since they don't install ``dbt`` nor build docker containers. However, it may not be an option for users using managed Airflow services such as +Google Cloud Composer, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`), the user may not be able to install ``dbt`` in a custom path. + +The ``local`` execution mode assumes a ``dbt`` binary is reachable within the Airflow worker node. + +If ``dbt`` was not installed as part of the Cosmos packages, +users can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. + +.. note:: + Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. + This feature is bound to `dbt partial parsing limitations `_. + Learn more: :ref:`partial-parsing`. + +When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). + +Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: + +.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py + :language: python + :start-after: [START local_example] + :end-before: [END local_example] + + +Virtualenv +---------- + +If you're using managed Airflow on GCP (Cloud Composer), for instance, we recommend you use the ``virtualenv`` execution mode. + +The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. + +In this case, users are responsible for declaring which version of ``dbt`` they want to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. + +Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). +Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. + +Some drawbacks of this approach: + +- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. +- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. +- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. + +Example of how to use: + +.. literalinclude:: ../../dev/dags/example_virtualenv.py + :language: python + :start-after: [START virtualenv_example] + :end-before: [END virtualenv_example] + +Docker +------ + +The ``docker`` approach assumes users have a previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml``, managed by the user. + +The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). + +The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. + +This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. +If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. + +Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. + +Example DAG: + +.. code-block:: python + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.DOCKER, + ), + operator_args={ + "image": "dbt-jaffle-shop:1.0.0", + "network_mode": "bridge", + }, + ) + + +Kubernetes +---------- + +The ``kubernetes`` approach is a very isolated way of running ``dbt`` since the ``dbt`` run commands from within a Kubernetes Pod, usually in a separate host. + +It assumes the user has a Kubernetes cluster. It also expects the user to ensure the Docker container has up-to-date ``dbt`` pipelines and profiles, potentially leading the user to declare secrets in two places (Airflow and Docker container). + +The ``Kubernetes`` deployment may be slower than ``Docker`` and ``Virtualenv`` assuming that the container image is built (which is slower than creating a Python ``virtualenv`` and installing ``dbt-core``) and the Airflow task needs to spin up a new ``Pod`` in Kubernetes. + +Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref:`kubernetes`. + +Example DAG: + +.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py + :language: python + :start-after: [START kubernetes_seed_example] + :end-before: [END kubernetes_seed_example] + +AWS_EKS +---------- + +The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. +It uses the `EKSPodOperator `_ +to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. + + +Example DAG: + +.. code-block:: python + + postgres_password_secret = Secret( + deploy_type="env", + deploy_target="POSTGRES_PASSWORD", + secret="postgres-secrets", + key="password", + ) + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AWS_EKS, + ), + operator_args={ + "image": "dbt-jaffle-shop:1.0.0", + "cluster_name": CLUSTER_NAME, + "get_logs": True, + "is_delete_operator_pod": False, + "secrets": [postgres_password_secret], + }, + ) + +Azure Container Instance +------------------------ +.. versionadded:: 1.4 + +Similar to the ``kubernetes`` approach, using ``Azure Container Instances`` as the execution mode gives a very isolated way of running ``dbt``, since the ``dbt`` run itself is run within a container running in an Azure Container Instance. + +This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. Please checkout the step-by-step guide for using Azure Container Instance as the execution mode + + +.. code-block:: python + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AZURE_CONTAINER_INSTANCE + ), + operator_args={ + "ci_conn_id": "aci", + "registry_conn_id": "acr", + "resource_group": "my-rg", + "name": "my-aci-{{ ti.task_id.replace('.','-').replace('_','-') }}", + "region": "West Europe", + "image": "dbt-jaffle-shop:1.0.0", + }, + ) + +GCP Cloud Run Job +------------------------ +.. versionadded:: 1.7 + +The ``gcp_cloud_run_job`` execution mode is particularly useful for users who prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. + +For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . + +This execution mode allows users to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. + + +.. code-block:: python + + gcp_cloud_run_job_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig(execution_mode=ExecutionMode.GCP_CLOUD_RUN_JOB), + operator_args={ + "project_id": "my-gcp-project-id", + "region": "europe-west1", + "job_name": "my-crj-{{ ti.task_id.replace('.','-').replace('_','-') }}", + }, + ) + + +AWS ECS +--------- +.. versionadded:: 1.9.0 + +Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. + +This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. For users who require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. + +Please refer to the step-by-step guide for using AWS ECS as the execution mode. + +.. code-block:: python + + aws_ecs_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig(execution_mode=ExecutionMode.AWS_ECS), + operator_args={ + "aws_conn_id": "aws_default", + "cluster": "my-ecs-cluster", + "task_definition": "my-dbt-task", + "container_name": "dbt-container", + "launch_type": "FARGATE", + "deferrable": True, + "network_configuration": { + "awsvpcConfiguration": { + "subnets": ["<<>>"], + "assignPublicIp": "ENABLED", + }, + }, + "environment_variables": {"DBT_PROFILE_NAME": "default"}, + }, + ) + +.. _airflow-async-execution-mode: + +Airflow Async +------------- + +.. versionadded:: 1.9.0 + +Although this execution mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. +In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. + +The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's +`Deferrable operators `__. +This execution mode could be preferred when you've long running resources and you want to run them asynchronously by +leveraging Airflow's deferrable operators. With that, you would be able to potentially observe higher throughput of tasks +as more dbt nodes will be run in parallel since they won't be blocking Airflow's worker slots. + +Example DAG: + +.. literalinclude:: ../../dev/dags/simple_dag_async.py + :language: python + :start-after: [START airflow_async_execution_mode_example] + :end-before: [END airflow_async_execution_mode_example] + +For a full step-by-step guide and limitations, check the :ref:`async-execution-mode` page. + + +Watcher Execution Mode (Experimental) +------------------------------------- + +.. versionadded:: 1.11.0 + +The ``watcher`` execution mode is an experimental execution mode that runs a single ``dbt build`` command from a producer task and has sensor tasks to watch the progress of the producer. +It is designed to improve DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. + +Check the :ref:`watcher-execution-mode` for more details. + + +Watcher Kubernetes Execution Mode (Experimental) +------------------------------------------------ + +.. versionadded:: 1.13.0 + +The ``watcher_kubernetes`` execution mode combines the speed of the ``watcher`` execution mode with the isolation of the ``kubernetes`` execution mode. It runs a single ``dbt build`` command from a producer task inside a Kubernetes pod and has sensor tasks to watch the progress of the producer. + +Check the :ref:`watcher-kubernetes-execution-mode` for more details. + + +.. _invocation_modes: + +Invocation Modes +================ +.. versionadded:: 1.4 + +For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: + +1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. + +2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ + In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ + This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. + +The invocation mode can be set in the ``ExecutionConfig`` as shown below: + +.. code-block:: python + + from cosmos.constants import InvocationMode + + dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.LOCAL, + invocation_mode=InvocationMode.DBT_RUNNER, + ), + ) + +If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. From 6ca9d1f1bcc6054a0644d162cd5cedd45ae0b63d Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:53:56 -0500 Subject: [PATCH 02/48] move execution methods --- docs/getting_started/astro.rst | 4 +- docs/getting_started/async-execution-mode.rst | 249 --------- .../getting_started/aws-container-run-job.rst | 193 ------- .../azure-container-instance.rst | 138 ----- docs/getting_started/dbt-airflow-concepts.rst | 4 +- docs/getting_started/docker.rst | 111 ---- .../execution-modes-local-conflicts.rst | 133 ----- docs/getting_started/execution-modes.rst | 374 -------------- docs/getting_started/gcp-cloud-run-job.rst | 265 ---------- docs/getting_started/index.rst | 44 +- docs/getting_started/kubernetes.rst | 167 ------ docs/getting_started/mwaa.rst | 4 +- docs/getting_started/open-source.rst | 4 +- .../watcher-execution-mode.rst | 480 ------------------ .../watcher-kubernetes-execution-mode.rst | 214 -------- 15 files changed, 32 insertions(+), 2352 deletions(-) delete mode 100644 docs/getting_started/async-execution-mode.rst delete mode 100644 docs/getting_started/aws-container-run-job.rst delete mode 100644 docs/getting_started/azure-container-instance.rst delete mode 100644 docs/getting_started/docker.rst delete mode 100644 docs/getting_started/execution-modes-local-conflicts.rst delete mode 100644 docs/getting_started/execution-modes.rst delete mode 100644 docs/getting_started/gcp-cloud-run-job.rst delete mode 100644 docs/getting_started/kubernetes.rst delete mode 100644 docs/getting_started/watcher-execution-mode.rst delete mode 100644 docs/getting_started/watcher-kubernetes-execution-mode.rst diff --git a/docs/getting_started/astro.rst b/docs/getting_started/astro.rst index b590575f2e..56e9fa0d53 100644 --- a/docs/getting_started/astro.rst +++ b/docs/getting_started/astro.rst @@ -1,7 +1,7 @@ .. _astro: -Getting Started on Astro -======================== +Getting Started with Cosmos on Astro +==================================== While it is possible to use Cosmos on Astro with all :ref:`Execution Modes `, we recommend using the ``local`` execution mode. It's the simplest to set up and use. diff --git a/docs/getting_started/async-execution-mode.rst b/docs/getting_started/async-execution-mode.rst deleted file mode 100644 index 6d61bcf22b..0000000000 --- a/docs/getting_started/async-execution-mode.rst +++ /dev/null @@ -1,249 +0,0 @@ -.. _async-execution-mode: - -.. title:: Getting Started with Deferrable Operator - -Airflow Async Execution Mode -============================ - -This execution mode can reduce the runtime by 35% in comparison to Cosmos LOCAL execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. - -It can be particularly useful for long-running transformations, since it leverages Airflow's `deferrable operators `__. - -In this mode, there is a ``SetupAsyncOperator`` that will pre-generate the SQL files for the dbt project and upload them to Airflow XCom or a remote location. A remote location will only be used if users set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator is run before the remaining pipeline. -All the pipeline dbt model transformations will be run using ``DbtRunAirflowAsyncOperator`` which, instead of running the ``dbt run`` command for each model. They will download the SQL files from the Airflow XCom or remote location and execute them directly leveraging the Airflow ``BigQueryInsertJobOperator``. - -Users can leverage other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. - - -Advantages of Airflow Async Mode -++++++++++++++++++++++++++++++++ - -- **Improved Task Throughput:** Async tasks free up Airflow workers by leveraging the Airflow Trigger framework. While long-running SQL transformations are executing in the data warehouse, the worker is released and can handle other tasks, increasing overall task throughput. -- **Better Resource Utilization:** By minimizing idle time on Airflow workers, async tasks allow more efficient use of compute resources. Workers aren't blocked waiting for external systems and can be reused for other work while waiting on async operations. -- **Faster Task Execution:** With Cosmos ``SetupAsyncOperator``, the SQL transformations are precompiled and uploaded to XCom (default behaviour) or a remote location. Instead of invoking a full dbt run during each dbt model task, the SQL files are downloaded from this XCom or remote path and executed directly. This eliminates unnecessary overhead from running the full dbt command, resulting in faster and more efficient task execution. - -We have `observed `_ the following performance improvements by running a dbt project with 129 models: - -+----------------------------------------------+--------------------------+ -| How the dbt pipeline was executed | Execution Time (seconds) | -+==============================================+==========================+ -| ``dbt run`` with dbt Core 1.10 | 13 | -+----------------------------------------------+--------------------------+ -| Cosmos 1.11 with ExecutionMode.LOCAL | 11 | -+----------------------------------------------+--------------------------+ -| Cosmos 1.11 with ExecutionMode.AIRFLOW_ASYNC | 7 | -+----------------------------------------------+--------------------------+ - - -Getting Started with Airflow Async Mode -+++++++++++++++++++++++++++++++++++++++ - -This guide walks you through setting up an Astro CLI project and running a Cosmos-based DAG with a deferrable operator, enabling asynchronous task execution in Apache Airflow. - -Prerequisites -+++++++++++++ - -- `Astro CLI `_ -- Airflow>=2.9 - -1. Create Astro-CLI Project -+++++++++++++++++++++++++++ - -Run the following command in your terminal: - -.. code-block:: bash - - astro dev init - -This will create an Astro project with the following structure: - -.. code-block:: bash - - . - ├── Dockerfile - ├── README.md - ├── airflow_settings.yaml - ├── dags/ - ├── include/ - ├── packages.txt - ├── plugins/ - ├── requirements.txt - └── tests/ - - -2. Update Dockerfile -++++++++++++++++++++ - -Edit your Dockerfile to ensure all necessary requirements are included. - -.. code-block:: bash - - FROM astrocrpublic.azurecr.io/runtime:3.0-2 - - -3. Add astronomer-cosmos Dependency -+++++++++++++++++++++++++++++++++++ - -In your ``requirements.txt``, add: - -.. code-block:: bash - - astronomer-cosmos[dbt-bigquery, google]>=1.9 - - -4. Create Airflow DAG -+++++++++++++++++++++ - -1. Create a new DAG file: ``dags/cosmos_async_dag.py`` - -- Update the ``dataset`` and ``project`` - -.. code-block:: python - - import os - from datetime import datetime - from pathlib import Path - - from cosmos import ( - DbtDag, - ExecutionConfig, - ExecutionMode, - ProfileConfig, - ProjectConfig, - ) - from cosmos.constants import TestBehavior - from cosmos.profiles import GoogleCloudServiceAccountDictProfileMapping - - DEFAULT_DBT_ROOT_PATH = Path(__file__).resolve().parent / "dbt" - DBT_ROOT_PATH = Path(os.getenv("DBT_ROOT_PATH", DEFAULT_DBT_ROOT_PATH)) - DBT_ADAPTER_VERSION = os.getenv("DBT_ADAPTER_VERSION", "1.9") - - cosmos_async_dag = DbtDag( - project_config=ProjectConfig( - DBT_ROOT_PATH / "jaffle_shop", - ), - profile_config=ProfileConfig( - profile_name="default", - target_name="dev", - profile_mapping=GoogleCloudServiceAccountDictProfileMapping( - conn_id="gcp_conn", - profile_args={ - "dataset": "cosmos_async_demo", - "project": "astronomer-**", - }, - ), - ), - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AIRFLOW_ASYNC, - async_py_requirements=[f"dbt-bigquery=={DBT_ADAPTER_VERSION}"], - ), - schedule=None, - start_date=datetime(2025, 1, 1), - catchup=False, - dag_id="cosmos_async_dag", - operator_args={ - "location": "US", - "install_deps": True, - "full_refresh": True, - "virtualenv_dir": "dbt_venv", - }, - ) - -2. Folder structure for dbt project - -- Add a valid dbt project inside your Airflow project under ``dags/dbt/``. - - -5. Start the Project -++++++++++++++++++++ - -Launch the Airflow project locally: - -.. code-block:: bash - - astro dev start - -This will: - -- Spin up the scheduler, webserver, and triggerer (needed for deferrable operators) -- Expose Airflow UI at http://localhost:8080 - -6. Create Airflow Connection -++++++++++++++++++++++++++++ - -Create an Airflow connection with following configurations - -- Connection ID: gcp_conn -- Connection Type: google_cloud_platform -- Extra Fields JSON: - -.. code-block:: bash - - { - "project": "astronomer-**", - "keyfile_dict": { - "type": "***", - "project_id": "***", - "private_key_id": "***", - "private_key": "***", - "client_email": "***", - "client_id": "***", - "auth_uri": "***", - "token_uri": "***", - "auth_provider_x509_cert_url": "***", - "client_x509_cert_url": "***", - "universe_domain": "***" - } - } - - -7. Execute the DAG -++++++++++++++++++ - -1. Visit the Airflow UI at ``http://localhost:8080`` -2. Enable the DAG: ``cosmos_async_dag`` -3. Trigger the DAG manually - -.. image:: /_static/jaffle_shop_async_execution_mode.png - :alt: Cosmos dbt Async DAG - :align: center - -The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks. - - -Control of where to upload the SQL files -++++++++++++++++++++++++++++++++++++++++ - -For optimal performance we encourage to keep Cosmos standard behaviour (introduced in 1.11), which is to upload the SQL files to XCom, instead of a remote object location. - -For the benchmark example described in a previous section, there was an overhead of ~500 seconds with remote SQL file upload/download, but only ~2 seconds using XCom, which can outweigh the performance improvements introduced by using deferrable operators. - -However, if you want to upload the SQL files to a remote object location instead of XCom, you can set the following environment variables: - -.. code-block:: bash - - AIRFLOW__COSMOS__REMOTE_TARGET_PATH=gs://cosmos_remote_target_demo - AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID=gcp_conn - - -Limitations -+++++++++++ - - -1. **Limited to dbt models**: Only dbt resource type models are run asynchronously using Airflow deferrable operators. Other resource types are executed synchronously, similar to the local execution mode. - -2. **BigQuery support only**: This mode only supports BigQuery as the target database. If a different target is specified, Cosmos will throw an error indicating the target database is unsupported in this mode. Adding support for other adapters is on the roadmap. - -3. **ProfileMapping parameter required**: You need to specify the ``ProfileMapping`` parameter in the ``ProfileConfig`` for your DAG. Refer to the example DAG below for details on setting this parameter. - -4. **Location parameter required**: You must specify the location of the BigQuery dataset in the ``operator_args`` of the ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. - -5. **async_py_requirements parameter required**: If you're using the default approach of having a setup task, you must specify the necessary dbt adapter Python requirements based on your profile type for the async execution mode in the ``ExecutionConfig`` of your ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. - -6. **Creation of new isolated virtual environment for each task run**: By default, the ``SetupAsyncOperator`` creates and executes within a new isolated virtual environment for each task run, which can cause performance issues. To reuse an existing virtual environment, use the ``virtualenv_dir`` parameter within the ``operator_args`` of the ``DbtDag``. We have observed that for ``dbt-bigquery``, the ``SetupAsyncOperator`` executes approximately 30% faster when reusing an existing virtual environment, particularly for transformations that take around 10 minutes to complete. - -7. **Performance degradation when uploading to remote object location**: Even though it is possible to upload the SQL files to a remote object location by setting environment variables, it is slow. We observed that this introduces a significant overhead in the execution time (500s for 129 models). - -8. **TeardownAsyncOperator limitation**: When using a remote object location, in addition to the ``SetupAsyncOperator``, a ``TeardownAsyncOperator`` is also added to the DAG. This task will delete the SQL files from the remote location by the end of the DAG Run. This is can lead to a limitation from a retry perspective, as described in the issue `#2066 `_. This can be avoided by setting the ``enable_teardown_async_task`` configuration to ``False``, as described in the :ref:`enable_teardown_async_task` section. - -For a comparison between different Cosmos execution modes, please, check the :ref:`execution-modes-comparison` section. diff --git a/docs/getting_started/aws-container-run-job.rst b/docs/getting_started/aws-container-run-job.rst deleted file mode 100644 index db00fc8c3c..0000000000 --- a/docs/getting_started/aws-container-run-job.rst +++ /dev/null @@ -1,193 +0,0 @@ -.. _aws-container-run-job: - -.. title:: Getting Started with Astronomer Cosmos on AWS ECS - -Getting Started with Astronomer Cosmos on AWS ECS -================================================== - -Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. -Schematically, the guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aws_ecs_schematic.png - :width: 800 - -Prerequisites -+++++++++++++ - -Before you begin, ensure you have the following: - -- An active **AWS Account** with permissions to create ECS clusters, register task definitions, and run tasks. -- The **AWS CLI** installed and configured with the proper credentials. -- **Docker** installed for building your container image. -- Access to your container registry (for example, **Amazon ECR**) where your job image is stored. -- Basic familiarity with AWS ECS concepts (clusters, task definitions, services, and Fargate). -- An existing installation of **Astronomer Cosmos** (refer to the `Cosmos documentation `_ for more details). - - - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: - -.. code-block:: bash - - python3 -m venv venv - source venv/bin/activate - python3 -m pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[amazon]" - pip install "aiobotocore[boto3]" -.. note:: - The package aiobotocore[boto3] is optional; you will need it if you plan to use **deferred tasks**. - -**Set up your ECR** - -1. **Set your secrets** - On the `cosmos-examples `_ repository, you can find a ready-to-use Docker image for the AWS ECS service. Just replace your secrets, or you can create your own. - -2. **AWS CLI login** - Before building and pushing your image, you first need to log in to the AWS service using the AWS CLI tool. - Use the following command: - - .. code-block:: bash - - aws ecr-public get-login-password --region | docker login --username AWS --password-stdin - -3. **Build and tag your image** - Once you have your image ready, run the following commands: - - .. code-block:: bash - - docker build -f Dockerfile.aws_ecs . --platform=linux/amd64 -t - docker tag - -4. **Push your image** - - .. code-block:: bash - - docker push - -**Configure Your AWS Environment** - -1. **Create an ECS Cluster** - - Create an ECS cluster to host your Cosmos jobs. You can do this from the AWS Console or using the AWS CLI: - - .. code-block:: bash - - aws ecs create-cluster --cluster-name my-cosmos-cluster - -2. **Set Up an IAM Role for ECS Tasks** - - Ensure you have an IAM role that your ECS tasks can assume. This role should include permissions for ECS, ECR, and CloudWatch (for logs). For example, you might create a role named ``ecsTaskExecutionRole`` with the managed policies: - - - ``AmazonECSTaskExecutionRolePolicy`` - - (Optional) Additional policies for custom resource access - -3. **Configure Networking** - - For Fargate tasks, make sure you have at least one subnet (preferably in multiple Availability Zones) and a security group that permits outbound internet access if needed. Note the subnet IDs for later use. - -**Prepare Your Cosmos Job Definition** - -Cosmos jobs are defined as container tasks. Create a task definition file (e.g., ``cosmos-task-definition.json``) with the configuration for your job. - -For example: - -.. code-block:: json - - { - "family": "cosmos-job", - "networkMode": "awsvpc", - "requiresCompatibilities": [ - "FARGATE" - ], - "cpu": "512", - "memory": "1024", - "executionRoleArn": "arn:aws:iam:::role/ecsTaskExecutionRole", - "containerDefinitions": [ - { - "name": "cosmos-job", - "image": "/your_image:latest", - "essential": true, - "environment": [ - { "name": "VAR1", "value": "value1" }, - { "name": "VAR2", "value": "value2" } - ], - "logConfiguration": { - "logDriver": "awslogs", - "options": { - "awslogs-group": "/ecs/cosmos-job", - "awslogs-region": "us-east-1", - "awslogs-stream-prefix": "ecs" - } - } - } - ] - } - -.. note:: - - Replace ````, ````, and adjust the CPU, memory, and environment variables as needed. - -**Deploy Your Cosmos Job on AWS ECS** - -1. **Register the Task Definition** - - Use the AWS CLI to register your task definition: - - .. code-block:: bash - - aws ecs register-task-definition --cli-input-json file://cosmos-task-definition.json - -2. **Run the Task** - - Run a test task on your ECS cluster. Specify the subnets and security groups in your network configuration. For example: - - .. code-block:: bash - - aws ecs run-task \ - --cluster my-cosmos-cluster \ - --launch-type FARGATE \ - --task-definition cosmos-job \ - --network-configuration "awsvpcConfiguration={subnets=[subnet-12345678,subnet-87654321],securityGroups=[sg-abcdef12],assignPublicIp=ENABLED}" - - Once the test is ok, we are able to run the dbt commands in our Cosmos DAG: - - .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run.png - :width: 800 - - .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run_logs.png - :width: 800 - - Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! - - -**Monitor and Debug Your Job** - -1. **Check Task Status** - - You can view the status of your task from the AWS Console under your ECS cluster or via the CLI: - - .. code-block:: bash - - aws ecs describe-tasks --cluster my-cosmos-cluster --tasks - -2. **View Logs** - - Since the task definition configures AWS CloudWatch Logs, you can view your job’s output in the CloudWatch Logs console. Look for log streams with the prefix you set (e.g., ``ecs/cosmos-job``). - -**Conclusion** - - -By following this guide, you can deploy Astronomer Cosmos jobs on AWS ECS using Fargate. This integration enables you to leverage the scalability and managed infrastructure of ECS while maintaining a consistent container orchestration experience with Cosmos. - -For more detailed information on AWS ECS, please refer to the `AWS ECS Developer Guide `_. - -Happy deploying! :rocket: - - -Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! diff --git a/docs/getting_started/azure-container-instance.rst b/docs/getting_started/azure-container-instance.rst deleted file mode 100644 index 86ce3ab9ef..0000000000 --- a/docs/getting_started/azure-container-instance.rst +++ /dev/null @@ -1,138 +0,0 @@ -.. _azure-container-instance: - -Azure Container Instance Execution Mode -======================================= -.. versionadded:: 1.4 - -This tutorial will guide you through the steps required to use Azure Container Instance as the Execution Mode for your dbt code with Astronomer Cosmos. Schematically, the guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aci_schematic.png - :width: 800 - -Prerequisites -+++++++++++++ -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Azure CLI (install guide here: `Azure CLI `_) -4. Astronomer-cosmos package containing the dbt Azure Container Instance operators -5. Azure account with: - 1. A resource group - 2. A service principal with `Contributor` permissions on the resource group - 3. A Container Registry - 4. A Postgres instance accessible from Azure. (we use an Azure Postgres instance in the example) -6. Docker image built with required dbt project and dbt DAG -7. dbt DAG with dbt Azure Container Instance operators in the Airflow DAGs directory to run in Airflow - -More information on how to achieve 2-6 is detailed below. - -Note that the steps below will walk you through an example, for which the code can be found HERE - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install apache airflow & astronomer-postgres - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-postgres,azure-container-instance]" - -**Setup Postgres database** - -You will need a postgres database running to be used as the database for the dbt project. In order to have it accessible from Azure Container Instance, the easiest way is to create an Azure Postgres instance. For this, run the following (assuming you are logged into your Azure account) - -.. code-block:: bash - - az postgres server create -l westeurope -g <<>> -n <<>> -u dbadmin -p <<>> --sku-name B_Gen5_1 --ssl-enforcement Enabled - - -**Setup Azure Container Registry** -In order to run a container in Azure Container Instance, it needs access to the container image. In our setup, we will use Azure Container Registry for this. To set an Azure Container Registry up, you can use the following bash command: - -.. code-block:: bash - - az acr create --name <<>> --resource-group <<>> --sku Basic --admin-enabled - -**Build the dbt Docker image** - -For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. - -Clone the `cosmos-example `_ repo - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. - -.. code-block:: bash - - docker build -t <<>:1.0.0 -f Dockerfile.azure_container_instance . - -After this, the image needs to be pushed to the registry of your choice. Note that your image name should contain the name of your registry: -.. code-block:: bash - - docker push <<>>:1.0.0 - -.. note:: - - You may need to ensure docker knows to use the right credentials. If using Azure Container Registry, this can be done by running the following command: - .. code-block:: bash - - az acr login - -.. note:: - - If running on M1, you may need to set the following envvars for running the docker build command in case it fails - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The `dbt profile `_ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Setup Airflow Connections** -Now you have the required Azure infrastructure, you still need to add configuration to Airflow to ensure the infrastructure can be used. You'll need 3 connections: - -1. ``aci_db``: a Postgres connection to your Azure Postgres instance. -2. ``aci``: an Azure Container Instance connection configured with a Service Principal with sufficient permissions (i.e. ``Contributor`` on the resource group in which you will use Azure Container Instances). -3. ``acr``: an Azure Container Registry connection configured for your Azure Container Registry. - -Check out the ``airflow-settings.yml`` file `here `_ for an example. If you are using Astro CLI, filling in the right values here will be enough for this to work. - -**Setup and Trigger the DAG with Airflow** - -Copy the dags directory from cosmos-example repo to your Airflow home - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_azure_container_instance `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_azure_container_instance.png - :width: 800 diff --git a/docs/getting_started/dbt-airflow-concepts.rst b/docs/getting_started/dbt-airflow-concepts.rst index 70c4feae8d..ee55abe694 100644 --- a/docs/getting_started/dbt-airflow-concepts.rst +++ b/docs/getting_started/dbt-airflow-concepts.rst @@ -1,7 +1,7 @@ .. _dbt-airflow-concepts: -Similar dbt & Airflow concepts -============================== +Similar dbt and Airflow concepts +================================ While dbt is an open source tool for data transformations and analysis, using SQL, Airflow focuses on being a platform for the development, scheduling and monitoring of batch-oriented workflows, using Python. Although both tools have many diff --git a/docs/getting_started/docker.rst b/docs/getting_started/docker.rst deleted file mode 100644 index 0005914886..0000000000 --- a/docs/getting_started/docker.rst +++ /dev/null @@ -1,111 +0,0 @@ -.. _docker: - -Docker Execution Mode -======================================== - -The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. - -Requirements -++++++++++++ - -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Astronomer-cosmos package containing the dbt Docker operators -4. Postgres docker container -5. Docker image built with required dbt project and dbt DAG -6. dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow - -More information on how to achieve 2-6 is detailed below. - -Step-by-step instructions -+++++++++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install `Apache Airflow® `_ & astronomer-postgres - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-postgres]" - -**Setup Postgres database** - -You will need a postgres database running to be used as the database for the dbt project. Run the following command to run and expose a postgres database - -.. code-block:: bash - - docker run --name some-postgres -e POSTGRES_PASSWORD="" -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p5432:5432 -d postgres - -**Build the dbt Docker image** - -For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. - -Clone the `cosmos-example `_ repo - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. - -.. code-block:: bash - - docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . - -.. note:: - - If running on M1, you may need to set the following envvars for running the docker build command in case it fails - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The `dbt profile `_ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Setup and Trigger the DAG with Airflow** - -Copy the dags directory from cosmos-example repo to your Airflow home - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_docker `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_docker_dag_run.png - :width: 800 - - -Specifying ProfileConfig -+++++++++++++++++++++++++ - -Starting with Cosmos 1.8.0, you can use the ``profile_config`` argument in your Dbt DAG Docker operators to reference -profiles for your dbt project defined in a profiles.yml file. To do so, provide the file’s path via the -``profiles_yml_path`` parameter in ``profile_config``. - -Note that in ``ExecutionMode.DOCKER``, the ``profile_config`` is only compatible with the ``profiles_yml_path`` -approach. The ``profile_mapping`` method will not work because the required Airflow connections cannot be accessed -within the Docker container to map them to the dbt profile. diff --git a/docs/getting_started/execution-modes-local-conflicts.rst b/docs/getting_started/execution-modes-local-conflicts.rst deleted file mode 100644 index 9fec173751..0000000000 --- a/docs/getting_started/execution-modes-local-conflicts.rst +++ /dev/null @@ -1,133 +0,0 @@ -:orphan: - -.. _execution-modes-local-conflicts: - -Airflow and dbt dependencies conflicts -====================================== - -When using the `Local Execution Mode `__, users may face dependency conflicts between -`Apache Airflow® `_ and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used. - -If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. -With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. - -The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. - -In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): - -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| Airflow / DBT | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | -+===============+=====+=====+=====+=====+=====+=====+=====+=====+=====+=====+======+ -| 2.2 | | | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.3 | x | x | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.4 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.5 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.6 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.7 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.8 | x | x | x | x | x | | x | | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.9 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.10 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.11 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 3.0 | x | x | x | x | x | x | x | x | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ - -Examples of errors ------------------------------------ - -.. code-block:: bash - - The conflict is caused by: - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10 - - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.2.4 depends on jinja2<3.1 and >=2.10.1 - dbt-core 1.5.0 depends on Jinja2==3.1.2 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.6.0 and dbt-core because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.6.0 depends on importlib-metadata<5.0.0 and >=1.7; python_version < "3.9" - dbt-semantic-interfaces 0.1.0.dev7 depends on importlib-metadata==6.6.0 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow, apache-airflow==2.7.0 and dbt-core==1.4.0 because these package versions have conflicting dependencies. - - The conflict is caused by: - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.12.0 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.2 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.1 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.0 depends on PyYAML<6 and >=5.1 - apache-airflow 2.7.0 depends on jsonschema>=4.18.0 - flask-appbuilder 4.3.3 depends on jsonschema<5 and >=3 - connexion 2.10.0 depends on jsonschema<4 and >=2.5.1 - -.. code-block:: bash - -ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. - -The conflict is caused by: - dbt-core 1.10.0 depends on pydantic<2 - apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 - - - -How to reproduce ----------------- - -The table was created by running `nox `__ with the following ``noxfile.py``: - -.. code-block:: python - - import nox - - nox.options.sessions = ["compatibility"] - nox.options.reuse_existing_virtualenvs = True - - - @nox.session(python=["3.10"]) - @nox.parametrize( - "dbt_version", - ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9", "1.10"], - ) - @nox.parametrize( - "airflow_version", - ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "3.0"], - ) - def compatibility(session: nox.Session, airflow_version, dbt_version) -> None: - """Run both unit and integration tests.""" - session.run( - "pip3", - "install", - "--pre", - f"apache-airflow=={airflow_version}", - f"dbt-core=={dbt_version}", - ) diff --git a/docs/getting_started/execution-modes.rst b/docs/getting_started/execution-modes.rst deleted file mode 100644 index ea6a03f283..0000000000 --- a/docs/getting_started/execution-modes.rst +++ /dev/null @@ -1,374 +0,0 @@ -.. _execution-modes: - -Execution Modes -=============== - -Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: - -1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) -2. **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos -3. **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) -4. **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) -5. **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) -6. **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) -7. **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) -8. **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) -9. **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ -10. **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. -11. **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. - -The choice of the ``execution mode`` can vary based on each user's needs and concerns. For more details, check each execution mode described below. - -.. _execution-modes-comparison: - -.. list-table:: Execution Modes Comparison - :widths: 25 25 25 25 - :header-rows: 1 - - * - Execution Mode - - Task Duration - - Environment Isolation - - Cosmos Profile Management - * - Local - - Fast - - None - - Yes - * - Virtualenv - - Medium - - Lightweight - - Yes - * - Docker - - Slow - - Medium - - No - * - Kubernetes - - Slow - - High - - No - * - AWS_EKS - - Slow - - High - - No - * - Azure Container Instance - - Slow - - High - - No - * - GCP Cloud Run Job Instance - - Slow - - High - - No - * - AWS ECS - - Slow - - High - - No - * - Airflow Async - - Very Fast - - Medium - - Yes - * - Watcher - - Very Fast - - None - - Yes - * - Watcher Kubernetes - - Fast - - High - - No - -Local ------ - -By default, Cosmos uses the ``local`` execution mode. - -The ``local`` execution mode is the fastest way to run Cosmos operators since they don't install ``dbt`` nor build docker containers. However, it may not be an option for users using managed Airflow services such as -Google Cloud Composer, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`), the user may not be able to install ``dbt`` in a custom path. - -The ``local`` execution mode assumes a ``dbt`` binary is reachable within the Airflow worker node. - -If ``dbt`` was not installed as part of the Cosmos packages, -users can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. - -.. note:: - Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. - This feature is bound to `dbt partial parsing limitations `_. - Learn more: :ref:`partial-parsing`. - -When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). - -Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: - -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py - :language: python - :start-after: [START local_example] - :end-before: [END local_example] - - -Virtualenv ----------- - -If you're using managed Airflow on GCP (Cloud Composer), for instance, we recommend you use the ``virtualenv`` execution mode. - -The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. - -In this case, users are responsible for declaring which version of ``dbt`` they want to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. - -Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). -Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. - -Some drawbacks of this approach: - -- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. -- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. -- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. - -Example of how to use: - -.. literalinclude:: ../../dev/dags/example_virtualenv.py - :language: python - :start-after: [START virtualenv_example] - :end-before: [END virtualenv_example] - -Docker ------- - -The ``docker`` approach assumes users have a previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml``, managed by the user. - -The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). - -The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. - -This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. -If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. - -Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. - -Example DAG: - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.DOCKER, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "network_mode": "bridge", - }, - ) - - -Kubernetes ----------- - -The ``kubernetes`` approach is a very isolated way of running ``dbt`` since the ``dbt`` run commands from within a Kubernetes Pod, usually in a separate host. - -It assumes the user has a Kubernetes cluster. It also expects the user to ensure the Docker container has up-to-date ``dbt`` pipelines and profiles, potentially leading the user to declare secrets in two places (Airflow and Docker container). - -The ``Kubernetes`` deployment may be slower than ``Docker`` and ``Virtualenv`` assuming that the container image is built (which is slower than creating a Python ``virtualenv`` and installing ``dbt-core``) and the Airflow task needs to spin up a new ``Pod`` in Kubernetes. - -Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref:`kubernetes`. - -Example DAG: - -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py - :language: python - :start-after: [START kubernetes_seed_example] - :end-before: [END kubernetes_seed_example] - -AWS_EKS ----------- - -The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. -It uses the `EKSPodOperator `_ -to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. - - -Example DAG: - -.. code-block:: python - - postgres_password_secret = Secret( - deploy_type="env", - deploy_target="POSTGRES_PASSWORD", - secret="postgres-secrets", - key="password", - ) - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AWS_EKS, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "cluster_name": CLUSTER_NAME, - "get_logs": True, - "is_delete_operator_pod": False, - "secrets": [postgres_password_secret], - }, - ) - -Azure Container Instance ------------------------- -.. versionadded:: 1.4 - -Similar to the ``kubernetes`` approach, using ``Azure Container Instances`` as the execution mode gives a very isolated way of running ``dbt``, since the ``dbt`` run itself is run within a container running in an Azure Container Instance. - -This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. Please checkout the step-by-step guide for using Azure Container Instance as the execution mode - - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AZURE_CONTAINER_INSTANCE - ), - operator_args={ - "ci_conn_id": "aci", - "registry_conn_id": "acr", - "resource_group": "my-rg", - "name": "my-aci-{{ ti.task_id.replace('.','-').replace('_','-') }}", - "region": "West Europe", - "image": "dbt-jaffle-shop:1.0.0", - }, - ) - -GCP Cloud Run Job ------------------------- -.. versionadded:: 1.7 - -The ``gcp_cloud_run_job`` execution mode is particularly useful for users who prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. - -For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . - -This execution mode allows users to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. - - -.. code-block:: python - - gcp_cloud_run_job_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.GCP_CLOUD_RUN_JOB), - operator_args={ - "project_id": "my-gcp-project-id", - "region": "europe-west1", - "job_name": "my-crj-{{ ti.task_id.replace('.','-').replace('_','-') }}", - }, - ) - - -AWS ECS ---------- -.. versionadded:: 1.9.0 - -Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. - -This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. For users who require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. - -Please refer to the step-by-step guide for using AWS ECS as the execution mode. - -.. code-block:: python - - aws_ecs_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.AWS_ECS), - operator_args={ - "aws_conn_id": "aws_default", - "cluster": "my-ecs-cluster", - "task_definition": "my-dbt-task", - "container_name": "dbt-container", - "launch_type": "FARGATE", - "deferrable": True, - "network_configuration": { - "awsvpcConfiguration": { - "subnets": ["<<>>"], - "assignPublicIp": "ENABLED", - }, - }, - "environment_variables": {"DBT_PROFILE_NAME": "default"}, - }, - ) - -.. _airflow-async-execution-mode: - -Airflow Async -------------- - -.. versionadded:: 1.9.0 - -Although this execution mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. -In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. - -The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's -`Deferrable operators `__. -This execution mode could be preferred when you've long running resources and you want to run them asynchronously by -leveraging Airflow's deferrable operators. With that, you would be able to potentially observe higher throughput of tasks -as more dbt nodes will be run in parallel since they won't be blocking Airflow's worker slots. - -Example DAG: - -.. literalinclude:: ../../dev/dags/simple_dag_async.py - :language: python - :start-after: [START airflow_async_execution_mode_example] - :end-before: [END airflow_async_execution_mode_example] - -For a full step-by-step guide and limitations, check the :ref:`async-execution-mode` page. - - -Watcher Execution Mode (Experimental) -------------------------------------- - -.. versionadded:: 1.11.0 - -The ``watcher`` execution mode is an experimental execution mode that runs a single ``dbt build`` command from a producer task and has sensor tasks to watch the progress of the producer. -It is designed to improve DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. - -Check the :ref:`watcher-execution-mode` for more details. - - -Watcher Kubernetes Execution Mode (Experimental) ------------------------------------------------- - -.. versionadded:: 1.13.0 - -The ``watcher_kubernetes`` execution mode combines the speed of the ``watcher`` execution mode with the isolation of the ``kubernetes`` execution mode. It runs a single ``dbt build`` command from a producer task inside a Kubernetes pod and has sensor tasks to watch the progress of the producer. - -Check the :ref:`watcher-kubernetes-execution-mode` for more details. - - -.. _invocation_modes: - -Invocation Modes -================ -.. versionadded:: 1.4 - -For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: - -1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. - -2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ - In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ - This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. - -The invocation mode can be set in the ``ExecutionConfig`` as shown below: - -.. code-block:: python - - from cosmos.constants import InvocationMode - - dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.LOCAL, - invocation_mode=InvocationMode.DBT_RUNNER, - ), - ) - -If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. diff --git a/docs/getting_started/gcp-cloud-run-job.rst b/docs/getting_started/gcp-cloud-run-job.rst deleted file mode 100644 index fa4d0c60c4..0000000000 --- a/docs/getting_started/gcp-cloud-run-job.rst +++ /dev/null @@ -1,265 +0,0 @@ -.. _gcp-cloud-run-job: - -GCP Cloud Run Job Execution Mode -======================================= -.. versionadded:: 1.7 - -This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_gcp_crj_schematic.png - :width: 600 - -Prerequisites -+++++++++++++ -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Google Cloud SDK (`install guide `_) -4. Astronomer-cosmos package containing the dbt Cloud Run Job operators -5. GCP account with: - 1. A GCP project (`setup guide `_) - 2. IAM roles: - * Basic Role: `Owner `_ (control over whole project) or - * Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) - 3. Enabled service APIs: - * Artifact Registry API - * Cloud Run Admin API - * BigQuery API - 4. A service account with BigQuery roles: `JobUser `_ and `DataEditor `_ -6. Docker image built with required dbt project and dbt DAG -7. dbt DAG with Cloud Run Job operators in the Airflow DAGs directory to run in Airflow - -.. note:: - - Google Cloud Platform provides free tier on many resources, as well as Free Trial with $300 in credit. Learn more `here `_. - -More information on how to achieve 2-6 is detailed below. - - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: - -.. code-block:: bash - - python3 -m venv venv - source venv/bin/activate - python3 -m pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-bigquery,gcp-cloud-run-job]" - -**Setup gcloud and environment variables** - -Set environment variables that will be used to create cloud infrastructure. Replace placeholders with your unique GCP ``project id`` and ``region`` of the project: - -.. code-block:: bash - - export PROJECT_ID=<<>> - export REGION=<<>> - export REPO_NAME="astronomer-cosmos-dbt" - export IMAGE_NAME="$REGION-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/cosmos-example" - export SERVICE_ACCOUNT_NAME="cloud-run-job-sa" - export DATASET_NAME="astronomer_cosmos_example" - export CLOUD_RUN_JOB_NAME="astronomer-cosmos-example" - -Before we do anything in the GCP project, we first need to authorize gcloud to access the Cloud Platform with Google user credentials: - -.. code-block:: bash - - gcloud auth login - -You'll receive a link to sign into Google Cloud SDK using a Google Account. - -Next, set default ``project id`` using below command: - -.. code-block:: bash - - gcloud config set project $PROJECT_ID - -In case BigQuery has never been used before in the project, run below command to enable BigQuery API: - -.. code-block:: bash - - gcloud services enable bigquery.googleapis.com - -**Setup Artifact Registry** - -In order to run a container in Cloud Run Job, it needs access to the container image. In our setup, we will use Artifact Registry repository that stores images. -To use Artifact Registry, you need to enable the API first: - -.. code-block:: bash - - gcloud services enable artifactregistry.googleapis.com - -To set an Artifact Registry repository up, you can use the following bash command: - -.. code-block:: bash - - gcloud artifacts repositories create $REPO_NAME \ - --repository-format=docker \ - --location=$REGION \ - --project $PROJECT_ID - -**Setup Service Account** - -In order to use dbt and make transformations in BigQuery, Cloud Run Job needs some BigQuery permissions. One way to achieve that is to set up a separate ``Service Account`` with needed permissions: - -.. code-block:: bash - - # create a service account - gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME - -.. code-block:: bash - - # grant JobUser role - gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/bigquery.jobUser" - -.. code-block:: bash - - # grant DataEditor role - gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/bigquery.dataEditor" - -**Build the dbt Docker image** - -Now, we are going to download an example dbt project and build a Docker image with it. - -.. important:: - - You need to ensure Docker is using the right credentials to push images. For Artifact Registry, this can be done by running the following command: - - .. code-block:: bash - - gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://$REGION-docker.pkg.dev - - The token will be valid for 1 hour. After that, you need to create another one, if still needed. - -Clone the `cosmos-example `_ repo: - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Open `Dockerfile `_ located in ``gcp_cloud_run_job_example`` folder and change environments variables ``GCP_PROJECT_ID`` and ``GCP_REGION`` to your GCP project id and project region. - -Build a Docker image using previously modified ``Dockerfile``, which will be used by Cloud Run Job: - -.. code-block:: bash - - docker build -t $IMAGE_NAME -f gcp_cloud_run_job_example/Dockerfile.gcp_cloud_run_job . - -.. important:: - - Make sure to stay in ``cosmos-example`` directory when running ``docker build`` command. - -After this, the image needs to be pushed to the Artifact Registry: - -.. code-block:: bash - - docker push $IMAGE_NAME - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The `bigquery dbt profile `_ file is added to the image - - The dbt_project.yml is replaced with `bigquery_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Create Cloud Run Job instance** - -When the image is pushed to Artifact Registry, you can finally create Cloud Run Job with the image and previously created service account. - -First, enable Cloud Run Admin API using below command: - -.. code-block:: bash - - gcloud services enable run.googleapis.com - - -Next, set default Cloud Run region to your GCP region: - -.. code-block:: bash - - gcloud config set run/region $REGION - -Then, run below command to create Cloud Run Job instance: - -.. code-block:: bash - - gcloud run jobs create $CLOUD_RUN_JOB_NAME \ - --image=$IMAGE_NAME \ - --task-timeout=180s \ - --max-retries=0 \ - --cpu=1 \ - --memory=512Mi \ - --service-account=$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com - -**Setup Airflow Connections** - -Now, when you have the required Google Cloud infrastructure, you still need to check Airflow configuration to ensure the infrastructure can be used. You'll need a ``google_cloud_default`` connection in order to work on GCP resources. - -Check out an `example `_ of the ``airflow-settings.yml`` file. If you are using Astro CLI, filling in the right values here will be enough for this to work. - -**Setup and Trigger the DAG with Airflow** - -Open `jaffle_shop_gcp_cloud_run_job `_ DAG file and update ``GCP_PROJECT_ID`` and ``GCP_LOCATION`` constants with your GCP project id and project region. - -When the DAG is configured, copy the ``dags`` directory from ``cosmos-example`` repo to your Airflow home: - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow: - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_gcp_cloud_run_job `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_gcp_cloud_run_job.png - :width: 800 - - -You can also verify the tables that were created using dbt in BigQuery Studio: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_big_query.png - :width: 800 - - -**Delete resources** - -After the successful tests, don't forget to delete Google Cloud resources to save up costs: - -.. code-block:: bash - - # Delete Cloud Run Job instance - - gcloud run jobs delete $CLOUD_RUN_JOB_NAME - -.. code-block:: bash - - # Delete BigQuery main and custom dataset specified in dbt schema.yml with all tables included - - bq rm -r -f -d $PROJECT_ID:$DATASET_NAME - - bq rm -r -f -d $PROJECT_ID:dbt_dev - -.. code-block:: bash - - # Delete Artifact Registry repository with all images included - - gcloud artifacts repositories delete $REPO_NAME \ - --location=$REGION diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 2bb43dfa3f..5184c867c8 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -1,27 +1,31 @@ .. _getting-started: .. toctree:: + :maxdepth: 1 :hidden: - :caption: Contents: + :caption: Cosmos Fundamentals + Similar dbt and Airflow concepts + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Get started with Cosmos + + Open-source Airflow Astro - MWAA - GCC - Open-Source - Execution Modes - Docker Execution Mode - Kubernetes Execution Mode - Azure Container Instance Execution Mode - AWS Container Run Job Execution Mode - GCP Cloud Run Job Execution Mode - Airflow Async Execution Mode - Watcher Execution Mode - Watcher Kubernetes Execution Mode - dbt and Airflow Similar Concepts + Google Cloud Composer (GCC) + Amazon Managed Workflows for Apache Airflow (MWAA) + + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Operators + Operators Custom Airflow Properties - Getting Started =============== @@ -45,11 +49,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/getting_started/kubernetes.rst b/docs/getting_started/kubernetes.rst deleted file mode 100644 index 607ba07bd7..0000000000 --- a/docs/getting_started/kubernetes.rst +++ /dev/null @@ -1,167 +0,0 @@ -.. _kubernetes: - -Kubernetes Execution Mode -============================================== - -The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: - -- Postgres is run in the Kubernetes (K8s) cluster as a container -- Airflow is run locally, and it triggers a K8s Pod which runs dbt - -Requirements -++++++++++++ - -To test the DbtKubernetesOperators locally, we encourage you to install the following: - -- Local Airflow (either standalone or using Astro CLI) -- `Kind `_ to run K8s locally -- `Helm `_ to install Postgres in K8s -- `Docker `_ to create the dbt container image, which will allow Airflow to create a K8s pod which will run dbt - -At the moment, the user is expected to add to the Docker image both: - -- The dbt project files -- The dbt Profile, which contains the information for dbt to access the database while parsing the project from Apache Airflow nodes -- Handle secrets - -Additional KubernetesPodOperator parameters can be added to the ``operator_args`` parameter of the ``DbtKubernetesOperator``. - -For instance, - -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py - :language: python - :start-after: [START kubernetes_tg_example] - :end-before: [END kubernetes_tg_example] - -Step-by-step instructions -+++++++++++++++++++++++++ - -Using installed `Kind `_, you can setup a local kubernetes cluster - -.. code-block:: bash - - kind create cluster - -Deploy a Postgres pod to Kind using `Helm `_ - -.. code-block:: bash - - helm repo add bitnami https://charts.bitnami.com/bitnami - helm repo update - helm install postgres bitnami/postgresql - -Retrieve the Postgres password and set it as an environment variable. - -.. code-block:: bash - - export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgres-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d) - -Check that the environment variable was set and that it is not empty - -.. code-block:: bash - - echo $POSTGRES_PASSWORD - -Expose the Postgres to the host running Docker/Kind. - -.. code-block:: bash - - kubectl port-forward --namespace default postgres-postgresql-0 5432:5432 - -Check that you're able to connect to the exposed pod. - -.. code-block:: bash - - PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432 - - postgres=# \dt - \q - -Create a K8s secret which contains the credentials to access Postgres. - -.. code-block:: bash - - kubectl create secret generic postgres-secrets --from-literal=host=postgres-postgresql.default.svc.cluster.local --from-literal=password=$POSTGRES_PASSWORD - -Clone the example repo that contains the Airflow DAG and dbt project files. - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example/ - -Create a Docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be run in K8s. - -.. code-block:: bash - - docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . - -.. note:: - - If running on M1, you may need to set the following environment variables to run the Docker build command in case it fails. - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a look at the Dockerfile to understand its purpose so that you can use it as a reference in your project. - - - The `dbt profile `__ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -Make the build image available in the Kind K8s cluster. - -.. code-block:: bash - - kind load docker-image dbt-jaffle-shop:1.0.0 - -Create a Python virtual environment and install the latest version of Astronomer Cosmos, which contains the K8s Operator. - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install "astronomer-cosmos[dbt-postgres]" apache-airflow-providers-cncf-kubernetes - -Make the `jaffle_shop_kubernetes.py `__ file at your Airflow DAG home: - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You may need to run Airflow standalone with ``sudo`` if your Airflow user is unable to access the Docker socket URL or pull images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_k8s `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_k8s_dag_run.png - :width: 800 - -.. _kubernetes-known-limitations: - -Known Limitations -+++++++++++++++++ - -The Kubernetes execution mode has the following limitations: - -- Does not emit OpenLineage events (there is an `open ticket #496 `__ to address this) -- Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) -- Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) -- Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) -- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) -- Does not expose Compiled SQL as a `templated field `_ -- Does not benefit from `Cosmos caching mechanisms `_ -- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/getting_started/mwaa.rst b/docs/getting_started/mwaa.rst index 5b7c41bde5..5b1da23439 100644 --- a/docs/getting_started/mwaa.rst +++ b/docs/getting_started/mwaa.rst @@ -1,7 +1,7 @@ .. _mwaa: -Getting Started on MWAA -======================= +Getting Started with Cosmos on Amazon Managed Workflows +======================================================= Users can face Python dependency issues when trying to use the Cosmos `Local Execution Mode `_ in Amazon Managed Workflows for `Apache Airflow® `_ (MWAA). diff --git a/docs/getting_started/open-source.rst b/docs/getting_started/open-source.rst index ba9bbdb15c..f5d1db832b 100644 --- a/docs/getting_started/open-source.rst +++ b/docs/getting_started/open-source.rst @@ -1,7 +1,7 @@ .. _open-source: -Getting Started on Open Source Airflow -====================================== +Getting Started with Cosmos on Open-source Airflow +================================================== When running open-source Airflow, your setup may vary. This guide assumes you have access to edit the underlying image. diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst deleted file mode 100644 index af7589650c..0000000000 --- a/docs/getting_started/watcher-execution-mode.rst +++ /dev/null @@ -1,480 +0,0 @@ -.. _watcher-execution-mode: - -Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Execution in Cosmos -============================================================================================ - -With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. - -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_ and Airflow deferrable sensors, the performance gains will depend on three major factors: - -- The amount of dbt ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag -- The topology of the dbt pipeline -- The ``poke_interval`` and ``timeout`` settings of the ``DbtConsumerWatcherSensor`` operator, which determine the frequency and duration of the sensor's polling. - -------------------------------------------------------------------------------- - -Background: The Problem with the Local Execution Mode in Cosmos ---------------------------------------------------------------- - -When running dbt via Cosmos using the default ``ExecutionMode.LOCAL``, each dbt model is executed as a separate Airflow task. - -This provides strong observability and task-level retry control — but it comes at a cost. Each model runs a new dbt process, which introduces significant overhead. - -Consider the `google/fhir-dbt-analytics `_ project: - -+-------------------------------------------------------------+-----------------------------------+------------------+ -| Run Type | Description | Total Runtime | -+=============================================================+===================================+==================+ -| Single ``dbt run`` (dbt CLI) | Runs the whole DAG in one command | ~5m 30s | -+-------------------------------------------------------------+-----------------------------------+------------------+ -| One ``dbt run`` per model, totalling 184 commands (dbt CLI) | Each model is its own task | ~32m | -+-------------------------------------------------------------+-----------------------------------+------------------+ - -This difference motivated a rethinking of how Cosmos interacts with dbt. - -------------------------------------------------------------------------------- - -Concept: ``ExecutionMode.WATCHER`` ----------------------------------- - -``ExecutionMode.WATCHER`` combines the **speed of a single dbt run** with the **observability and task management of Airflow**. - -It is built on two operator types: - -* ``DbtProducerWatcherOperator`` (`#1982 `_) - Runs dbt **once** across the entire pipeline, register to `dbt event callbacks `_ and sends model progress updates via Airflow **XComs**. - -* ``DbtConsumerWatcherSensor`` (`#1998 `_) - Watches those XComs and marks individual Airflow tasks as complete when their corresponding dbt models finish. - -Together, these operators let you: - -* Run dbt as a single command (for speed) -* Retain model-level observability (for clarity) -* Retry specific models (for resilience) - -------------------------------------------------------------------------------- - -Performance Gains ------------------ - -We used a dbt project developed by Google, the `google/fhir-dbt-analytics `_ project, that interfaces with BigQuery. It contains: -* 2 seeds -* 52 sources -* 185 models - -Initial benchmarks, using illustrate significant improvements: - -+-----------------------------------------------+-----------+--------------------+ -| Environment | Threads | Execution Time (s) | -+===============================================+===========+====================+ -| dbt build (dbt CLI) | 4 | 6–7 | -+-----------------------------------------------+-----------+--------------------+ -| dbt run per model (dbt CLI) | — | 30 | -| similar to the Cosmos ``ExecutionMode.LOCAL`` | | | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.LOCAL`` (Astro CLI) | — | 10–15 | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.WATCHER`` (Astro CLI) | 1 | 26 | -| | 2 | 14 | -| | 4 | 7 | -| | 8 | 4 | -| | 16 | 2 | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.WATCHER`` (Astro Cloud | 8 | ≈5 | -| Standard Deployment with A10 workers | | | -+-----------------------------------------------+-----------+--------------------+ - -The last line represents the performance improvement in a real-world Airflow deployment, using `Astro Cloud `_. - -Depending on the dbt workflow topology, if your dbt DAG previously took 5 minutes with ``ExecutionMode.LOCAL``, you can expect it to complete in roughly **1 minute** with ``ExecutionMode.WATCHER``. - -We plan to repeat these benchmarks and share the code with the community in the future. - - -.. note:: - ``ExecutionMode.WATCHER`` relies on the ``threads`` value defined in your dbt profile. Start with a conservative value that matches the CPU capacity of your Airflow workers, then gradually increase it to find the sweet spot between faster runs and acceptable memory/CPU usage. - -When we ran the `astronomer/cosmos-benchmark `_ project with ``ExecutionMode.WATCHER``, that same ``threads`` setting directly affected runtime: moving from 1 to 8 threads reduced the end-to-end ``dbt build`` duration from roughly 26 seconds to about 4 seconds (see table above), while 16 threads squeezed it to around 2 seconds at the cost of higher CPU usage. Use those numbers as a reference point when evaluating how thread counts scale in your own environment. - -To increase the number of threads, edit your dbt ``profiles.yml`` (or Helm values if you manage the profile there) and update the ``threads`` key for the target you use with Cosmos: - -.. code-block:: yaml - - your_dbt_project: - target: prod - outputs: - prod: - type: postgres - host: your-host - user: your-user - password: your-password - schema: analytics - threads: 8 # increase or decrease to match available resources - - -If you prefer to manage threads through Cosmos profile mappings instead of editing ``profiles.yml`` directly, pass ``profile_args={"threads": }`` to your ``ProfileConfig``. For example, using the built-in ``PostgresUserPasswordProfileMapping``: - -.. code-block:: python - - from cosmos.config import ProfileConfig - from cosmos.profiles import PostgresUserPasswordProfileMapping - - profile_config = ProfileConfig( - profile_name="jaffle_shop", - target_name="prod", - profile_mapping=PostgresUserPasswordProfileMapping( - conn_id="postgres_connection", - profile_args={"threads": 8}, - ), - ) - - -------------------------------------------------------------------------------- - -Example Usage of ``ExecutionMode.WATCHER`` ------------------------------------------- - -There are two main ways to use the new execution mode in Cosmos — directly within a ``DbtDag``, or embedded as part of a ``DbtTaskGroup`` inside a larger DAG. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can enable WATCHER mode directly in your ``DbtDag`` configuration. -This approach is best when your Airflow DAG is fully dedicated to a dbt project. - -.. literalinclude:: ../../dev/dags/example_watcher.py - :language: python - :start-after: [START example_watcher] - :end-before: [END example_watcher] - -As it can be observed, the only difference with the default ``ExecutionMode.LOCAL`` is the addition of the ``execution_config`` parameter with the ``execution_mode`` set to ``ExecutionMode.WATCHER``. The ``ExecutionMode`` enum can be imported from ``cosmos.constants``. For more information on the ``ExecutionMode.LOCAL``, please, check the `dedicated page `__ - -**How it works:** - -* Cosmos executes your dbt project once via a producer task. -* Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. -* The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). - -**How it looks like:** - -.. image:: /_static/jaffle_shop_watcher_dbt_dag_dag_run.png - :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` - :align: center - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Example 2 — Using ``DbtTaskGroup`` with ``ExecutionMode.WATCHER`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If your Airflow DAG includes multiple stages or integrations (e.g., data ingestion → dbt → reporting), use ``DbtTaskGroup`` to embed your dbt project into a larger DAG — still benefiting from WATCHER performance. - -.. code-block:: python - :caption: example_watcher_taskgroup.py - :name: example_watcher_taskgroup - - from airflow.models import DAG - from airflow.operators.empty import EmptyOperator - from cosmos import DbtTaskGroup - - with DAG( - dag_id="example_watcher_taskgroup", - schedule="@daily", - start_date=datetime(2023, 1, 1), - catchup=False, - ): - """ - The simplest example of using Cosmos to render a dbt project as a TaskGroup. - """ - pre_dbt = EmptyOperator(task_id="pre_dbt") - - first_dbt_task_group = DbtTaskGroup( - group_id="first_dbt_task_group", - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER, - ), - project_config=ProjectConfig(DBT_PROJECT_PATH), - profile_config=profile_config, - operator_args=operator_args, - ) - - pre_dbt >> first_dbt_task_group - -**Key advantages:** - -* Integrates seamlessly into complex Airflow DAGs. -* Uses the same high-performance producer/consumer execution model. -* Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. - -.. image:: /_static/jaffle_shop_watcher_dbt_taskgroup_dag_run.png - :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` - :align: center - -------------------------------------------------------------------------------- - -Additional details -------------------- - -~~~~~~~~~~~~~~~~ -How retries work -~~~~~~~~~~~~~~~~ - -When the ``dbt build`` command run by ``DbtProducerWatcherOperator`` fails, it will notify all the ``DbtConsumerWatcherSensor``. - -The individual watcher tasks that subclass ``DbtConsumerWatcherSensor`` can retry the dbt command themselves, using the same behavior as ``ExecutionMode.LOCAL``. - -If a branch of the DAG fails, users can clear the status of a failed consumer task, including its downstream tasks, via the Airflow UI, and each of them will run in ``ExecutionMode.LOCAL``. - -**Producer retry behavior** - -.. versionadded:: 1.12.2 - -When the ``DbtProducerWatcherOperator`` is triggered for a retry (try_number > 1), it will not re-run the dbt build command and will succeed. In previous versions of Cosmos, the producer task would fail during retries. -This behavior is designed to support TaskGroup-level retries, as reported in `#2282 `_. - -**Why this matters:** - -- In earlier versions, attempting to retry the producer task would raise an ``AirflowException``, causing the retry to fail immediately. -- Now, the producer gracefully skips execution on retries, logging an informational message explaining that the retry was skipped to avoid running a second ``dbt build``. -- This allows users to retry entire TaskGroups and/or DAGs without the producer task blocking the retry flow. - -**Important considerations:** - -- The producer task should still be configured with ``retries=0`` (which Cosmos enforces by default) to avoid unintended duplicate ``dbt build`` runs. - -- By default, Cosmos sets ``retries`` to ``0`` in``DbtProducerWatcherOperator``. Users can retry manually by clearing the status of the producer task and all its downstream tasks, keeping in mind that the producer task will not re-run the ``dbt build`` command and will succeed. - -The overall retry behavior will be further improved once `#1978 `_ is implemented. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Watcher dbt Execution Queue -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. versionadded:: 1.14.0 - -In watcher execution mode, by default, consumer sensor tasks are lightweight sensors that wait for the producer task to complete. On their first attempt, they require minimal CPU and memory resources. However, when these tasks retry, they execute the dbt command for the node, which may require significantly more resources. - -The ``watcher_dbt_execution_queue`` configuration allows you to specify a different worker queue for retry attempts. This enables you to: - -- **Optimize resource allocation** — Use lightweight workers for initial sensor execution and high-resource workers for retries -- **Improve scheduling efficiency** — Prevent resource contention between initial sensor tasks and retry executions -- **Scale independently** — Scale retry queues separately based on retry workload patterns - -**Configuration:** - -Set the ``watcher_dbt_execution_queue`` in your Airflow configuration: - -.. code-block:: ini - - [cosmos] - watcher_dbt_execution_queue = high_memory_queue - -Or via environment variable: - -.. code-block:: bash - - export AIRFLOW__COSMOS__WATCHER_DBT_EXECUTION_QUEUE=high_memory_queue - -**How it works:** - -- For watcher producer tasks (``DbtProducerWatcherOperator``), the configured queue is used during their first execution -- For watcher consumer tasks (``DbtConsumerWatcherSensor``), from their first retry onwards, if ``watcher_dbt_execution_queue`` is configured, the task is automatically assigned to the specified queue -- This behavior is enforced by Cosmos via an `Airflow cluster policy `_ (``task_instance_mutation_hook``) that mutates ``task_instance.queue`` at runtime for retry attempts - -.. note:: - - For producer task execution, we encourage users to set the ``watcher_dbt_execution_queue`` configuration. If, for any reason, users prefer to use a different node pool for producer tasks without setting an Airflow Cluster Policy, they can set the ``queue`` argument via ``setup_operator_args``. This, however, would not solve the problem of assigning consumer retries to nodes that may have more memory and CPU available. - - The effective precedence is: - - ``watcher_dbt_execution_queue`` > explicit ``queue`` on the producer (from ``setup_operator_args``) > ``operator_args`` > your Airflow deployment’s default queue. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Installation of Airflow and dbt -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Since Cosmos 1.12.0, ``ExecutionMode.WATCHER`` works well regardless of whether dbt and Airflow are installed in the same Python virtual environment. - -When dbt and Airflow are installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` uses dbt `callback features `_. - -When dbt and Airflow are not installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` consumes the dbt `structured logging `_ to update the consumer tasks. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Synchronous versus Asynchronous sensor execution -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In Cosmos 1.11.0, the ``DbtConsumerWatcherSensor`` operator is implemented as a synchronous XCom sensor, which continuously occupies the worker slot - even if they're just sleeping and checking periodically. - -Starting with Cosmos 1.12.0, the ``DbtConsumerWatcherSensor`` supports -`deferrable (asynchronous) execution `_. Deferrable execution frees up the Airflow worker slot, while task status monitoring is handled by the Airflow triggerer component, -which increases overall task throughput. By default, the sensor now runs in deferrable mode. - -------------------------------------------------------------------------------- - -Known Limitations -------------------- - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Producer task implementation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The producer task is implemented as a ``DbtProducerWatcherOperator`` and currently relies on dbt being installed alongside the Airflow deployment, as in the ``ExecutionMode.LOCAL`` implementation. - -The alternative to this implementation is to use ``ExecutionMode.WATCHER_KUBERNETES``, which is built on top of ``ExecutionMode.KUBERNETES``. Check :ref:`watcher-kubernetes-execution-mode` for more information. - -~~~~~~~~~~~~~~~~~~~~~~~~ -Individual dbt Operators -~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``ExecutionMode.WATCHER`` efficiently implements the following operators: -* ``DbtSeedWatcherOperator`` -* ``DbtSnapshotWatcherOperator`` -* ``DbtRunWatcherOperator`` - -However, other operators that are available in the ``ExecutionMode.LOCAL`` mode are not implemented. - -The ``DbtBuildWatcherOperator`` is not implemented, since the build command is executed by the producer ``DbtProducerWatcherOperator`` operator. - -Additionally, since the ``dbt build`` command does not run ``source`` nodes, the operator ``DbtSourceWatcherOperator`` is equivalent to the ``DbtSourceLocalOperator`` operator, from ``ExecutionMode.LOCAL``. - -Finally, the following features are not implemented as operators under ``ExecutionMode.WATCHER``: - -* ``dbt ls`` -* ``dbt run-operation`` -* ``dbt docs`` -* ``dbt clone`` - -You can still invoke these operators using the default ``ExecutionMode.LOCAL`` mode. - -~~~~~~~~~~~~~ -Test behavior -~~~~~~~~~~~~~ - -By default, the watcher mode runs tests alongside models via the ``dbt build`` command being executed by the producer ``DbtProducerWatcherOperator`` operator. - -As a starting point, this execution mode does not support the ``TestBehavior.AFTER_EACH`` behavior, since the tests are not run as individual tasks. Since this is the default ``TestBehavior`` in Cosmos, we are injecting ``EmptyOperator`` as a starting point to ensure a seamless transition to the new mode. - -The ``TestBehavior.BUILD`` behavior is embedded in the producer ``DbtProducerWatcherOperator`` operator. - -The ``TestBehavior.NONE`` and ``TestBehavior.AFTER_ALL`` behave similarly to ``ExecutionMode.LOCAL``. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Airflow Datasets and Assets -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -While the ``ExecutionMode.WATCHER`` supports the ``emit_datasets`` parameter, the Airflow Datasets and Assets are emitted from the ``DbtProducerWatcherOperator`` task instead of the consumer tasks, as done for other Cosmos' execution modes. - -~~~~~~~~~~~~~~~~~~~~~~ -Source freshness nodes -~~~~~~~~~~~~~~~~~~~~~~ - -Since Cosmos 1.6, it `supports the rendering of source nodes `_. - -We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. - -This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. - -We have a follow-up ticket to `further investigate this use case `_. - - -Advanced config -------------------- - -~~~~~~~~~~~~~~~~ -Callback support -~~~~~~~~~~~~~~~~ - -The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` will use the user-defined callback function similar to ``ExecutionMode.LOCAL`` mode. - -You can define different ``callback`` behaviors for producer and consumer nodes by using ``operator_args`` to configure the consumer callback and ``setup_operator_args`` to override the callback for the producer, as described below. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Overriding ``operator_args`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` operators handle ``operator_args`` similar to the ``ExecutionMode.LOCAL`` mode. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Using Custom Args for the Producer and Watcher -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. versionadded:: 1.12.0 - -If you need to override ``operator_args`` for the ``DbtProducerWatcherOperator``, you can do so using ``setup_operator_args``. - -When using ``ExecutionMode.WATCHER``, you may want to configure specific properties, such as ``retries`` specifically for the ``DbtProducerWatcherOperator`` task. This can be useful for several reasons: -- Improved resilience - transient issues (e.g., temporary database or network failures) can be automatically retried. -- Reduced manual intervention - failed producer runs can recover without requiring operator restarts. -- Better reliability - retry behavior can be tuned independently from sensor tasks. - -Example: Configure the producer task with custom retry settings. - -.. code-block:: python - - from datetime import timedelta - from cosmos.config import ExecutionConfig - from cosmos.constants import ExecutionMode - - execution_config = ExecutionConfig( - execution_mode=ExecutionMode.WATCHER, - setup_operator_args={ - "retries": 0, - "retry_delay": timedelta(minutes=5), - }, - ) - -This allows you to customize ``DbtProducerWatcherOperator`` retry behavior without affecting the arguments used by the other sensor tasks. - -If configuring queues, we suggest using the previously mentioned ``watcher_dbt_execution_queue`` configuration instead of the ``setup_operator_args``. - -.. note:: - Please note that ``setup_operator_args`` is specific to Cosmos and is not related to Airflow setup or teardown task. - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Sensor slot allocation and polling -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Each ``DbtDag`` or ``DbtTaskGroup`` root node will startup during DAG runs at - potentially - the same time as the DAG Run. This may not happen, since it is dependent on the -concurrency settings and available task slots in the Airflow deployment. - -The consequence is that tasks may take longer to be updated if they are not sensing at the moment that the transformation happens. - -We plan to review this behaviour and alternative approaches in the future. - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Asynchronous sensor execution -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Deferrable execution is currently supported only for dbt models, seeds and snapshots. -- Deferrable execution applies only to the first task attempt (try number 1). For subsequent retries, the sensor falls back to synchronous execution. - -To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. - -.. literalinclude:: ../../dev/dags/example_watcher.py - :language: python - :start-after: [START example_watcher_synchronous] - :end-before: [END example_watcher_synchronous] - -------------------------------------------------------------------------------- - -Troubleshooting ---------------- - -Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." -Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. Check the `dbt threading docs `_ for more information on how to set the number of threads. - - -Summary -------- - -``ExecutionMode.WATCHER`` represents a significant leap forward for running dbt in Airflow via Cosmos: - -* ✅ Up to **5× faster** dbt DAG runs -* ✅ Maintains **model-level visibility** in Airflow -* ✅ Enables **smarter resource allocation** -* ✅ Built on proven Cosmos rendering techniques - -This is an experimental feature, and we are looking for feedback from the community. - -Stay tuned for further documentation and base image support for the ``ExecutionMode.WATCHER`` in upcoming releases. diff --git a/docs/getting_started/watcher-kubernetes-execution-mode.rst b/docs/getting_started/watcher-kubernetes-execution-mode.rst deleted file mode 100644 index 16dbbffd0a..0000000000 --- a/docs/getting_started/watcher-kubernetes-execution-mode.rst +++ /dev/null @@ -1,214 +0,0 @@ -.. _watcher-kubernetes-execution-mode: - -``ExecutionMode.WATCHER_KUBERNETES``: High-Performance dbt Execution in Kubernetes -=================================================================================== - -.. versionadded:: 1.13.0 - -The ``ExecutionMode.WATCHER_KUBERNETES`` combines the **speed of the** :ref:`watcher-execution-mode` **with the isolation of** :ref:`kubernetes`. - -This execution mode is ideal for users who: - -* Want to leverage the performance benefits of the watcher execution mode -* Need to run dbt in isolated Kubernetes pods -* Prefer not to install dbt in their Airflow deployment - -------------------------------------------------------------------------------- - -Background ----------- - -The :ref:`watcher-execution-mode` introduced in Cosmos 1.11.0 significantly reduces dbt pipeline run times by running dbt as a single command while maintaining model-level observability in Airflow. - -However, the original ``ExecutionMode.WATCHER`` requires dbt to be installed alongside Airflow. The ``ExecutionMode.WATCHER_KUBERNETES`` removes this limitation by running the dbt command inside Kubernetes pods, similar to ``ExecutionMode.KUBERNETES``. - -For more details on the watcher concept and how it works, please refer to the :ref:`watcher-execution-mode` documentation. - -------------------------------------------------------------------------------- - -How to Use ----------- - -Users previously using ``ExecutionMode.KUBERNETES`` can simply replace the ``execution_mode`` to use ``ExecutionMode.WATCHER_KUBERNETES``. - -The following example shows how to configure a ``DbtDag`` with ``ExecutionMode.WATCHER_KUBERNETES``: - -.. code-block:: python - - from cosmos import DbtDag - from cosmos.config import ExecutionConfig - from cosmos.constants import ExecutionMode - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - "image": DBT_IMAGE, - "get_logs": True, - "log_events_on_failure": True, - }, - ) - -**Key differences from** ``ExecutionMode.KUBERNETES``: - -* The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` -* The producer task runs the entire ``dbt build`` command in a single Kubernetes pod -* Consumer tasks (sensors) watch for the completion of their corresponding dbt models - -For the complete setup including Kubernetes secrets, Docker image configuration, and profile setup, refer to the :ref:`kubernetes` documentation. - -------------------------------------------------------------------------------- - -Performance Gains ------------------ - -Early benchmarks using the ``jaffle_shop_watcher_kubernetes`` DAG show significant improvements: - -+-----------------------------------------------+------------------+ -| Execution Mode | Total Runtime | -+===============================================+==================+ -| ``ExecutionMode.KUBERNETES`` | 00:00:32.155 | -+-----------------------------------------------+------------------+ -| ``ExecutionMode.WATCHER_KUBERNETES`` | 00:00:11.783 | -+-----------------------------------------------+------------------+ - -This represents approximately a **63% reduction** in total DAG runtime. - -The performance improvement comes from: - -* Running dbt as a single command (reducing Kubernetes pod startup overhead) -* Leveraging dbt's native threading capabilities -* Eliminating repeated dbt initialization for each model - -------------------------------------------------------------------------------- - -Known Limitations ------------------ - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Kubernetes Provider Version Compatibility -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -``ExecutionMode.WATCHER_KUBERNETES`` does not work with older versions of the ``apache-airflow-providers-cncf-kubernetes`` provider (<=10.7.0). - -Please ensure you have a compatible version installed: - -.. code-block:: bash - - pip install "apache-airflow-providers-cncf-kubernetes>10.7.0" - -We successfully tested against the most recent release of the provider (`10.12.2 `_). - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Support for KPO deferrable mode -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The producer node created by the ``ExecutionMode.WATCHER_KUBERNETES`` producer task can be set to deferrable mode as long as: - -- The correct version of Airflow Kubernetes is installed (``>=10.12.2``). This version fixed a bug (`PR `_) that prevented setting callbacks and parsing the logs when the Kubernetes Operator run using ``deferrable``. The experience should be further improved once `this other PR is merged `_. - -.. code-block:: bash - - pip install "apache-airflow-providers-cncf-kubernetes>=10.12.2" - -- The arguments ``deferrable=True`` and ``is_delete_operator_pod=True`` are set: - -.. code-block:: python - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - "deferrable": True, - "is_delete_operator_pod": True, - "image": DBT_IMAGE, - "get_logs": True, - "log_events_on_failure": True, - }, - ) - -Conversely, the consumer tasks that subclass ``DbtConsumerWatcherKubernetesSensor`` run in deferrable mode by default when operating as a sensor. They can also operate in deferrable mode if they are running dbt themselves upon retry. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Mandatory ``operator_args`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``operator_args`` must define ``get_logs`` and ``log_events_on_failure``: - -.. code-block: python - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - # ... other KPO mandatory args ... - "get_logs": True, - "log_events_on_failure": True, - }, - ) - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Other Inherited Limitations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following limitations from ``ExecutionMode.WATCHER`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``: - -* **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. - -* **Test behavior**: The ``TestBehavior.AFTER_EACH`` is not supported. Tests are run as part of the ``dbt build`` command by the producer task. - -* **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. - -For more details on these limitations, refer to the :ref:`watcher-execution-mode` documentation. - -Additionally, the limitations from ``ExecutionMode.KUBERNETES`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``. For details, refer to the :ref:`kubernetes-known-limitations` documentation. - -------------------------------------------------------------------------------- - -Example DAG ------------ - -Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: - -.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py - :language: python - -------------------------------------------------------------------------------- - -Prerequisites -------------- - -Before using ``ExecutionMode.WATCHER_KUBERNETES``, ensure you have: - -1. A Kubernetes cluster configured and accessible from your Airflow deployment -2. A Docker image containing your dbt project and profile -3. The ``apache-airflow-providers-cncf-kubernetes`` provider installed (version >10.7.0) - -For detailed setup instructions, refer to the :ref:`kubernetes` documentation. - -------------------------------------------------------------------------------- - -Summary -------- - -``ExecutionMode.WATCHER_KUBERNETES`` provides: - -* ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` -* ✅ **Isolation** between dbt and Airflow dependencies -* ✅ **Model-level visibility** in Airflow -* ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` - -This execution mode is ideal for teams who want the performance benefits of the watcher mode while maintaining the isolation provided by Kubernetes execution. From df0c4911b0ae3e1b84fc3b4686dc91b92e792ce1 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:57:44 -0500 Subject: [PATCH 03/48] rename to guides --- docs/{configuration => guides}/caching.rst | 0 docs/{configuration => guides}/callbacks/callbacks.rst | 0 docs/{configuration => guides}/compiled-sql.rst | 0 .../configure-tests/testing-behavior.rst | 0 docs/{configuration => guides}/cosmos-conf.rst | 0 docs/{configuration => guides}/dag-customization.rst | 0 docs/{configuration => guides}/dbt-docs/generating-docs.rst | 0 docs/{configuration => guides}/dbt-docs/hosting-docs.rst | 0 docs/{configuration => guides}/dbt-fusion.rst | 0 docs/{configuration => guides}/execution-config.rst | 0 .../{configuration => guides}/execution-modes-local-conflicts.rst | 0 docs/{configuration => guides}/index.rst | 0 docs/{configuration => guides}/lineage.rst | 0 docs/{configuration => guides}/logging.rst | 0 docs/{configuration => guides}/memory_optimization.rst | 0 docs/{configuration => guides}/multi-project.rst | 0 docs/{configuration => guides}/operator-args.rst | 0 docs/{configuration => guides}/parsing-methods.rst | 0 docs/{configuration => guides}/partial-parsing.rst | 0 docs/{configuration => guides}/profile-config.rst | 0 docs/{configuration => guides}/project-config.rst | 0 docs/{configuration => guides}/render-config.rst | 0 .../run-dbt/airflow-worker/async-execution-mode.rst | 0 docs/{configuration => guides}/run-dbt/airflow-worker/index.rst | 0 .../run-dbt/airflow-worker/watcher-execution-mode.rst | 0 .../run-dbt/container/aws-container-run-job.rst | 0 .../run-dbt/container/azure-container-instance.rst | 0 docs/{configuration => guides}/run-dbt/container/docker.rst | 0 .../run-dbt/container/gcp-cloud-run-job.rst | 0 docs/{configuration => guides}/run-dbt/container/index.rst | 0 docs/{configuration => guides}/run-dbt/container/kubernetes.rst | 0 .../run-dbt/container/watcher-kubernetes-execution-mode.rst | 0 docs/{configuration => guides}/run-dbt/execution-modes.rst | 0 docs/{configuration => guides}/scheduling.rst | 0 docs/{configuration => guides}/selecting-excluding.rst | 0 docs/{configuration => guides}/source-nodes-rendering.rst | 0 docs/{configuration => guides}/task-display-name.rst | 0 37 files changed, 0 insertions(+), 0 deletions(-) rename docs/{configuration => guides}/caching.rst (100%) rename docs/{configuration => guides}/callbacks/callbacks.rst (100%) rename docs/{configuration => guides}/compiled-sql.rst (100%) rename docs/{configuration => guides}/configure-tests/testing-behavior.rst (100%) rename docs/{configuration => guides}/cosmos-conf.rst (100%) rename docs/{configuration => guides}/dag-customization.rst (100%) rename docs/{configuration => guides}/dbt-docs/generating-docs.rst (100%) rename docs/{configuration => guides}/dbt-docs/hosting-docs.rst (100%) rename docs/{configuration => guides}/dbt-fusion.rst (100%) rename docs/{configuration => guides}/execution-config.rst (100%) rename docs/{configuration => guides}/execution-modes-local-conflicts.rst (100%) rename docs/{configuration => guides}/index.rst (100%) rename docs/{configuration => guides}/lineage.rst (100%) rename docs/{configuration => guides}/logging.rst (100%) rename docs/{configuration => guides}/memory_optimization.rst (100%) rename docs/{configuration => guides}/multi-project.rst (100%) rename docs/{configuration => guides}/operator-args.rst (100%) rename docs/{configuration => guides}/parsing-methods.rst (100%) rename docs/{configuration => guides}/partial-parsing.rst (100%) rename docs/{configuration => guides}/profile-config.rst (100%) rename docs/{configuration => guides}/project-config.rst (100%) rename docs/{configuration => guides}/render-config.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/async-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/index.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/watcher-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/container/aws-container-run-job.rst (100%) rename docs/{configuration => guides}/run-dbt/container/azure-container-instance.rst (100%) rename docs/{configuration => guides}/run-dbt/container/docker.rst (100%) rename docs/{configuration => guides}/run-dbt/container/gcp-cloud-run-job.rst (100%) rename docs/{configuration => guides}/run-dbt/container/index.rst (100%) rename docs/{configuration => guides}/run-dbt/container/kubernetes.rst (100%) rename docs/{configuration => guides}/run-dbt/container/watcher-kubernetes-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/execution-modes.rst (100%) rename docs/{configuration => guides}/scheduling.rst (100%) rename docs/{configuration => guides}/selecting-excluding.rst (100%) rename docs/{configuration => guides}/source-nodes-rendering.rst (100%) rename docs/{configuration => guides}/task-display-name.rst (100%) diff --git a/docs/configuration/caching.rst b/docs/guides/caching.rst similarity index 100% rename from docs/configuration/caching.rst rename to docs/guides/caching.rst diff --git a/docs/configuration/callbacks/callbacks.rst b/docs/guides/callbacks/callbacks.rst similarity index 100% rename from docs/configuration/callbacks/callbacks.rst rename to docs/guides/callbacks/callbacks.rst diff --git a/docs/configuration/compiled-sql.rst b/docs/guides/compiled-sql.rst similarity index 100% rename from docs/configuration/compiled-sql.rst rename to docs/guides/compiled-sql.rst diff --git a/docs/configuration/configure-tests/testing-behavior.rst b/docs/guides/configure-tests/testing-behavior.rst similarity index 100% rename from docs/configuration/configure-tests/testing-behavior.rst rename to docs/guides/configure-tests/testing-behavior.rst diff --git a/docs/configuration/cosmos-conf.rst b/docs/guides/cosmos-conf.rst similarity index 100% rename from docs/configuration/cosmos-conf.rst rename to docs/guides/cosmos-conf.rst diff --git a/docs/configuration/dag-customization.rst b/docs/guides/dag-customization.rst similarity index 100% rename from docs/configuration/dag-customization.rst rename to docs/guides/dag-customization.rst diff --git a/docs/configuration/dbt-docs/generating-docs.rst b/docs/guides/dbt-docs/generating-docs.rst similarity index 100% rename from docs/configuration/dbt-docs/generating-docs.rst rename to docs/guides/dbt-docs/generating-docs.rst diff --git a/docs/configuration/dbt-docs/hosting-docs.rst b/docs/guides/dbt-docs/hosting-docs.rst similarity index 100% rename from docs/configuration/dbt-docs/hosting-docs.rst rename to docs/guides/dbt-docs/hosting-docs.rst diff --git a/docs/configuration/dbt-fusion.rst b/docs/guides/dbt-fusion.rst similarity index 100% rename from docs/configuration/dbt-fusion.rst rename to docs/guides/dbt-fusion.rst diff --git a/docs/configuration/execution-config.rst b/docs/guides/execution-config.rst similarity index 100% rename from docs/configuration/execution-config.rst rename to docs/guides/execution-config.rst diff --git a/docs/configuration/execution-modes-local-conflicts.rst b/docs/guides/execution-modes-local-conflicts.rst similarity index 100% rename from docs/configuration/execution-modes-local-conflicts.rst rename to docs/guides/execution-modes-local-conflicts.rst diff --git a/docs/configuration/index.rst b/docs/guides/index.rst similarity index 100% rename from docs/configuration/index.rst rename to docs/guides/index.rst diff --git a/docs/configuration/lineage.rst b/docs/guides/lineage.rst similarity index 100% rename from docs/configuration/lineage.rst rename to docs/guides/lineage.rst diff --git a/docs/configuration/logging.rst b/docs/guides/logging.rst similarity index 100% rename from docs/configuration/logging.rst rename to docs/guides/logging.rst diff --git a/docs/configuration/memory_optimization.rst b/docs/guides/memory_optimization.rst similarity index 100% rename from docs/configuration/memory_optimization.rst rename to docs/guides/memory_optimization.rst diff --git a/docs/configuration/multi-project.rst b/docs/guides/multi-project.rst similarity index 100% rename from docs/configuration/multi-project.rst rename to docs/guides/multi-project.rst diff --git a/docs/configuration/operator-args.rst b/docs/guides/operator-args.rst similarity index 100% rename from docs/configuration/operator-args.rst rename to docs/guides/operator-args.rst diff --git a/docs/configuration/parsing-methods.rst b/docs/guides/parsing-methods.rst similarity index 100% rename from docs/configuration/parsing-methods.rst rename to docs/guides/parsing-methods.rst diff --git a/docs/configuration/partial-parsing.rst b/docs/guides/partial-parsing.rst similarity index 100% rename from docs/configuration/partial-parsing.rst rename to docs/guides/partial-parsing.rst diff --git a/docs/configuration/profile-config.rst b/docs/guides/profile-config.rst similarity index 100% rename from docs/configuration/profile-config.rst rename to docs/guides/profile-config.rst diff --git a/docs/configuration/project-config.rst b/docs/guides/project-config.rst similarity index 100% rename from docs/configuration/project-config.rst rename to docs/guides/project-config.rst diff --git a/docs/configuration/render-config.rst b/docs/guides/render-config.rst similarity index 100% rename from docs/configuration/render-config.rst rename to docs/guides/render-config.rst diff --git a/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/async-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst rename to docs/guides/run-dbt/airflow-worker/async-execution-mode.rst diff --git a/docs/configuration/run-dbt/airflow-worker/index.rst b/docs/guides/run-dbt/airflow-worker/index.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/index.rst rename to docs/guides/run-dbt/airflow-worker/index.rst diff --git a/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst rename to docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst diff --git a/docs/configuration/run-dbt/container/aws-container-run-job.rst b/docs/guides/run-dbt/container/aws-container-run-job.rst similarity index 100% rename from docs/configuration/run-dbt/container/aws-container-run-job.rst rename to docs/guides/run-dbt/container/aws-container-run-job.rst diff --git a/docs/configuration/run-dbt/container/azure-container-instance.rst b/docs/guides/run-dbt/container/azure-container-instance.rst similarity index 100% rename from docs/configuration/run-dbt/container/azure-container-instance.rst rename to docs/guides/run-dbt/container/azure-container-instance.rst diff --git a/docs/configuration/run-dbt/container/docker.rst b/docs/guides/run-dbt/container/docker.rst similarity index 100% rename from docs/configuration/run-dbt/container/docker.rst rename to docs/guides/run-dbt/container/docker.rst diff --git a/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst b/docs/guides/run-dbt/container/gcp-cloud-run-job.rst similarity index 100% rename from docs/configuration/run-dbt/container/gcp-cloud-run-job.rst rename to docs/guides/run-dbt/container/gcp-cloud-run-job.rst diff --git a/docs/configuration/run-dbt/container/index.rst b/docs/guides/run-dbt/container/index.rst similarity index 100% rename from docs/configuration/run-dbt/container/index.rst rename to docs/guides/run-dbt/container/index.rst diff --git a/docs/configuration/run-dbt/container/kubernetes.rst b/docs/guides/run-dbt/container/kubernetes.rst similarity index 100% rename from docs/configuration/run-dbt/container/kubernetes.rst rename to docs/guides/run-dbt/container/kubernetes.rst diff --git a/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst rename to docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst diff --git a/docs/configuration/run-dbt/execution-modes.rst b/docs/guides/run-dbt/execution-modes.rst similarity index 100% rename from docs/configuration/run-dbt/execution-modes.rst rename to docs/guides/run-dbt/execution-modes.rst diff --git a/docs/configuration/scheduling.rst b/docs/guides/scheduling.rst similarity index 100% rename from docs/configuration/scheduling.rst rename to docs/guides/scheduling.rst diff --git a/docs/configuration/selecting-excluding.rst b/docs/guides/selecting-excluding.rst similarity index 100% rename from docs/configuration/selecting-excluding.rst rename to docs/guides/selecting-excluding.rst diff --git a/docs/configuration/source-nodes-rendering.rst b/docs/guides/source-nodes-rendering.rst similarity index 100% rename from docs/configuration/source-nodes-rendering.rst rename to docs/guides/source-nodes-rendering.rst diff --git a/docs/configuration/task-display-name.rst b/docs/guides/task-display-name.rst similarity index 100% rename from docs/configuration/task-display-name.rst rename to docs/guides/task-display-name.rst From 2e0afe7d6040e70a94eded995690adb6c797418f Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 17:26:26 -0500 Subject: [PATCH 04/48] update hradcoded URLs --- docs/getting_started/index.rst | 10 +++++----- docs/guides/execution-modes-local-conflicts.rst | 4 ++-- docs/guides/index.rst | 6 +++--- .../run-dbt/airflow-worker/watcher-execution-mode.rst | 2 +- docs/guides/run-dbt/container/kubernetes.rst | 8 ++++---- docs/index.rst | 2 +- 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 5184c867c8..add5111ccf 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -49,11 +49,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/guides/execution-modes-local-conflicts.rst b/docs/guides/execution-modes-local-conflicts.rst index 9fec173751..0f9120127c 100644 --- a/docs/guides/execution-modes-local-conflicts.rst +++ b/docs/guides/execution-modes-local-conflicts.rst @@ -10,8 +10,8 @@ When using the `Local Execution Mode `__, users may If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. +Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../guides/execution-config.html>`_ and +`RenderConfig.dbt_executable_path <../guides/render-config.html>`_ parameters. The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. diff --git a/docs/guides/index.rst b/docs/guides/index.rst index d699e6189e..df227dea90 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -1,7 +1,7 @@ -.. _configuration: +.. _guides: -Configuration -============= +Guides +====== Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. diff --git a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst index af7589650c..f33ef15900 100644 --- a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst @@ -370,7 +370,7 @@ Source freshness nodes Since Cosmos 1.6, it `supports the rendering of source nodes `_. -We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. +We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. diff --git a/docs/guides/run-dbt/container/kubernetes.rst b/docs/guides/run-dbt/container/kubernetes.rst index 607ba07bd7..4ea8ccd4b9 100644 --- a/docs/guides/run-dbt/container/kubernetes.rst +++ b/docs/guides/run-dbt/container/kubernetes.rst @@ -161,7 +161,7 @@ The Kubernetes execution mode has the following limitations: - Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) - Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) - Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) -- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) -- Does not expose Compiled SQL as a `templated field `_ -- Does not benefit from `Cosmos caching mechanisms `_ -- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) +- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) +- Does not expose Compiled SQL as a `templated field `_ +- Does not benefit from `Cosmos caching mechanisms `_ +- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/index.rst b/docs/index.rst index beee4f40bb..8926b5c198 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,7 +7,7 @@ Home Getting Started - Configuration + Guides Profiles Contributing Airflow 3 compatibility From a9af3231226d459bb93541a1ef33549235abc295 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 18:08:47 -0500 Subject: [PATCH 05/48] redistribute files --- docs/contributing.rst | 4 +++- docs/guides/{ => cosmos_devex}/compiled-sql.rst | 0 docs/guides/{ => cosmos_devex}/lineage.rst | 0 docs/guides/{ => cosmos_devex}/logging.rst | 0 docs/guides/{dbt-docs => dbt_docs}/generating-docs.rst | 0 docs/guides/{dbt-docs => dbt_docs}/hosting-docs.rst | 0 docs/guides/{ => dbt_setup}/dbt-fusion.rst | 0 docs/guides/{ => multi_project}/multi-project.rst | 0 docs/guides/{ => optimize_performance}/caching.rst | 0 .../{ => optimize_performance}/memory_optimization.rst | 0 .../{ => optimize_performance}/partial-parsing.rst | 0 .../{ => optimize_performance}/selecting-excluding.rst | 0 .../airflow-worker/async-execution-mode.rst | 0 .../airflow-worker}/execution-modes-local-conflicts.rst | 0 .../guides/{run-dbt => run_dbt}/airflow-worker/index.rst | 0 .../airflow-worker/watcher-execution-mode.rst | 0 docs/guides/{ => run_dbt}/callbacks/callbacks.rst | 0 .../container/aws-container-run-job.rst | 0 .../container/azure-container-instance.rst | 0 docs/guides/{run-dbt => run_dbt}/container/docker.rst | 0 .../{run-dbt => run_dbt}/container/gcp-cloud-run-job.rst | 0 docs/guides/{run-dbt => run_dbt}/container/index.rst | 0 .../guides/{run-dbt => run_dbt}/container/kubernetes.rst | 0 .../container/watcher-kubernetes-execution-mode.rst | 0 docs/guides/run_dbt/customization/index.rst | 9 +++++++++ .../guides/{ => run_dbt/customization}/operator-args.rst | 0 docs/guides/{ => run_dbt/customization}/scheduling.rst | 0 .../{ => run_dbt/customization}/task-display-name.rst | 0 .../{run-dbt/execution-modes.rst => run_dbt/index.rst} | 3 +++ .../run_dbt/operators}/operators.rst | 0 .../configure-tests/testing-behavior.rst | 0 .../custom-airflow-properties.rst | 0 .../{ => translate_dbt_to_airflow}/dag-customization.rst | 0 .../{ => translate_dbt_to_airflow}/parsing-methods.rst | 0 .../{ => translate_dbt_to_airflow}/render-config.rst | 0 .../source-nodes-rendering.rst | 0 docs/index.rst | 2 +- 37 files changed, 16 insertions(+), 2 deletions(-) rename docs/guides/{ => cosmos_devex}/compiled-sql.rst (100%) rename docs/guides/{ => cosmos_devex}/lineage.rst (100%) rename docs/guides/{ => cosmos_devex}/logging.rst (100%) rename docs/guides/{dbt-docs => dbt_docs}/generating-docs.rst (100%) rename docs/guides/{dbt-docs => dbt_docs}/hosting-docs.rst (100%) rename docs/guides/{ => dbt_setup}/dbt-fusion.rst (100%) rename docs/guides/{ => multi_project}/multi-project.rst (100%) rename docs/guides/{ => optimize_performance}/caching.rst (100%) rename docs/guides/{ => optimize_performance}/memory_optimization.rst (100%) rename docs/guides/{ => optimize_performance}/partial-parsing.rst (100%) rename docs/guides/{ => optimize_performance}/selecting-excluding.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/async-execution-mode.rst (100%) rename docs/guides/{ => run_dbt/airflow-worker}/execution-modes-local-conflicts.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/index.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/watcher-execution-mode.rst (100%) rename docs/guides/{ => run_dbt}/callbacks/callbacks.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/aws-container-run-job.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/azure-container-instance.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/docker.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/gcp-cloud-run-job.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/index.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/kubernetes.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/watcher-kubernetes-execution-mode.rst (100%) create mode 100644 docs/guides/run_dbt/customization/index.rst rename docs/guides/{ => run_dbt/customization}/operator-args.rst (100%) rename docs/guides/{ => run_dbt/customization}/scheduling.rst (100%) rename docs/guides/{ => run_dbt/customization}/task-display-name.rst (100%) rename docs/guides/{run-dbt/execution-modes.rst => run_dbt/index.rst} (99%) rename docs/{getting_started => guides/run_dbt/operators}/operators.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/configure-tests/testing-behavior.rst (100%) rename docs/{getting_started => guides/translate_dbt_to_airflow}/custom-airflow-properties.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/dag-customization.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/parsing-methods.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/render-config.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/source-nodes-rendering.rst (100%) diff --git a/docs/contributing.rst b/docs/contributing.rst index 006149faac..d50c120398 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -155,17 +155,19 @@ To run the checks manually, run: Writing Docs ____________ -`Hatch `_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatchto declare the dependencies required for the project itself, as well as for tests and documentation builds. +`Hatch `_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatch to declare the dependencies required for the project itself, as well as for tests and documentation builds. If you don’t already have Hatch installed, please `install it `_ before proceeding. As an example, on macOS, you can do so with: .. code-block:: bash + brew install hatch You can run the docs locally by running the following: .. code-block:: bash + hatch run docs:serve diff --git a/docs/guides/compiled-sql.rst b/docs/guides/cosmos_devex/compiled-sql.rst similarity index 100% rename from docs/guides/compiled-sql.rst rename to docs/guides/cosmos_devex/compiled-sql.rst diff --git a/docs/guides/lineage.rst b/docs/guides/cosmos_devex/lineage.rst similarity index 100% rename from docs/guides/lineage.rst rename to docs/guides/cosmos_devex/lineage.rst diff --git a/docs/guides/logging.rst b/docs/guides/cosmos_devex/logging.rst similarity index 100% rename from docs/guides/logging.rst rename to docs/guides/cosmos_devex/logging.rst diff --git a/docs/guides/dbt-docs/generating-docs.rst b/docs/guides/dbt_docs/generating-docs.rst similarity index 100% rename from docs/guides/dbt-docs/generating-docs.rst rename to docs/guides/dbt_docs/generating-docs.rst diff --git a/docs/guides/dbt-docs/hosting-docs.rst b/docs/guides/dbt_docs/hosting-docs.rst similarity index 100% rename from docs/guides/dbt-docs/hosting-docs.rst rename to docs/guides/dbt_docs/hosting-docs.rst diff --git a/docs/guides/dbt-fusion.rst b/docs/guides/dbt_setup/dbt-fusion.rst similarity index 100% rename from docs/guides/dbt-fusion.rst rename to docs/guides/dbt_setup/dbt-fusion.rst diff --git a/docs/guides/multi-project.rst b/docs/guides/multi_project/multi-project.rst similarity index 100% rename from docs/guides/multi-project.rst rename to docs/guides/multi_project/multi-project.rst diff --git a/docs/guides/caching.rst b/docs/guides/optimize_performance/caching.rst similarity index 100% rename from docs/guides/caching.rst rename to docs/guides/optimize_performance/caching.rst diff --git a/docs/guides/memory_optimization.rst b/docs/guides/optimize_performance/memory_optimization.rst similarity index 100% rename from docs/guides/memory_optimization.rst rename to docs/guides/optimize_performance/memory_optimization.rst diff --git a/docs/guides/partial-parsing.rst b/docs/guides/optimize_performance/partial-parsing.rst similarity index 100% rename from docs/guides/partial-parsing.rst rename to docs/guides/optimize_performance/partial-parsing.rst diff --git a/docs/guides/selecting-excluding.rst b/docs/guides/optimize_performance/selecting-excluding.rst similarity index 100% rename from docs/guides/selecting-excluding.rst rename to docs/guides/optimize_performance/selecting-excluding.rst diff --git a/docs/guides/run-dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/async-execution-mode.rst rename to docs/guides/run_dbt/airflow-worker/async-execution-mode.rst diff --git a/docs/guides/execution-modes-local-conflicts.rst b/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst similarity index 100% rename from docs/guides/execution-modes-local-conflicts.rst rename to docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst diff --git a/docs/guides/run-dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/index.rst rename to docs/guides/run_dbt/airflow-worker/index.rst diff --git a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst rename to docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst diff --git a/docs/guides/callbacks/callbacks.rst b/docs/guides/run_dbt/callbacks/callbacks.rst similarity index 100% rename from docs/guides/callbacks/callbacks.rst rename to docs/guides/run_dbt/callbacks/callbacks.rst diff --git a/docs/guides/run-dbt/container/aws-container-run-job.rst b/docs/guides/run_dbt/container/aws-container-run-job.rst similarity index 100% rename from docs/guides/run-dbt/container/aws-container-run-job.rst rename to docs/guides/run_dbt/container/aws-container-run-job.rst diff --git a/docs/guides/run-dbt/container/azure-container-instance.rst b/docs/guides/run_dbt/container/azure-container-instance.rst similarity index 100% rename from docs/guides/run-dbt/container/azure-container-instance.rst rename to docs/guides/run_dbt/container/azure-container-instance.rst diff --git a/docs/guides/run-dbt/container/docker.rst b/docs/guides/run_dbt/container/docker.rst similarity index 100% rename from docs/guides/run-dbt/container/docker.rst rename to docs/guides/run_dbt/container/docker.rst diff --git a/docs/guides/run-dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst similarity index 100% rename from docs/guides/run-dbt/container/gcp-cloud-run-job.rst rename to docs/guides/run_dbt/container/gcp-cloud-run-job.rst diff --git a/docs/guides/run-dbt/container/index.rst b/docs/guides/run_dbt/container/index.rst similarity index 100% rename from docs/guides/run-dbt/container/index.rst rename to docs/guides/run_dbt/container/index.rst diff --git a/docs/guides/run-dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst similarity index 100% rename from docs/guides/run-dbt/container/kubernetes.rst rename to docs/guides/run_dbt/container/kubernetes.rst diff --git a/docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst rename to docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst diff --git a/docs/guides/run_dbt/customization/index.rst b/docs/guides/run_dbt/customization/index.rst new file mode 100644 index 0000000000..44021154dc --- /dev/null +++ b/docs/guides/run_dbt/customization/index.rst @@ -0,0 +1,9 @@ +Additional Customization +======================== + +.. toctree:: + :maxdepth: 1 + :caption: Additional Customization + + operator-args + scheduling diff --git a/docs/guides/operator-args.rst b/docs/guides/run_dbt/customization/operator-args.rst similarity index 100% rename from docs/guides/operator-args.rst rename to docs/guides/run_dbt/customization/operator-args.rst diff --git a/docs/guides/scheduling.rst b/docs/guides/run_dbt/customization/scheduling.rst similarity index 100% rename from docs/guides/scheduling.rst rename to docs/guides/run_dbt/customization/scheduling.rst diff --git a/docs/guides/task-display-name.rst b/docs/guides/run_dbt/customization/task-display-name.rst similarity index 100% rename from docs/guides/task-display-name.rst rename to docs/guides/run_dbt/customization/task-display-name.rst diff --git a/docs/guides/run-dbt/execution-modes.rst b/docs/guides/run_dbt/index.rst similarity index 99% rename from docs/guides/run-dbt/execution-modes.rst rename to docs/guides/run_dbt/index.rst index a9bd3f1e2b..1b73d56a88 100644 --- a/docs/guides/run-dbt/execution-modes.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,5 +1,8 @@ .. _execution-modes: + + + Execution Modes =============== diff --git a/docs/getting_started/operators.rst b/docs/guides/run_dbt/operators/operators.rst similarity index 100% rename from docs/getting_started/operators.rst rename to docs/guides/run_dbt/operators/operators.rst diff --git a/docs/guides/configure-tests/testing-behavior.rst b/docs/guides/translate_dbt_to_airflow/configure-tests/testing-behavior.rst similarity index 100% rename from docs/guides/configure-tests/testing-behavior.rst rename to docs/guides/translate_dbt_to_airflow/configure-tests/testing-behavior.rst diff --git a/docs/getting_started/custom-airflow-properties.rst b/docs/guides/translate_dbt_to_airflow/custom-airflow-properties.rst similarity index 100% rename from docs/getting_started/custom-airflow-properties.rst rename to docs/guides/translate_dbt_to_airflow/custom-airflow-properties.rst diff --git a/docs/guides/dag-customization.rst b/docs/guides/translate_dbt_to_airflow/dag-customization.rst similarity index 100% rename from docs/guides/dag-customization.rst rename to docs/guides/translate_dbt_to_airflow/dag-customization.rst diff --git a/docs/guides/parsing-methods.rst b/docs/guides/translate_dbt_to_airflow/parsing-methods.rst similarity index 100% rename from docs/guides/parsing-methods.rst rename to docs/guides/translate_dbt_to_airflow/parsing-methods.rst diff --git a/docs/guides/render-config.rst b/docs/guides/translate_dbt_to_airflow/render-config.rst similarity index 100% rename from docs/guides/render-config.rst rename to docs/guides/translate_dbt_to_airflow/render-config.rst diff --git a/docs/guides/source-nodes-rendering.rst b/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst similarity index 100% rename from docs/guides/source-nodes-rendering.rst rename to docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst diff --git a/docs/index.rst b/docs/index.rst index 8926b5c198..fc215be0b7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,7 +2,7 @@ .. toctree:: :hidden: - :maxdepth: 2 + :maxdepth: 0 :caption: Contents: Home From 21ed49e39c1de2572861a392f95884e7fc3d1756 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 19:28:24 -0500 Subject: [PATCH 06/48] improve build errors --- docs/getting_started/index.rst | 19 +++------ docs/guides/cosmos_devex/index.rst | 14 +++++++ .../task-display-name.rst | 0 docs/guides/index.rst | 42 +++++-------------- docs/guides/optimize_performance/index.rst | 13 ++++++ docs/guides/run_dbt/index.rst | 25 ++++++++--- .../guides/translate_dbt_to_airflow/index.rst | 26 ++++++++++++ ...des-rendering.rst => managing-sources.rst} | 6 +-- 8 files changed, 91 insertions(+), 54 deletions(-) create mode 100644 docs/guides/cosmos_devex/index.rst rename docs/guides/{run_dbt/customization => cosmos_devex}/task-display-name.rst (100%) create mode 100644 docs/guides/optimize_performance/index.rst create mode 100644 docs/guides/translate_dbt_to_airflow/index.rst rename docs/guides/translate_dbt_to_airflow/{source-nodes-rendering.rst => managing-sources.rst} (97%) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index add5111ccf..9863851496 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -17,15 +17,6 @@ Google Cloud Composer (GCC) Amazon Managed Workflows for Apache Airflow (MWAA) - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Operators - - Operators - Custom Airflow Properties - Getting Started =============== @@ -49,11 +40,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/guides/cosmos_devex/index.rst b/docs/guides/cosmos_devex/index.rst new file mode 100644 index 0000000000..614e4c3c17 --- /dev/null +++ b/docs/guides/cosmos_devex/index.rst @@ -0,0 +1,14 @@ +.. _cosmos_devex: + + +Cosmos DevEx +============ + +.. toctree:: + :maxdepth: 1 + :caption: Cosmos DevEx + + lineage + compiled-sql + logging + task-display-name \ No newline at end of file diff --git a/docs/guides/run_dbt/customization/task-display-name.rst b/docs/guides/cosmos_devex/task-display-name.rst similarity index 100% rename from docs/guides/run_dbt/customization/task-display-name.rst rename to docs/guides/cosmos_devex/task-display-name.rst diff --git a/docs/guides/index.rst b/docs/guides/index.rst index df227dea90..6234012779 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -10,70 +10,48 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: Translating dbt into Airflow - Source Nodes Rendering - Post-rendering DAG customization + translate_dbt_to_airflow/index .. toctree:: :maxdepth: 3 :hidden: :caption: How Cosmos runs dbt - execution-modes-local-conflicts - run-dbt/execution-modes - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Configure tests - - configure-tests/testing-behavior - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Callbacks - - callbacks/callbacks + run_dbt/index .. toctree:: :maxdepth: 1 :hidden: :caption: Multi-project Setups - Multi-Project Setups + multi_project/multi-project .. toctree:: :maxdepth: 1 :hidden: - :caption: Operators + :caption: Documentation - Operator Args + dbt-docs/generating-docs + dbt-docs/hosting-docs .. toctree:: :maxdepth: 1 :hidden: - :caption: Documentation + :caption: Cosmos DevEx - dbt-docs/generating-docs - dbt-docs/hosting-docs + cosmos_devex/index .. toctree:: :maxdepth: 1 :hidden: :caption: Optimizing Performance - Memory Optimization - dbt Fusion - Selecting & Excluding - Parsing Methods - Partial Parsing - Caching - Render Config + optimize_performance/index .. toctree:: :maxdepth: 1 :hidden: - :caption: Configurations + :caption: Configuration References Project Config Profile Config diff --git a/docs/guides/optimize_performance/index.rst b/docs/guides/optimize_performance/index.rst new file mode 100644 index 0000000000..0ed84470d0 --- /dev/null +++ b/docs/guides/optimize_performance/index.rst @@ -0,0 +1,13 @@ +.. _optimize-performance: + +Optimize your Cosmos Performance +================================ + +.. toctree:: + :maxdepth: 1 + :caption: Optimize Performance + + partial-parsing + memory_optimization + selecting-excluding + caching diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index 1b73d56a88..2827fe39dd 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,10 +1,7 @@ .. _execution-modes: - - - -Execution Modes -=============== +How Cosmos runs dbt +=================== .. toctree:: :maxdepth: 3 @@ -18,6 +15,24 @@ Execution Modes container/index +.. toctree:: + :maxdepth: 3 + :caption: Callbacks + + callbacks/callbacks + +.. toctree:: + :maxdepth: 3 + :caption: Operators + + operators/operators + +.. toctree:: + :maxdepth: 3 + :caption: Customize Airflow + + customization/index + Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst new file mode 100644 index 0000000000..d0f8cdbefc --- /dev/null +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -0,0 +1,26 @@ +.. _translate-dbt-to-airflow + +Translate dbt code into Airflow +=============================== + +.. toctree:: + :maxdepth: 1 + :caption: Mapping dbt into dags + + parsing-methods + custom-airflow-properties + + +.. toctree:: + :maxdepth: 1 + :caption: Configure tests + + configure-tests/testing-behavior + +.. toctree:: + :maxdepth: 1 + :caption: Translate nodes + + source-nodes-rendering + render-config + dag-customization \ No newline at end of file diff --git a/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst b/docs/guides/translate_dbt_to_airflow/managing-sources.rst similarity index 97% rename from docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst rename to docs/guides/translate_dbt_to_airflow/managing-sources.rst index 9bfcf0e97b..0d7a9d9644 100644 --- a/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst +++ b/docs/guides/translate_dbt_to_airflow/managing-sources.rst @@ -1,7 +1,7 @@ -.. _source-nodes-rendering: +.. _managing-sources: -Source Nodes Rendering -====================== +Managing Sources +================ .. note:: This feature is only available for dbt-core >= 1.5 and cosmos >= 1.6.0. From d49821bb6aa34cc3339d6d92fd5def61145accff Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 19:36:24 -0500 Subject: [PATCH 07/48] fix index --- docs/guides/index.rst | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 6234012779..50f3059402 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -5,6 +5,13 @@ Guides Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Set up dbt with Airflow + + dbt_setup/dbt-fusion + .. toctree:: :maxdepth: 1 :hidden: @@ -31,8 +38,8 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: Documentation - dbt-docs/generating-docs - dbt-docs/hosting-docs + dbt_docs/generating-docs + dbt_docs/hosting-docs .. toctree:: :maxdepth: 1 @@ -56,16 +63,4 @@ Cosmos offers a number of configuration options to customize its behavior. For m Project Config Profile Config Execution Config - - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Customizing Airflow - - Configuring in Airflow - Configuring Lineage - Scheduling - Compiled SQL - Logging - Task display name \ No newline at end of file + Cosmos Config From d34f21b87337d3a496ebf4ff6dd3bcef066c1727 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 20:04:34 -0500 Subject: [PATCH 08/48] fix rel-link build errors --- docs/guides/multi_project/multi-project.rst | 4 ++-- docs/guides/optimize_performance/caching.rst | 4 ++-- .../airflow-worker/watcher-execution-mode.rst | 4 ++-- docs/guides/run_dbt/callbacks/callbacks.rst | 4 ++-- docs/guides/run_dbt/container/kubernetes.rst | 2 +- .../container/watcher-kubernetes-execution-mode.rst | 2 +- docs/guides/run_dbt/customization/scheduling.rst | 2 +- docs/guides/run_dbt/index.rst | 8 ++++---- docs/guides/run_dbt/operators/operators.rst | 4 ++-- .../configure-tests/testing-behavior.rst | 12 ++++++------ .../custom-airflow-properties.rst | 2 +- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- .../translate_dbt_to_airflow/managing-sources.rst | 2 +- .../translate_dbt_to_airflow/parsing-methods.rst | 8 ++++---- .../translate_dbt_to_airflow/render-config.rst | 2 +- 15 files changed, 31 insertions(+), 31 deletions(-) diff --git a/docs/guides/multi_project/multi-project.rst b/docs/guides/multi_project/multi-project.rst index 70283bc410..5dfd79eea0 100644 --- a/docs/guides/multi_project/multi-project.rst +++ b/docs/guides/multi_project/multi-project.rst @@ -169,7 +169,7 @@ You can use either separate DAGs or a combined DAG with task groups. **Option 1: Combined DAG with Task Groups using dbt ls Load Mode (Recommended)** -.. literalinclude:: ../../dev/dags/cross_project_dbt_ls_dag.py +.. literalinclude:: ../../../dev/dags/cross_project_dbt_ls_dag.py :language: python :start-after: [START cross_project_dbt_ls_dag] :end-before: [END cross_project_dbt_ls_dag] @@ -178,7 +178,7 @@ You can use either separate DAGs or a combined DAG with task groups. This option uses pre-generated ``manifest.json`` files for faster DAG parsing (no ``dbt ls`` execution required). -.. literalinclude:: ../../dev/dags/cross_project_manifest_dag.py +.. literalinclude:: ../../../dev/dags/cross_project_manifest_dag.py :language: python :start-after: [START cross_project_manifest_dag] :end-before: [END cross_project_manifest_dag] diff --git a/docs/guides/optimize_performance/caching.rst b/docs/guides/optimize_performance/caching.rst index 7289d00742..5bf8a6406c 100644 --- a/docs/guides/optimize_performance/caching.rst +++ b/docs/guides/optimize_performance/caching.rst @@ -84,7 +84,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] @@ -161,7 +161,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] diff --git a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst index f33ef15900..05bb21c7f7 100644 --- a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst @@ -144,7 +144,7 @@ Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` You can enable WATCHER mode directly in your ``DbtDag`` configuration. This approach is best when your Airflow DAG is fully dedicated to a dbt project. -.. literalinclude:: ../../dev/dags/example_watcher.py +.. literalinclude:: ../../../../dev/dags/example_watcher.py :language: python :start-after: [START example_watcher] :end-before: [END example_watcher] @@ -451,7 +451,7 @@ Asynchronous sensor execution To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. -.. literalinclude:: ../../dev/dags/example_watcher.py +.. literalinclude:: ../../../../dev/dags/example_watcher.py :language: python :start-after: [START example_watcher_synchronous] :end-before: [END example_watcher_synchronous] diff --git a/docs/guides/run_dbt/callbacks/callbacks.rst b/docs/guides/run_dbt/callbacks/callbacks.rst index c754245525..4b602ece3f 100644 --- a/docs/guides/run_dbt/callbacks/callbacks.rst +++ b/docs/guides/run_dbt/callbacks/callbacks.rst @@ -34,7 +34,7 @@ Example: Using Callbacks with a Single Operator To demonstrate how to specify a callback function for uploading files from the target directory, here’s an example using a single operator in an Airflow DAG: -.. literalinclude:: ../../dev/dags/example_operators.py +.. literalinclude:: ../../../../dev/dags/example_operators.py :language: python :start-after: [START single_operator_callback] :end-before: [END single_operator_callback] @@ -46,7 +46,7 @@ You can leverage the :ref:`remote_target_path` configuration to upload files from the target directory to a remote storage. Below is an example of how to define a callback helper function in your ``DbtDag`` that utilizes this configuration: -.. literalinclude:: ../../dev/dags/cosmos_callback_dag.py +.. literalinclude:: ../../../../dev/dags/cosmos_callback_dag.py :language: python :start-after: [START cosmos_callback_example] :end-before: [END cosmos_callback_example] diff --git a/docs/guides/run_dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst index 4ea8ccd4b9..d200589429 100644 --- a/docs/guides/run_dbt/container/kubernetes.rst +++ b/docs/guides/run_dbt/container/kubernetes.rst @@ -28,7 +28,7 @@ Additional KubernetesPodOperator parameters can be added to the ``operator_args` For instance, -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py +.. literalinclude:: ../../../../dev/dags/jaffle_shop_kubernetes.py :language: python :start-after: [START kubernetes_tg_example] :end-before: [END kubernetes_tg_example] diff --git a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst index 16dbbffd0a..d3f8a80a49 100644 --- a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst +++ b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst @@ -183,7 +183,7 @@ Example DAG Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: -.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py +.. literalinclude:: ../../../../dev/dags/jaffle_shop_watcher_kubernetes.py :language: python ------------------------------------------------------------------------------- diff --git a/docs/guides/run_dbt/customization/scheduling.rst b/docs/guides/run_dbt/customization/scheduling.rst index 2d4e729c5b..0040135d37 100644 --- a/docs/guides/run_dbt/customization/scheduling.rst +++ b/docs/guides/run_dbt/customization/scheduling.rst @@ -77,7 +77,7 @@ This example DAG: .. The following renders in Sphinx but not Github: -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py +.. literalinclude:: ../../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] :end-before: [END local_example] diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index 2827fe39dd..a8c96d1b93 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -127,7 +127,7 @@ When using the ``local`` execution mode, Cosmos converts Airflow Connections int Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py +.. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] :end-before: [END local_example] @@ -153,7 +153,7 @@ Some drawbacks of this approach: Example of how to use: -.. literalinclude:: ../../dev/dags/example_virtualenv.py +.. literalinclude:: ../../../dev/dags/example_virtualenv.py :language: python :start-after: [START virtualenv_example] :end-before: [END virtualenv_example] @@ -201,7 +201,7 @@ Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref: Example DAG: -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py +.. literalinclude:: ../../../dev/dags/jaffle_shop_kubernetes.py :language: python :start-after: [START kubernetes_seed_example] :end-before: [END kubernetes_seed_example] @@ -345,7 +345,7 @@ as more dbt nodes will be run in parallel since they won't be blocking Airflow's Example DAG: -.. literalinclude:: ../../dev/dags/simple_dag_async.py +.. literalinclude:: ../../../dev/dags/simple_dag_async.py :language: python :start-after: [START airflow_async_execution_mode_example] :end-before: [END airflow_async_execution_mode_example] diff --git a/docs/guides/run_dbt/operators/operators.rst b/docs/guides/run_dbt/operators/operators.rst index 9f6658b6b1..448e037e77 100644 --- a/docs/guides/run_dbt/operators/operators.rst +++ b/docs/guides/run_dbt/operators/operators.rst @@ -18,7 +18,7 @@ The ``DbtCloneLocalOperator`` implement `dbt clone Date: Sun, 1 Mar 2026 20:32:36 -0500 Subject: [PATCH 09/48] fix typo --- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst index faaf31b950..35adec52e4 100644 --- a/docs/guides/translate_dbt_to_airflow/index.rst +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -1,4 +1,4 @@ -.. _translate-dbt-to-airflow +.. _translate-dbt-to-airflow: Translate dbt code into Airflow =============================== From 7d36d9fd054411aef0553051b61abd6102ffee12 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 20:59:08 -0500 Subject: [PATCH 10/48] reformat ref structure --- docs/guides/index.rst | 13 +++----- docs/guides/run_dbt/index.rst | 33 +------------------ docs/index.rst | 1 + .../optimize_performance/caching.rst | 4 +-- .../optimize_performance/index.rst | 0 .../memory_optimization.rst | 0 .../optimize_performance/partial-parsing.rst | 0 .../selecting-excluding.rst | 0 8 files changed, 9 insertions(+), 42 deletions(-) rename docs/{guides => }/optimize_performance/caching.rst (98%) rename docs/{guides => }/optimize_performance/index.rst (100%) rename docs/{guides => }/optimize_performance/memory_optimization.rst (100%) rename docs/{guides => }/optimize_performance/partial-parsing.rst (100%) rename docs/{guides => }/optimize_performance/selecting-excluding.rst (100%) diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 50f3059402..140a9c4554 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -24,7 +24,11 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: How Cosmos runs dbt - run_dbt/index + run_dbt/airflow-worker/index + run_dbt/container/index + run_dbt/callbacks/callbacks + run_dbt/operators/operators + run_dbt/customization/index .. toctree:: :maxdepth: 1 @@ -48,13 +52,6 @@ Cosmos offers a number of configuration options to customize its behavior. For m cosmos_devex/index -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Optimizing Performance - - optimize_performance/index - .. toctree:: :maxdepth: 1 :hidden: diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index a8c96d1b93..71d581c25b 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,39 +1,8 @@ .. _execution-modes: -How Cosmos runs dbt +Execution Modes =================== -.. toctree:: - :maxdepth: 3 - :caption: Run dbt in the Airflow worker - - airflow-worker/index - -.. toctree:: - :maxdepth: 3 - :caption: Run dbt in a container - - container/index - -.. toctree:: - :maxdepth: 3 - :caption: Callbacks - - callbacks/callbacks - -.. toctree:: - :maxdepth: 3 - :caption: Operators - - operators/operators - -.. toctree:: - :maxdepth: 3 - :caption: Customize Airflow - - customization/index - - Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: 1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) diff --git a/docs/index.rst b/docs/index.rst index fc215be0b7..8af15370e0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -8,6 +8,7 @@ Home Getting Started Guides + Optimize Performance Profiles Contributing Airflow 3 compatibility diff --git a/docs/guides/optimize_performance/caching.rst b/docs/optimize_performance/caching.rst similarity index 98% rename from docs/guides/optimize_performance/caching.rst rename to docs/optimize_performance/caching.rst index 5bf8a6406c..7289d00742 100644 --- a/docs/guides/optimize_performance/caching.rst +++ b/docs/optimize_performance/caching.rst @@ -84,7 +84,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] @@ -161,7 +161,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] diff --git a/docs/guides/optimize_performance/index.rst b/docs/optimize_performance/index.rst similarity index 100% rename from docs/guides/optimize_performance/index.rst rename to docs/optimize_performance/index.rst diff --git a/docs/guides/optimize_performance/memory_optimization.rst b/docs/optimize_performance/memory_optimization.rst similarity index 100% rename from docs/guides/optimize_performance/memory_optimization.rst rename to docs/optimize_performance/memory_optimization.rst diff --git a/docs/guides/optimize_performance/partial-parsing.rst b/docs/optimize_performance/partial-parsing.rst similarity index 100% rename from docs/guides/optimize_performance/partial-parsing.rst rename to docs/optimize_performance/partial-parsing.rst diff --git a/docs/guides/optimize_performance/selecting-excluding.rst b/docs/optimize_performance/selecting-excluding.rst similarity index 100% rename from docs/guides/optimize_performance/selecting-excluding.rst rename to docs/optimize_performance/selecting-excluding.rst From 6788dbecbb151723b6d8cf0297007087ec88d4c0 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 21:13:07 -0500 Subject: [PATCH 11/48] restructure --- docs/guides/index.rst | 10 +--------- .../run_dbt/{index.rst => execution-modes.rst} | 0 docs/index.rst | 1 + docs/{guides => reference/configs}/cosmos-conf.rst | 0 .../configs}/execution-config.rst | 0 docs/reference/configs/index.rst | 13 +++++++++++++ .../configs}/profile-config.rst | 0 .../configs}/project-config.rst | 0 docs/reference/index.rst | 9 +++++++++ 9 files changed, 24 insertions(+), 9 deletions(-) rename docs/guides/run_dbt/{index.rst => execution-modes.rst} (100%) rename docs/{guides => reference/configs}/cosmos-conf.rst (100%) rename docs/{guides => reference/configs}/execution-config.rst (100%) create mode 100644 docs/reference/configs/index.rst rename docs/{guides => reference/configs}/profile-config.rst (100%) rename docs/{guides => reference/configs}/project-config.rst (100%) create mode 100644 docs/reference/index.rst diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 140a9c4554..03970c5959 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -24,6 +24,7 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: How Cosmos runs dbt + run_dbt/execution-modes run_dbt/airflow-worker/index run_dbt/container/index run_dbt/callbacks/callbacks @@ -52,12 +53,3 @@ Cosmos offers a number of configuration options to customize its behavior. For m cosmos_devex/index -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Configuration References - - Project Config - Profile Config - Execution Config - Cosmos Config diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/execution-modes.rst similarity index 100% rename from docs/guides/run_dbt/index.rst rename to docs/guides/run_dbt/execution-modes.rst diff --git a/docs/index.rst b/docs/index.rst index 8af15370e0..fc5728dc78 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -9,6 +9,7 @@ Getting Started Guides Optimize Performance + Reference Profiles Contributing Airflow 3 compatibility diff --git a/docs/guides/cosmos-conf.rst b/docs/reference/configs/cosmos-conf.rst similarity index 100% rename from docs/guides/cosmos-conf.rst rename to docs/reference/configs/cosmos-conf.rst diff --git a/docs/guides/execution-config.rst b/docs/reference/configs/execution-config.rst similarity index 100% rename from docs/guides/execution-config.rst rename to docs/reference/configs/execution-config.rst diff --git a/docs/reference/configs/index.rst b/docs/reference/configs/index.rst new file mode 100644 index 0000000000..d83b28614a --- /dev/null +++ b/docs/reference/configs/index.rst @@ -0,0 +1,13 @@ + +# Configurations - Uncomment this section to turn the page into a dropdown navigation +# ============== + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + + cosmos-conf + execution-config + profile-config + project-config \ No newline at end of file diff --git a/docs/guides/profile-config.rst b/docs/reference/configs/profile-config.rst similarity index 100% rename from docs/guides/profile-config.rst rename to docs/reference/configs/profile-config.rst diff --git a/docs/guides/project-config.rst b/docs/reference/configs/project-config.rst similarity index 100% rename from docs/guides/project-config.rst rename to docs/reference/configs/project-config.rst diff --git a/docs/reference/index.rst b/docs/reference/index.rst new file mode 100644 index 0000000000..39b5e65f62 --- /dev/null +++ b/docs/reference/index.rst @@ -0,0 +1,9 @@ +Reference +========= + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + + configs/index \ No newline at end of file From d663aec6c001074f3fc2a4db7de1649cd942449a Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 21:38:57 -0500 Subject: [PATCH 12/48] Make local exec mode page --- docs/guides/run_dbt/airflow-worker/index.rst | 2 + ...conflicts.rst => local-execution-mode.rst} | 45 ++++++--- .../airflow-worker/watcher-execution-mode.rst | 4 +- .../container/aws-container-run-job.rst | 4 +- docs/guides/run_dbt/container/docker.rst | 2 +- .../run_dbt/container/gcp-cloud-run-job.rst | 2 +- docs/guides/run_dbt/container/kubernetes.rst | 2 +- docs/guides/run_dbt/execution-modes.rst | 98 ++++++++----------- 8 files changed, 83 insertions(+), 76 deletions(-) rename docs/guides/run_dbt/airflow-worker/{execution-modes-local-conflicts.rst => local-execution-mode.rst} (71%) diff --git a/docs/guides/run_dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst index 00cb281bc8..5e102f15a9 100644 --- a/docs/guides/run_dbt/airflow-worker/index.rst +++ b/docs/guides/run_dbt/airflow-worker/index.rst @@ -5,5 +5,7 @@ Run dbt in an Airflow worker :maxdepth: 1 :caption: Run dbt in an Airflow worker + local-execution-mode + execution-modes-local-conflicts async-execution-mode watcher-execution-mode \ No newline at end of file diff --git a/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst similarity index 71% rename from docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst rename to docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index 0f9120127c..d159f0e60e 100644 --- a/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -1,19 +1,42 @@ -:orphan: +Local Execution Mode +==================== + +By default, Cosmos uses the ``local`` execution mode. + +The ``local`` execution mode is the fastest way to run Cosmos operators since they don't neither install ``dbt`` nor build docker containers. If you use managed Airflow services such as +Google Cloud Composer, you might want to use a different execution mode, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`). On an managed Airflow service, you you might not be able to install ``dbt`` in a custom path. + +The ``local`` execution mode assumes that the Airflow worker node can access a ``dbt`` binary. + +If ``dbt`` was not installed as part of the Cosmos packages, you can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. + +.. note:: + Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. + This feature is bound to `dbt partial parsing limitations `_. + Learn more: :ref:`partial-parsing`. + +When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). + +Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: + +.. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py + :language: python + :start-after: [START local_example] + :end-before: [END local_example] .. _execution-modes-local-conflicts: Airflow and dbt dependencies conflicts -====================================== +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When using the `Local Execution Mode `__, users may face dependency conflicts between -`Apache Airflow® `_ and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used. +When using the local execution mode, you can encounter dependency conflicts between +`Apache Airflow® `_ and dbt. The conflicts might increase depending on the Airflow providers and dbt adapters you use. -If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. -With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../guides/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../guides/render-config.html>`_ parameters. +If you find errors, we recommend you isolate the installation of dbt from the Airflow installation. For your Local execution mode, install dbt in a separate +Python virtualenv and set the `ExecutionConfig.dbt_executable_path <../reference/configs/execution-config.html>`_ and +`RenderConfig.dbt_executable_path <../reference/configs/render-config.html>`_ parameters. -The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. +Cosmos includes many other `execution mode methods `__ that support isolating dbt from Airflow. In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): @@ -44,7 +67,7 @@ In the following table, ``x`` represents combinations that lead to conflicts (va +---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ Examples of errors ------------------------------------ +------------------ .. code-block:: bash @@ -130,4 +153,4 @@ The table was created by running `nox `__ wi "--pre", f"apache-airflow=={airflow_version}", f"dbt-core=={dbt_version}", - ) + ) \ No newline at end of file diff --git a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst index 05bb21c7f7..94005189e8 100644 --- a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst @@ -1,7 +1,7 @@ .. _watcher-execution-mode: -Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Execution in Cosmos -============================================================================================ +``ExecutionMode.WATCHER`` (Experimental) +======================================== With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. diff --git a/docs/guides/run_dbt/container/aws-container-run-job.rst b/docs/guides/run_dbt/container/aws-container-run-job.rst index 4321c8f346..cca4d7d319 100644 --- a/docs/guides/run_dbt/container/aws-container-run-job.rst +++ b/docs/guides/run_dbt/container/aws-container-run-job.rst @@ -1,7 +1,7 @@ .. _aws-container-run-job: -Getting Started with Astronomer Cosmos on AWS ECS -================================================== +AWS ECS Execution Mode +====================== Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. Schematically, the guide will walk you through the steps required to build the following architecture: diff --git a/docs/guides/run_dbt/container/docker.rst b/docs/guides/run_dbt/container/docker.rst index 0005914886..9c30c5256c 100644 --- a/docs/guides/run_dbt/container/docker.rst +++ b/docs/guides/run_dbt/container/docker.rst @@ -1,7 +1,7 @@ .. _docker: Docker Execution Mode -======================================== +===================== The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. diff --git a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst index fa4d0c60c4..2cd1068537 100644 --- a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst +++ b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst @@ -1,7 +1,7 @@ .. _gcp-cloud-run-job: GCP Cloud Run Job Execution Mode -======================================= +================================= .. versionadded:: 1.7 This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: diff --git a/docs/guides/run_dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst index d200589429..16e9d8950c 100644 --- a/docs/guides/run_dbt/container/kubernetes.rst +++ b/docs/guides/run_dbt/container/kubernetes.rst @@ -1,7 +1,7 @@ .. _kubernetes: Kubernetes Execution Mode -============================================== +========================== The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index 71d581c25b..a3f46c52e5 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -1,23 +1,33 @@ .. _execution-modes: Execution Modes -=================== +=============== -Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: +*Execution modes* describe *where* and *how* Cosmos runs dbt commands. -1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) -2. **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos -3. **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) -4. **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) -5. **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) -6. **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) -7. **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) -8. **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) -9. **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ -10. **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. -11. **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. -The choice of the ``execution mode`` can vary based on each user's needs and concerns. For more details, check each execution mode described below. +On the Airflow worker or triggerer +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) +- **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. +- **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos +- **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ + +In a container +~~~~~~~~~~~~~~ + +You can also execute dbt commands in a container outside of the Airflow environment. + +- **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) +- **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) +- **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) +- **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) +- **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) +- **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) +- **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. + +The choice of the ``execution mode`` can vary based on your needs and concerns. .. _execution-modes-comparison: @@ -33,10 +43,18 @@ The choice of the ``execution mode`` can vary based on each user's needs and con - Fast - None - Yes + * - Watcher + - Very Fast + - None + - Yes * - Virtualenv - Medium - Lightweight - Yes + * - Airflow Async + - Very Fast + - Medium + - Yes * - Docker - Slow - Medium @@ -61,47 +79,11 @@ The choice of the ``execution mode`` can vary based on each user's needs and con - Slow - High - No - * - Airflow Async - - Very Fast - - Medium - - Yes - * - Watcher - - Very Fast - - None - - Yes * - Watcher Kubernetes - Fast - High - No -Local ------ - -By default, Cosmos uses the ``local`` execution mode. - -The ``local`` execution mode is the fastest way to run Cosmos operators since they don't install ``dbt`` nor build docker containers. However, it may not be an option for users using managed Airflow services such as -Google Cloud Composer, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`), the user may not be able to install ``dbt`` in a custom path. - -The ``local`` execution mode assumes a ``dbt`` binary is reachable within the Airflow worker node. - -If ``dbt`` was not installed as part of the Cosmos packages, -users can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. - -.. note:: - Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. - This feature is bound to `dbt partial parsing limitations `_. - Learn more: :ref:`partial-parsing`. - -When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). - -Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: - -.. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py - :language: python - :start-after: [START local_example] - :end-before: [END local_example] - - Virtualenv ---------- @@ -109,7 +91,7 @@ If you're using managed Airflow on GCP (Cloud Composer), for instance, we recomm The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. -In this case, users are responsible for declaring which version of ``dbt`` they want to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. +In this case, you are responsible for declaring which version of ``dbt`` to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. @@ -117,7 +99,7 @@ Also similar to the ``local`` execution mode, Cosmos will by default attempt to Some drawbacks of this approach: - It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. -- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. +- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, you must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. - Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. Example of how to use: @@ -130,14 +112,14 @@ Example of how to use: Docker ------ -The ``docker`` approach assumes users have a previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml``, managed by the user. +The ``docker`` approach assumes you previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml`` that you manage. The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. -If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. +If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. @@ -240,11 +222,11 @@ GCP Cloud Run Job ------------------------ .. versionadded:: 1.7 -The ``gcp_cloud_run_job`` execution mode is particularly useful for users who prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. +The ``gcp_cloud_run_job`` execution mode is particularly useful if you prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . -This execution mode allows users to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. +This execution mode allows you to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. @@ -270,7 +252,7 @@ Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. -Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. For users who require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. +Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. If you require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. Please refer to the step-by-step guide for using AWS ECS as the execution mode. @@ -303,7 +285,7 @@ Airflow Async .. versionadded:: 1.9.0 -Although this execution mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. +Although this execution mode was introduced in Cosmos 1.9, we strongly encourage you to use Cosmos 1.11, which has significant performance improvements. In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's From 00e1e0e0e3136911383489cd7beae43181535c66 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 21:48:17 -0500 Subject: [PATCH 13/48] add cosmos-managed venv --- .../airflow-worker/cosmos-managed-venv.rst | 24 +++++++++++++++++++ docs/guides/run_dbt/airflow-worker/index.rst | 6 ++--- docs/guides/run_dbt/execution-modes.rst | 24 ------------------- 3 files changed, 27 insertions(+), 27 deletions(-) create mode 100644 docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst diff --git a/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst new file mode 100644 index 0000000000..14c6f0236e --- /dev/null +++ b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst @@ -0,0 +1,24 @@ +Cosmos-managed virtual environment +---------------------------------- + +If you're using managed Airflow, we recommend you use the ``virtualenv`` execution mode. + +The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. + +In this case, you are responsible for declaring which version of ``dbt`` to use by giving the argument ``py_requirements``. Set this argument directly in operator instances or when you instantiate ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. + +Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). +Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. + +Some drawbacks of this approach: + +- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. +- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, you must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. +- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. + +Example of how to use: + +.. literalinclude:: ../../../dev/dags/example_virtualenv.py + :language: python + :start-after: [START virtualenv_example] + :end-before: [END virtualenv_example] \ No newline at end of file diff --git a/docs/guides/run_dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst index 5e102f15a9..9a0be1970f 100644 --- a/docs/guides/run_dbt/airflow-worker/index.rst +++ b/docs/guides/run_dbt/airflow-worker/index.rst @@ -6,6 +6,6 @@ Run dbt in an Airflow worker :caption: Run dbt in an Airflow worker local-execution-mode - execution-modes-local-conflicts - async-execution-mode - watcher-execution-mode \ No newline at end of file + watcher-execution-mode + cosmos-managed-venv + async-execution-mode \ No newline at end of file diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index a3f46c52e5..ec7732db92 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -84,30 +84,6 @@ The choice of the ``execution mode`` can vary based on your needs and concerns. - High - No -Virtualenv ----------- - -If you're using managed Airflow on GCP (Cloud Composer), for instance, we recommend you use the ``virtualenv`` execution mode. - -The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. - -In this case, you are responsible for declaring which version of ``dbt`` to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. - -Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). -Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. - -Some drawbacks of this approach: - -- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. -- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, you must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. -- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. - -Example of how to use: - -.. literalinclude:: ../../../dev/dags/example_virtualenv.py - :language: python - :start-after: [START virtualenv_example] - :end-before: [END virtualenv_example] Docker ------ From a0108e0079f3a5a5bd5d999a1ecf21d072bfad32 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 21:58:00 -0500 Subject: [PATCH 14/48] Execution mode section refactor --- docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst | 4 +++- docs/guides/run_dbt/airflow-worker/local-execution-mode.rst | 2 +- docs/guides/run_dbt/execution-modes.rst | 5 ++++- docs/index.rst | 2 +- docs/reference/configs/cosmos-conf.rst | 4 ++-- 5 files changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst index 14c6f0236e..96539fa623 100644 --- a/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst +++ b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst @@ -1,3 +1,5 @@ +.. _cosmos-managed-venv: + Cosmos-managed virtual environment ---------------------------------- @@ -18,7 +20,7 @@ Some drawbacks of this approach: Example of how to use: -.. literalinclude:: ../../../dev/dags/example_virtualenv.py +.. literalinclude:: ../../../../dev/dags/example_virtualenv.py :language: python :start-after: [START virtualenv_example] :end-before: [END virtualenv_example] \ No newline at end of file diff --git a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index d159f0e60e..d06396c48f 100644 --- a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -19,7 +19,7 @@ When using the ``local`` execution mode, Cosmos converts Airflow Connections int Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: -.. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py +.. literalinclude:: ../../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] :end-before: [END local_example] diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index ec7732db92..0495531671 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -31,6 +31,9 @@ The choice of the ``execution mode`` can vary based on your needs and concerns. .. _execution-modes-comparison: +Execution modes comparison +~~~~~~~~~~~~~~~~~~~~~~~~~~ + .. list-table:: Execution Modes Comparison :widths: 25 25 25 25 :header-rows: 1 @@ -304,7 +307,7 @@ Check the :ref:`watcher-kubernetes-execution-mode` for more details. .. _invocation_modes: Invocation Modes -================ +~~~~~~~~~~~~~~~~ .. versionadded:: 1.4 For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: diff --git a/docs/index.rst b/docs/index.rst index fc5728dc78..ed303c4cd7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -112,7 +112,7 @@ for managing and scaling your data workflows. Getting Started with Airflow Async Execution Mode ------------------------------------------------- -See our :doc:`Getting Started with Airflow Async Execution Mode ` for details. +See our :doc:`Getting Started with Airflow Async Execution Mode ` for details. Airflow 3 compatibility diff --git a/docs/reference/configs/cosmos-conf.rst b/docs/reference/configs/cosmos-conf.rst index cc68c3b71f..a8928c3840 100644 --- a/docs/reference/configs/cosmos-conf.rst +++ b/docs/reference/configs/cosmos-conf.rst @@ -253,14 +253,14 @@ This page lists all available Airflow configurations that affect ``astronomer-co As an example, when this option is enabled, the following is an example of specifying the imports with full module paths: - .. literalinclude:: ../../dev/dags/basic_cosmos_dag_full_module_path_imports.py + .. literalinclude:: ../../../dev/dags/basic_cosmos_dag_full_module_path_imports.py :language: python :start-after: [START cosmos_explicit_imports] :end-before: [END cosmos_explicit_imports] as opposed to the following approach you might have when this option is disabled (default): - .. literalinclude:: ../../dev/dags/basic_cosmos_dag.py + .. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START cosmos_init_imports] :end-before: [END cosmos_init_imports] From 68377ff562bf6038a9415ffec9cf88da91ccbf3a Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 2 Mar 2026 03:11:28 +0000 Subject: [PATCH 15/48] =?UTF-8?q?=F0=9F=8E=A8=20[pre-commit.ci]=20Auto=20f?= =?UTF-8?q?ormat=20from=20pre-commit.com=20hooks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/guides/cosmos_devex/index.rst | 2 +- docs/guides/index.rst | 1 - docs/guides/run_dbt/airflow-worker/index.rst | 2 +- docs/guides/run_dbt/container/index.rst | 2 +- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- docs/reference/configs/index.rst | 2 +- docs/reference/index.rst | 2 +- 7 files changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/guides/cosmos_devex/index.rst b/docs/guides/cosmos_devex/index.rst index 614e4c3c17..2ad3dff71b 100644 --- a/docs/guides/cosmos_devex/index.rst +++ b/docs/guides/cosmos_devex/index.rst @@ -11,4 +11,4 @@ Cosmos DevEx lineage compiled-sql logging - task-display-name \ No newline at end of file + task-display-name diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 03970c5959..9b7e233814 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -52,4 +52,3 @@ Cosmos offers a number of configuration options to customize its behavior. For m :caption: Cosmos DevEx cosmos_devex/index - diff --git a/docs/guides/run_dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst index 00cb281bc8..eaa89c2d9f 100644 --- a/docs/guides/run_dbt/airflow-worker/index.rst +++ b/docs/guides/run_dbt/airflow-worker/index.rst @@ -6,4 +6,4 @@ Run dbt in an Airflow worker :caption: Run dbt in an Airflow worker async-execution-mode - watcher-execution-mode \ No newline at end of file + watcher-execution-mode diff --git a/docs/guides/run_dbt/container/index.rst b/docs/guides/run_dbt/container/index.rst index 634e9e8eb4..9cccdbb29a 100644 --- a/docs/guides/run_dbt/container/index.rst +++ b/docs/guides/run_dbt/container/index.rst @@ -10,4 +10,4 @@ Run dbt in a container docker gcp-cloud-run-job kubernetes - watcher-kubernetes-execution-mode \ No newline at end of file + watcher-kubernetes-execution-mode diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst index 35adec52e4..5ff278003e 100644 --- a/docs/guides/translate_dbt_to_airflow/index.rst +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -23,4 +23,4 @@ Translate dbt code into Airflow managing-sources render-config - dag-customization \ No newline at end of file + dag-customization diff --git a/docs/reference/configs/index.rst b/docs/reference/configs/index.rst index d83b28614a..741b7a76bd 100644 --- a/docs/reference/configs/index.rst +++ b/docs/reference/configs/index.rst @@ -10,4 +10,4 @@ cosmos-conf execution-config profile-config - project-config \ No newline at end of file + project-config diff --git a/docs/reference/index.rst b/docs/reference/index.rst index 39b5e65f62..4c430eb4d8 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -6,4 +6,4 @@ Reference :hidden: :caption: Configurations - configs/index \ No newline at end of file + configs/index From a8737d8d29094842e11e77b75a641ff77ea5b4e5 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 17:38:35 -0500 Subject: [PATCH 16/48] Update docker --- docs/guides/run_dbt/container/docker.rst | 86 +++++++++++++++--------- docs/guides/run_dbt/execution-modes.rst | 33 +-------- 2 files changed, 56 insertions(+), 63 deletions(-) diff --git a/docs/guides/run_dbt/container/docker.rst b/docs/guides/run_dbt/container/docker.rst index 9c30c5256c..91ac3166a7 100644 --- a/docs/guides/run_dbt/container/docker.rst +++ b/docs/guides/run_dbt/container/docker.rst @@ -3,26 +3,40 @@ Docker Execution Mode ===================== -The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. +The ``docker`` approach assumes you previously created Docker image, which contains all the ``dbt`` pipelines and a ``profiles.yml`` that you manage. + +Performance and maintenance considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can have better environment isolation with ``docker`` than when using ``local`` or ``virtualenv`` modes, but this mode also requires more maintenance and has some performance tradeoffs, depending on your project configurations. + +Using ``docker`` requires that you ensure the Docker container you use has up-to-date files and you might potentially need to manage secrets in multiple places. Another challenge of working with ``docker`` occurs when the Airflow worker is already running in Docker, which can cause problems related to running `Docker in Docker `__. + +Also, the Docker execution mode approach can be significantly slower than ``virtualenv``, since it might require building the ``Docker`` container before executing dbt commands, which is slower than creating a Virtualenv with ``dbt-core``. + + +Set up Docker execution mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following procedure illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. Requirements -++++++++++++ +------------ -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Astronomer-cosmos package containing the dbt Docker operators -4. Postgres docker container -5. Docker image built with required dbt project and dbt DAG -6. dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow +- Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -More information on how to achieve 2-6 is detailed below. +The following example setup steps include setting up the following: -Step-by-step instructions -+++++++++++++++++++++++++ +- Airflow +- Astronomer-cosmos package containing the dbt Docker operators +- Postgres docker container +- Docker image built with required dbt project and dbt DAG +- dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow -**Install Airflow and Cosmos** +1. Install Airflow and Cosmos +----------------------------- -Create a python virtualenv, activate it, upgrade pip to the latest version and install `Apache Airflow® `_ & astronomer-postgres +Create a python virtualenv, activate it, upgrade pip to the latest version, and install `Apache Airflow® `_ & ``astronomer-postgres``: .. code-block:: bash @@ -32,26 +46,28 @@ Create a python virtualenv, activate it, upgrade pip to the latest version and i pip install apache-airflow pip install "astronomer-cosmos[dbt-postgres]" -**Setup Postgres database** +2. Set up Postgres database +--------------------------- -You will need a postgres database running to be used as the database for the dbt project. Run the following command to run and expose a postgres database +You will need a postgres database running to use as the database for the dbt project. Run the following command to run and expose a postgres database: .. code-block:: bash docker run --name some-postgres -e POSTGRES_PASSWORD="" -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p5432:5432 -d postgres -**Build the dbt Docker image** +3. Build the dbt Docker image +----------------------------- For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. -Clone the `cosmos-example `_ repo +1. Clone the `cosmos-example `_ repo .. code-block:: bash git clone https://github.com/astronomer/cosmos-example.git cd cosmos-example -Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. +2. Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. .. code-block:: bash @@ -59,7 +75,7 @@ Create a docker image containing the dbt project files and dbt profile by using .. note:: - If running on M1, you may need to set the following envvars for running the docker build command in case it fails + If running on M1, you may need to set the following environment variables for running the docker build command, in case it fails. .. code-block:: bash @@ -67,21 +83,25 @@ Create a docker image containing the dbt project files and dbt profile by using export COMPOSE_DOCKER_CLI_BUILD=0 export DOCKER_DEFAULT_PLATFORM=linux/amd64 -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. +Read the following example Dockerfiles to understand what it does so that you can use them as a project reference. + +- The `dbt profile `_ file is added to the image. +- The ``dags`` directory containing the `dbt project jaffle_shop `_ is added to the image. +- The ``dbt_project.yml`` is replaced with `postgres_profile_dbt_project.yml `_, which contains the profile key pointing to ``postgres_profile`` because profile creation is not handled for K8s operators, like in local mode. - - The `dbt profile `_ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. -**Setup and Trigger the DAG with Airflow** +4. Set up and trigger the Dag with Airflow +------------------------------------------ -Copy the dags directory from cosmos-example repo to your Airflow home +1. Copy the ``dags`` directory from the ``cosmos-example`` repo to your Airflow home .. code-block:: bash cp -r dags $AIRFLOW_HOME/ -Run Airflow +This directory contains a Docker-specific example Dag. + +2. Run Airflow .. code-block:: bash @@ -89,18 +109,17 @@ Run Airflow .. note:: - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the ``Kind`` cluster. -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. +3. Log in to Airflow through a web browser, ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. -Enable and trigger a run of the `jaffle_shop_docker `_ DAG. You will be able to see the following successful DAG run. +4. Enable and trigger a run of the `jaffle_shop_docker `_ Dag. You can see the following successful Dag run example: .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_docker_dag_run.png :width: 800 - Specifying ProfileConfig -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~ Starting with Cosmos 1.8.0, you can use the ``profile_config`` argument in your Dbt DAG Docker operators to reference profiles for your dbt project defined in a profiles.yml file. To do so, provide the file’s path via the @@ -109,3 +128,8 @@ profiles for your dbt project defined in a profiles.yml file. To do so, provide Note that in ``ExecutionMode.DOCKER``, the ``profile_config`` is only compatible with the ``profiles_yml_path`` approach. The ``profile_mapping`` method will not work because the required Airflow connections cannot be accessed within the Docker container to map them to the dbt profile. + +Troubleshooting +~~~~~~~~~~~~~~~ + +If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, you must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index 0495531671..f272e2e44f 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -19,7 +19,7 @@ In a container You can also execute dbt commands in a container outside of the Airflow environment. -- **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) +- `docker `_ : Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) - **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) - **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) - **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) @@ -87,37 +87,6 @@ Execution modes comparison - High - No - -Docker ------- - -The ``docker`` approach assumes you previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml`` that you manage. - -The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). - -The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. - -This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. -If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. - -Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. - -Example DAG: - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.DOCKER, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "network_mode": "bridge", - }, - ) - - Kubernetes ---------- From 6d6828c34bd789804a06667b962542cde2dcbd01 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:05:01 -0500 Subject: [PATCH 17/48] add eks --- docs/guides/run_dbt/container/aws-eks.rst | 34 +++++++++++++++++++++++ docs/guides/run_dbt/container/index.rst | 10 ++++--- 2 files changed, 40 insertions(+), 4 deletions(-) create mode 100644 docs/guides/run_dbt/container/aws-eks.rst diff --git a/docs/guides/run_dbt/container/aws-eks.rst b/docs/guides/run_dbt/container/aws-eks.rst new file mode 100644 index 0000000000..18318e9ede --- /dev/null +++ b/docs/guides/run_dbt/container/aws-eks.rst @@ -0,0 +1,34 @@ +.. _aws-eks: + +Amazon Elastic Kubernetes Service (AWS EKS) +=========================================== + +The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. +It uses the `EKSPodOperator `_ +to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. + + +Example DAG + +.. code-block:: python + + postgres_password_secret = Secret( + deploy_type="env", + deploy_target="POSTGRES_PASSWORD", + secret="postgres-secrets", + key="password", + ) + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AWS_EKS, + ), + operator_args={ + "image": "dbt-jaffle-shop:1.0.0", + "cluster_name": CLUSTER_NAME, + "get_logs": True, + "is_delete_operator_pod": False, + "secrets": [postgres_password_secret], + }, + ) \ No newline at end of file diff --git a/docs/guides/run_dbt/container/index.rst b/docs/guides/run_dbt/container/index.rst index 634e9e8eb4..ba31890637 100644 --- a/docs/guides/run_dbt/container/index.rst +++ b/docs/guides/run_dbt/container/index.rst @@ -5,9 +5,11 @@ Run dbt in a container :maxdepth: 1 :caption: Run dbt in a container - aws-container-run-job - azure-container-instance docker - gcp-cloud-run-job kubernetes - watcher-kubernetes-execution-mode \ No newline at end of file + watcher-kubernetes-execution-mode + azure-container-instance + aws-container-run-job + aws-eks + gcp-cloud-run-job + From 0038103fb674dba7b30af44fbd624a7d06bbebe4 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:18:02 -0500 Subject: [PATCH 18/48] async mode --- .../airflow-worker/async-execution-mode.rst | 45 +++--- docs/guides/run_dbt/execution-modes.rst | 131 ++---------------- 2 files changed, 34 insertions(+), 142 deletions(-) diff --git a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst index 55d6778abc..3c11620867 100644 --- a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst @@ -3,18 +3,21 @@ Airflow Async Execution Mode ============================ -This execution mode can reduce the runtime by 35% in comparison to Cosmos LOCAL execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. +This execution mode can reduce the runtime by 35% in comparison to Cosmos ``LOCAL`` execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. -It can be particularly useful for long-running transformations, since it leverages Airflow's `deferrable operators `__. +The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's +`Deferrable operators `__. +This execution mode is well-suited for when you have long-running resources and you want to run them asynchronously by +leveraging Airflow's deferrable operators. With deferrable operators, you can potentially observe higher throughput of tasks +because more dbt nodes run in parallel, since they won't be blocking Airflow's worker slots. -In this mode, there is a ``SetupAsyncOperator`` that will pre-generate the SQL files for the dbt project and upload them to Airflow XCom or a remote location. A remote location will only be used if users set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator is run before the remaining pipeline. -All the pipeline dbt model transformations will be run using ``DbtRunAirflowAsyncOperator`` which, instead of running the ``dbt run`` command for each model. They will download the SQL files from the Airflow XCom or remote location and execute them directly leveraging the Airflow ``BigQueryInsertJobOperator``. - -Users can leverage other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. +In this mode, there is a ``SetupAsyncOperator`` that pre-generates the SQL files for the dbt project and uploads them to Airflow XCom or a remote location. Airflow only uses a remote location if you set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator runs before the remaining pipeline. +All the pipeline dbt model transformations run using ``DbtRunAirflowAsyncOperator`` instead of running the ``dbt run`` command for each model. They download the SQL files from the Airflow XCom or remote location, and then execute them directly using the Airflow ``BigQueryInsertJobOperator``. +You can also use other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. Advantages of Airflow Async Mode -++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **Improved Task Throughput:** Async tasks free up Airflow workers by leveraging the Airflow Trigger framework. While long-running SQL transformations are executing in the data warehouse, the worker is released and can handle other tasks, increasing overall task throughput. - **Better Resource Utilization:** By minimizing idle time on Airflow workers, async tasks allow more efficient use of compute resources. Workers aren't blocked waiting for external systems and can be reused for other work while waiting on async operations. @@ -34,18 +37,18 @@ We have `observed `_ Getting Started with Airflow Async Mode -+++++++++++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This guide walks you through setting up an Astro CLI project and running a Cosmos-based DAG with a deferrable operator, enabling asynchronous task execution in Apache Airflow. Prerequisites -+++++++++++++ +------------- - `Astro CLI `_ - Airflow>=2.9 1. Create Astro-CLI Project -+++++++++++++++++++++++++++ +--------------------------- Run the following command in your terminal: @@ -70,7 +73,7 @@ This will create an Astro project with the following structure: 2. Update Dockerfile -++++++++++++++++++++ +-------------------- Edit your Dockerfile to ensure all necessary requirements are included. @@ -80,7 +83,7 @@ Edit your Dockerfile to ensure all necessary requirements are included. 3. Add astronomer-cosmos Dependency -+++++++++++++++++++++++++++++++++++ +----------------------------------- In your ``requirements.txt``, add: @@ -90,7 +93,7 @@ In your ``requirements.txt``, add: 4. Create Airflow DAG -+++++++++++++++++++++ +--------------------- 1. Create a new DAG file: ``dags/cosmos_async_dag.py`` @@ -152,8 +155,8 @@ In your ``requirements.txt``, add: - Add a valid dbt project inside your Airflow project under ``dags/dbt/``. -5. Start the Project -++++++++++++++++++++ +5. Start the project +-------------------- Launch the Airflow project locally: @@ -166,8 +169,8 @@ This will: - Spin up the scheduler, webserver, and triggerer (needed for deferrable operators) - Expose Airflow UI at http://localhost:8080 -6. Create Airflow Connection -++++++++++++++++++++++++++++ +6. Create Airflow connection +---------------------------- Create an Airflow connection with following configurations @@ -196,7 +199,7 @@ Create an Airflow connection with following configurations 7. Execute the DAG -++++++++++++++++++ +------------------ 1. Visit the Airflow UI at ``http://localhost:8080`` 2. Enable the DAG: ``cosmos_async_dag`` @@ -209,8 +212,8 @@ Create an Airflow connection with following configurations The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks. -Control of where to upload the SQL files -++++++++++++++++++++++++++++++++++++++++ +Control where to upload the SQL files +------------------------------------- For optimal performance we encourage to keep Cosmos standard behaviour (introduced in 1.11), which is to upload the SQL files to XCom, instead of a remote object location. @@ -225,7 +228,7 @@ However, if you want to upload the SQL files to a remote object location instead Limitations -+++++++++++ +----------- 1. **Limited to dbt models**: Only dbt resource type models are run asynchronously using Airflow deferrable operators. Other resource types are executed synchronously, similar to the local execution mode. diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index f272e2e44f..e656fd5094 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -5,14 +5,13 @@ Execution Modes *Execution modes* describe *where* and *how* Cosmos runs dbt commands. - On the Airflow worker or triggerer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) -- **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. -- **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos -- **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ +- `local `_: Run ``dbt`` commands using a local ``dbt`` installation (default) +- `watcher `_: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. +- `virtualenv `_: Run ``dbt`` commands from Python virtual environments managed by Cosmos +- `airflow_async `_: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ In a container ~~~~~~~~~~~~~~ @@ -20,12 +19,12 @@ In a container You can also execute dbt commands in a container outside of the Airflow environment. - `docker `_ : Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) -- **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) -- **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) -- **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) -- **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) -- **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) -- **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. +- `kubernetes `_: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) +- `watcher_kubernetes `_: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. +- `azure_container_instance `_: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) +- `aws_ecs `_: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) +- `aws_eks `_: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) +- `gcp_cloud_run_job `_: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) The choice of the ``execution mode`` can vary based on your needs and concerns. @@ -105,39 +104,6 @@ Example DAG: :start-after: [START kubernetes_seed_example] :end-before: [END kubernetes_seed_example] -AWS_EKS ----------- - -The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. -It uses the `EKSPodOperator `_ -to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. - - -Example DAG: - -.. code-block:: python - - postgres_password_secret = Secret( - deploy_type="env", - deploy_target="POSTGRES_PASSWORD", - secret="postgres-secrets", - key="password", - ) - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AWS_EKS, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "cluster_name": CLUSTER_NAME, - "get_logs": True, - "is_delete_operator_pod": False, - "secrets": [postgres_password_secret], - }, - ) - Azure Container Instance ------------------------ .. versionadded:: 1.4 @@ -225,80 +191,3 @@ Please refer to the step-by-step guide for using AWS ECS as the execution mode. "environment_variables": {"DBT_PROFILE_NAME": "default"}, }, ) - -.. _airflow-async-execution-mode: - -Airflow Async -------------- - -.. versionadded:: 1.9.0 - -Although this execution mode was introduced in Cosmos 1.9, we strongly encourage you to use Cosmos 1.11, which has significant performance improvements. -In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. - -The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's -`Deferrable operators `__. -This execution mode could be preferred when you've long running resources and you want to run them asynchronously by -leveraging Airflow's deferrable operators. With that, you would be able to potentially observe higher throughput of tasks -as more dbt nodes will be run in parallel since they won't be blocking Airflow's worker slots. - -Example DAG: - -.. literalinclude:: ../../../dev/dags/simple_dag_async.py - :language: python - :start-after: [START airflow_async_execution_mode_example] - :end-before: [END airflow_async_execution_mode_example] - -For a full step-by-step guide and limitations, check the :ref:`async-execution-mode` page. - - -Watcher Execution Mode (Experimental) -------------------------------------- - -.. versionadded:: 1.11.0 - -The ``watcher`` execution mode is an experimental execution mode that runs a single ``dbt build`` command from a producer task and has sensor tasks to watch the progress of the producer. -It is designed to improve DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. - -Check the :ref:`watcher-execution-mode` for more details. - - -Watcher Kubernetes Execution Mode (Experimental) ------------------------------------------------- - -.. versionadded:: 1.13.0 - -The ``watcher_kubernetes`` execution mode combines the speed of the ``watcher`` execution mode with the isolation of the ``kubernetes`` execution mode. It runs a single ``dbt build`` command from a producer task inside a Kubernetes pod and has sensor tasks to watch the progress of the producer. - -Check the :ref:`watcher-kubernetes-execution-mode` for more details. - - -.. _invocation_modes: - -Invocation Modes -~~~~~~~~~~~~~~~~ -.. versionadded:: 1.4 - -For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: - -1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. - -2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ - In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ - This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. - -The invocation mode can be set in the ``ExecutionConfig`` as shown below: - -.. code-block:: python - - from cosmos.constants import InvocationMode - - dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.LOCAL, - invocation_mode=InvocationMode.DBT_RUNNER, - ), - ) - -If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. From 798eee7f4c715b13cd04fbcf12be09a57de6bb9e Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:18:15 -0500 Subject: [PATCH 19/48] invocation modes --- docs/optimize_performance/index.rst | 1 + .../optimize_performance/invocation-modes.rst | 30 +++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 docs/optimize_performance/invocation-modes.rst diff --git a/docs/optimize_performance/index.rst b/docs/optimize_performance/index.rst index 0ed84470d0..89f28cc168 100644 --- a/docs/optimize_performance/index.rst +++ b/docs/optimize_performance/index.rst @@ -10,4 +10,5 @@ Optimize your Cosmos Performance partial-parsing memory_optimization selecting-excluding + invocation_modes caching diff --git a/docs/optimize_performance/invocation-modes.rst b/docs/optimize_performance/invocation-modes.rst new file mode 100644 index 0000000000..4404327d09 --- /dev/null +++ b/docs/optimize_performance/invocation-modes.rst @@ -0,0 +1,30 @@ +.. _invocation_modes: + +Invocation Modes +================ + +.. versionadded:: 1.4 + +For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: + +1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. + +2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ + In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ + This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. + +The invocation mode can be set in the ``ExecutionConfig`` as shown below: + +.. code-block:: python + + from cosmos.constants import InvocationMode + + dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.LOCAL, + invocation_mode=InvocationMode.DBT_RUNNER, + ), + ) + +If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. From aa3c90d0ca156c7ef31c1748c3f7dbc287db37cb Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:24:44 -0500 Subject: [PATCH 20/48] kubernetes E.M. --- docs/guides/run_dbt/container/kubernetes.rst | 19 ++++++++++++++++--- docs/guides/run_dbt/execution-modes.rst | 17 ----------------- 2 files changed, 16 insertions(+), 20 deletions(-) diff --git a/docs/guides/run_dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst index 16e9d8950c..991aa29e12 100644 --- a/docs/guides/run_dbt/container/kubernetes.rst +++ b/docs/guides/run_dbt/container/kubernetes.rst @@ -3,13 +3,26 @@ Kubernetes Execution Mode ========================== +The ``kubernetes`` execution mode provides a very isolated method to run ``dbt`` from within a Kubernetes Pod, usually in a separate host. + +Performance and maintenance considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This execution mode assumes you have a Kubernetes cluster. It also expects you to ensure the Docker container has up-to-date ``dbt`` pipelines and profiles, potentially leading you to declare secrets in two places; Airflow and Docker container. + +The ``Kubernetes`` deployment might be slower than ``Docker`` and ``Virtualenv``, assuming that the container image is built (which is slower than creating a Python ``virtualenv`` and installing ``dbt-core``) and the Airflow task needs to spin up a new ``Pod`` in Kubernetes. + + +Set up Kubernetes execution mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: - Postgres is run in the Kubernetes (K8s) cluster as a container - Airflow is run locally, and it triggers a K8s Pod which runs dbt Requirements -++++++++++++ +~~~~~~~~~~~~ To test the DbtKubernetesOperators locally, we encourage you to install the following: @@ -34,7 +47,7 @@ For instance, :end-before: [END kubernetes_tg_example] Step-by-step instructions -+++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~ Using installed `Kind `_, you can setup a local kubernetes cluster @@ -153,7 +166,7 @@ Enable and trigger a run of the `jaffle_shop_k8s Date: Mon, 2 Mar 2026 19:24:50 -0500 Subject: [PATCH 21/48] Local execution mode --- docs/guides/run_dbt/airflow-worker/local-execution-mode.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index d06396c48f..fc0f611873 100644 --- a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -1,3 +1,5 @@ +.. _local-execution: + Local Execution Mode ==================== From cd3bfcc8df8a0bb25f6936467081c4224399b810 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:29:42 -0500 Subject: [PATCH 22/48] Azure container instance --- .../container/azure-container-instance.rst | 23 +++++++++++---- docs/guides/run_dbt/execution-modes.rst | 28 ------------------- 2 files changed, 18 insertions(+), 33 deletions(-) diff --git a/docs/guides/run_dbt/container/azure-container-instance.rst b/docs/guides/run_dbt/container/azure-container-instance.rst index 86ce3ab9ef..089bef0c99 100644 --- a/docs/guides/run_dbt/container/azure-container-instance.rst +++ b/docs/guides/run_dbt/container/azure-container-instance.rst @@ -4,13 +4,26 @@ Azure Container Instance Execution Mode ======================================= .. versionadded:: 1.4 -This tutorial will guide you through the steps required to use Azure Container Instance as the Execution Mode for your dbt code with Astronomer Cosmos. Schematically, the guide will walk you through the steps required to build the following architecture: +Using ``Azure Container Instances`` as the execution mode provides an isolated way of running ``dbt``, since the ``dbt`` run itself occurs within a container running in an Azure Container Instance. + +Performance and maintenance considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task creates a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. + +Setup +~~~~~ + +This tutorial guides you through the steps required to use Azure Container Instance as the execution mode for your dbt code with Astronomer Cosmos. Schematically, the guide demonstrates how to build the following architecture: .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aci_schematic.png :width: 800 Prerequisites -+++++++++++++ +~~~~~~~~~~~~~ + 1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. 2. Airflow 3. Azure CLI (install guide here: `Azure CLI `_) @@ -28,7 +41,7 @@ More information on how to achieve 2-6 is detailed below. Note that the steps below will walk you through an example, for which the code can be found HERE Step-by-step guide -++++++++++++++++++ +~~~~~~~~~~~~~~~~~~ **Install Airflow and Cosmos** @@ -103,7 +116,7 @@ Take a read of the Dockerfile to understand what it does so that you could use i - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. -**Setup Airflow Connections** +**Set up Airflow Connections** Now you have the required Azure infrastructure, you still need to add configuration to Airflow to ensure the infrastructure can be used. You'll need 3 connections: 1. ``aci_db``: a Postgres connection to your Azure Postgres instance. @@ -112,7 +125,7 @@ Now you have the required Azure infrastructure, you still need to add configurat Check out the ``airflow-settings.yml`` file `here `_ for an example. If you are using Astro CLI, filling in the right values here will be enough for this to work. -**Setup and Trigger the DAG with Airflow** +**Set up and trigger the Dag with Airflow** Copy the dags directory from cosmos-example repo to your Airflow home diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index c1c94e59a4..766a54394b 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -87,34 +87,6 @@ Execution modes comparison - No -Azure Container Instance ------------------------- -.. versionadded:: 1.4 - -Similar to the ``kubernetes`` approach, using ``Azure Container Instances`` as the execution mode gives a very isolated way of running ``dbt``, since the ``dbt`` run itself is run within a container running in an Azure Container Instance. - -This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. Please checkout the step-by-step guide for using Azure Container Instance as the execution mode - - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AZURE_CONTAINER_INSTANCE - ), - operator_args={ - "ci_conn_id": "aci", - "registry_conn_id": "acr", - "resource_group": "my-rg", - "name": "my-aci-{{ ti.task_id.replace('.','-').replace('_','-') }}", - "region": "West Europe", - "image": "dbt-jaffle-shop:1.0.0", - }, - ) - GCP Cloud Run Job ------------------------ .. versionadded:: 1.7 From ed87bbc176334474f477436d0e784e1143848268 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:32:05 -0500 Subject: [PATCH 23/48] fix filepath --- .../{invocation-modes.rst => invocation_modes.rst} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/optimize_performance/{invocation-modes.rst => invocation_modes.rst} (100%) diff --git a/docs/optimize_performance/invocation-modes.rst b/docs/optimize_performance/invocation_modes.rst similarity index 100% rename from docs/optimize_performance/invocation-modes.rst rename to docs/optimize_performance/invocation_modes.rst From 387b4d6a65c823382210512d07cfdbebf368aa90 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:37:05 -0500 Subject: [PATCH 24/48] Execution modes restructure --- .../container/aws-container-run-job.rst | 16 ++++- .../run_dbt/container/gcp-cloud-run-job.rst | 15 +++++ docs/guides/run_dbt/execution-modes.rst | 60 ------------------- 3 files changed, 30 insertions(+), 61 deletions(-) diff --git a/docs/guides/run_dbt/container/aws-container-run-job.rst b/docs/guides/run_dbt/container/aws-container-run-job.rst index cca4d7d319..6afc95ee75 100644 --- a/docs/guides/run_dbt/container/aws-container-run-job.rst +++ b/docs/guides/run_dbt/container/aws-container-run-job.rst @@ -3,7 +3,21 @@ AWS ECS Execution Mode ====================== -Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. +.. versionadded:: 1.9.0 + +Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. + +Performance and maintenance considerations +++++++++++++++++++++++++++++++++++++++++++ + +This execution mode requires you to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task creates a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. If you require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. + +Setup ++++++ + +In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. Schematically, the guide will walk you through the steps required to build the following architecture: .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aws_ecs_schematic.png diff --git a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst index 2cd1068537..9afeb7b54e 100644 --- a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst +++ b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst @@ -4,6 +4,21 @@ GCP Cloud Run Job Execution Mode ================================= .. versionadded:: 1.7 +The ``gcp_cloud_run_job`` execution mode is particularly useful if you prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. + +Performance and maintenance considerations +++++++++++++++++++++++++++++++++++++++++++ + +For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . + +This execution mode allows you to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. + + +Setup ++++++ + This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_gcp_crj_schematic.png diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst index 766a54394b..261c8e63c3 100644 --- a/docs/guides/run_dbt/execution-modes.rst +++ b/docs/guides/run_dbt/execution-modes.rst @@ -86,63 +86,3 @@ Execution modes comparison - High - No - -GCP Cloud Run Job ------------------------- -.. versionadded:: 1.7 - -The ``gcp_cloud_run_job`` execution mode is particularly useful if you prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. - -For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . - -This execution mode allows you to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. - - -.. code-block:: python - - gcp_cloud_run_job_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.GCP_CLOUD_RUN_JOB), - operator_args={ - "project_id": "my-gcp-project-id", - "region": "europe-west1", - "job_name": "my-crj-{{ ti.task_id.replace('.','-').replace('_','-') }}", - }, - ) - - -AWS ECS ---------- -.. versionadded:: 1.9.0 - -Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. - -This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. If you require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. - -Please refer to the step-by-step guide for using AWS ECS as the execution mode. - -.. code-block:: python - - aws_ecs_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.AWS_ECS), - operator_args={ - "aws_conn_id": "aws_default", - "cluster": "my-ecs-cluster", - "task_definition": "my-dbt-task", - "container_name": "dbt-container", - "launch_type": "FARGATE", - "deferrable": True, - "network_configuration": { - "awsvpcConfiguration": { - "subnets": ["<<>>"], - "assignPublicIp": "ENABLED", - }, - }, - "environment_variables": {"DBT_PROFILE_NAME": "default"}, - }, - ) From e914a2d5909e1a41a86417fe9d42b585ce503fe4 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Mon, 2 Mar 2026 19:42:30 -0500 Subject: [PATCH 25/48] fix formatting --- .../run_dbt/airflow-worker/local-execution-mode.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index fc0f611873..d01a56e6b3 100644 --- a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -117,11 +117,11 @@ Examples of errors .. code-block:: bash -ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. + ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. -The conflict is caused by: - dbt-core 1.10.0 depends on pydantic<2 - apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 + The conflict is caused by: + dbt-core 1.10.0 depends on pydantic<2 + apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 From 66b4afd497acd701c7298fcd40ee12935bff756f Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 16:38:12 -0500 Subject: [PATCH 26/48] Address consistency feedback --- docs/guides/run_dbt/airflow-worker/async-execution-mode.rst | 2 +- docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst | 4 ++-- docs/guides/run_dbt/airflow-worker/local-execution-mode.rst | 2 +- .../run_dbt/airflow-worker/watcher-execution-mode.rst | 2 +- docs/guides/run_dbt/container/aws-eks.rst | 6 +++--- docs/guides/run_dbt/container/azure-container-instance.rst | 2 +- docs/guides/run_dbt/container/docker.rst | 2 +- docs/guides/run_dbt/container/gcp-cloud-run-job.rst | 2 +- docs/guides/run_dbt/container/kubernetes.rst | 2 +- .../run_dbt/container/watcher-kubernetes-execution-mode.rst | 4 ++-- docs/optimize_performance/invocation_modes.rst | 2 +- 11 files changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst index 3c11620867..003dc34d52 100644 --- a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst @@ -1,6 +1,6 @@ .. _async-execution-mode: -Airflow Async Execution Mode +Airflow async execution mode ============================ This execution mode can reduce the runtime by 35% in comparison to Cosmos ``LOCAL`` execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. diff --git a/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst index 96539fa623..7a4863b9b1 100644 --- a/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst +++ b/docs/guides/run_dbt/airflow-worker/cosmos-managed-venv.rst @@ -1,7 +1,7 @@ .. _cosmos-managed-venv: -Cosmos-managed virtual environment ----------------------------------- +Cosmos-managed virtual environment execution mode +======================================================== If you're using managed Airflow, we recommend you use the ``virtualenv`` execution mode. diff --git a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index d01a56e6b3..55ab6c2b47 100644 --- a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -1,6 +1,6 @@ .. _local-execution: -Local Execution Mode +Local execution mode ==================== By default, Cosmos uses the ``local`` execution mode. diff --git a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst index 94005189e8..77451bec56 100644 --- a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst @@ -1,6 +1,6 @@ .. _watcher-execution-mode: -``ExecutionMode.WATCHER`` (Experimental) +Watcher execution mode (Experimental) ======================================== With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. diff --git a/docs/guides/run_dbt/container/aws-eks.rst b/docs/guides/run_dbt/container/aws-eks.rst index 18318e9ede..9894089d43 100644 --- a/docs/guides/run_dbt/container/aws-eks.rst +++ b/docs/guides/run_dbt/container/aws-eks.rst @@ -1,9 +1,9 @@ .. _aws-eks: -Amazon Elastic Kubernetes Service (AWS EKS) -=========================================== +AWS EKS execution mode +======================= -The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. +The Amazon Elastic Kubernetes Service (AWS EKS), ``aws_eks``, approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. It uses the `EKSPodOperator `_ to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. diff --git a/docs/guides/run_dbt/container/azure-container-instance.rst b/docs/guides/run_dbt/container/azure-container-instance.rst index 089bef0c99..e79ad3ffa0 100644 --- a/docs/guides/run_dbt/container/azure-container-instance.rst +++ b/docs/guides/run_dbt/container/azure-container-instance.rst @@ -1,6 +1,6 @@ .. _azure-container-instance: -Azure Container Instance Execution Mode +Azure Container Instance execution mode ======================================= .. versionadded:: 1.4 diff --git a/docs/guides/run_dbt/container/docker.rst b/docs/guides/run_dbt/container/docker.rst index 91ac3166a7..143e05f549 100644 --- a/docs/guides/run_dbt/container/docker.rst +++ b/docs/guides/run_dbt/container/docker.rst @@ -1,6 +1,6 @@ .. _docker: -Docker Execution Mode +Docker execution mode ===================== The ``docker`` approach assumes you previously created Docker image, which contains all the ``dbt`` pipelines and a ``profiles.yml`` that you manage. diff --git a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst index 9afeb7b54e..090f9aa395 100644 --- a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst +++ b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst @@ -1,6 +1,6 @@ .. _gcp-cloud-run-job: -GCP Cloud Run Job Execution Mode +GCP Cloud Run Job execution mode ================================= .. versionadded:: 1.7 diff --git a/docs/guides/run_dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst index 991aa29e12..163fd54f1f 100644 --- a/docs/guides/run_dbt/container/kubernetes.rst +++ b/docs/guides/run_dbt/container/kubernetes.rst @@ -1,6 +1,6 @@ .. _kubernetes: -Kubernetes Execution Mode +Kubernetes execution mode ========================== The ``kubernetes`` execution mode provides a very isolated method to run ``dbt`` from within a Kubernetes Pod, usually in a separate host. diff --git a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst index d3f8a80a49..7843e32fba 100644 --- a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst +++ b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst @@ -1,7 +1,7 @@ .. _watcher-kubernetes-execution-mode: -``ExecutionMode.WATCHER_KUBERNETES``: High-Performance dbt Execution in Kubernetes -=================================================================================== +Watcher Kubernetes execution mode +================================= .. versionadded:: 1.13.0 diff --git a/docs/optimize_performance/invocation_modes.rst b/docs/optimize_performance/invocation_modes.rst index 4404327d09..97b85ce66e 100644 --- a/docs/optimize_performance/invocation_modes.rst +++ b/docs/optimize_performance/invocation_modes.rst @@ -5,7 +5,7 @@ Invocation Modes .. versionadded:: 1.4 -For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: +For ``ExecutionMode.LOCAL`` and ``ExecutionMode.WATCHER`` execution mode, Cosmos supports two invocation modes for running dbt: 1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. From 01981c304a97f6b656350311198f4f526f8c76a4 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 16:48:07 -0500 Subject: [PATCH 27/48] fix title capitalization --- docs/guides/run_dbt/container/aws-container-run-job.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/run_dbt/container/aws-container-run-job.rst b/docs/guides/run_dbt/container/aws-container-run-job.rst index 6afc95ee75..a720cab4b7 100644 --- a/docs/guides/run_dbt/container/aws-container-run-job.rst +++ b/docs/guides/run_dbt/container/aws-container-run-job.rst @@ -1,6 +1,6 @@ .. _aws-container-run-job: -AWS ECS Execution Mode +AWS ECS execution mode ====================== .. versionadded:: 1.9.0 From 72dfe27d4da800def3472b05ab18c9fd8c1b9c1c Mon Sep 17 00:00:00 2001 From: Laura Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 20:16:00 -0500 Subject: [PATCH 28/48] Apply suggestions from code review Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com> --- docs/guides/run_dbt/airflow-worker/async-execution-mode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst index 003dc34d52..ec098b86a1 100644 --- a/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst @@ -3,7 +3,7 @@ Airflow async execution mode ============================ -This execution mode can reduce the runtime by 35% in comparison to Cosmos ``LOCAL`` execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. +This execution mode can reduce the runtime by 35% in comparison to Cosmos ``LOCAL`` execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use the latest version of Cosmos, which has significant performance improvements. The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's `Deferrable operators `__. From cb511ea0b471c574b6400afa9da34630fb51c257 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:52:27 -0500 Subject: [PATCH 29/48] Create config subfolder structure --- .../{ => callbacks}/callbacks.rst | 0 .../testing-behavior.rst | 0 .../{ => dbt-docs}/generating-docs.rst | 0 .../{ => dbt-docs}/hosting-docs.rst | 0 .../execution-modes-local-conflicts.rst | 133 +++++ docs/configuration/index.rst | 89 +++- .../airflow-worker/async-execution-mode.rst | 247 +++++++++ .../run-dbt/airflow-worker/index.rst | 9 + .../airflow-worker/watcher-execution-mode.rst | 480 ++++++++++++++++++ .../container/aws-container-run-job.rst | 191 +++++++ .../container/azure-container-instance.rst | 138 +++++ .../run-dbt/container/docker.rst | 111 ++++ .../run-dbt/container/gcp-cloud-run-job.rst | 265 ++++++++++ .../configuration/run-dbt/container/index.rst | 13 + .../run-dbt/container/kubernetes.rst | 167 ++++++ .../watcher-kubernetes-execution-mode.rst | 214 ++++++++ .../configuration/run-dbt/execution-modes.rst | 387 ++++++++++++++ 17 files changed, 2428 insertions(+), 16 deletions(-) rename docs/configuration/{ => callbacks}/callbacks.rst (100%) rename docs/configuration/{ => configure-tests}/testing-behavior.rst (100%) rename docs/configuration/{ => dbt-docs}/generating-docs.rst (100%) rename docs/configuration/{ => dbt-docs}/hosting-docs.rst (100%) create mode 100644 docs/configuration/execution-modes-local-conflicts.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/index.rst create mode 100644 docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst create mode 100644 docs/configuration/run-dbt/container/aws-container-run-job.rst create mode 100644 docs/configuration/run-dbt/container/azure-container-instance.rst create mode 100644 docs/configuration/run-dbt/container/docker.rst create mode 100644 docs/configuration/run-dbt/container/gcp-cloud-run-job.rst create mode 100644 docs/configuration/run-dbt/container/index.rst create mode 100644 docs/configuration/run-dbt/container/kubernetes.rst create mode 100644 docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst create mode 100644 docs/configuration/run-dbt/execution-modes.rst diff --git a/docs/configuration/callbacks.rst b/docs/configuration/callbacks/callbacks.rst similarity index 100% rename from docs/configuration/callbacks.rst rename to docs/configuration/callbacks/callbacks.rst diff --git a/docs/configuration/testing-behavior.rst b/docs/configuration/configure-tests/testing-behavior.rst similarity index 100% rename from docs/configuration/testing-behavior.rst rename to docs/configuration/configure-tests/testing-behavior.rst diff --git a/docs/configuration/generating-docs.rst b/docs/configuration/dbt-docs/generating-docs.rst similarity index 100% rename from docs/configuration/generating-docs.rst rename to docs/configuration/dbt-docs/generating-docs.rst diff --git a/docs/configuration/hosting-docs.rst b/docs/configuration/dbt-docs/hosting-docs.rst similarity index 100% rename from docs/configuration/hosting-docs.rst rename to docs/configuration/dbt-docs/hosting-docs.rst diff --git a/docs/configuration/execution-modes-local-conflicts.rst b/docs/configuration/execution-modes-local-conflicts.rst new file mode 100644 index 0000000000..9fec173751 --- /dev/null +++ b/docs/configuration/execution-modes-local-conflicts.rst @@ -0,0 +1,133 @@ +:orphan: + +.. _execution-modes-local-conflicts: + +Airflow and dbt dependencies conflicts +====================================== + +When using the `Local Execution Mode `__, users may face dependency conflicts between +`Apache Airflow® `_ and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used. + +If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. +With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate +Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and +`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. + +The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. + +In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): + ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| Airflow / DBT | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | ++===============+=====+=====+=====+=====+=====+=====+=====+=====+=====+=====+======+ +| 2.2 | | | | x | x | x | x | x | x | x | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.3 | x | x | | x | x | x | x | x | x | x | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.4 | x | x | x | | | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.5 | x | x | x | | | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.6 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.7 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.8 | x | x | x | x | x | | x | | | | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.9 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.10 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 2.11 | x | x | x | x | x | | | | | | | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ +| 3.0 | x | x | x | x | x | x | x | x | | | x | ++---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ + +Examples of errors +----------------------------------- + +.. code-block:: bash + + The conflict is caused by: + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10 + apache-airflow 2.8.0 depends on pydantic>=2.3.0 + dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10 + + +.. code-block:: bash + + ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies. + The conflict is caused by: + apache-airflow 2.2.4 depends on jinja2<3.1 and >=2.10.1 + dbt-core 1.5.0 depends on Jinja2==3.1.2 + +.. code-block:: bash + + ERROR: Cannot install apache-airflow==2.6.0 and dbt-core because these package versions have conflicting dependencies. + The conflict is caused by: + apache-airflow 2.6.0 depends on importlib-metadata<5.0.0 and >=1.7; python_version < "3.9" + dbt-semantic-interfaces 0.1.0.dev7 depends on importlib-metadata==6.6.0 + +.. code-block:: bash + + ERROR: Cannot install apache-airflow, apache-airflow==2.7.0 and dbt-core==1.4.0 because these package versions have conflicting dependencies. + + The conflict is caused by: + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.12.0 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.2 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.1 depends on PyYAML<6 and >=5.1 + dbt-core 1.4.0 depends on pyyaml>=6.0 + connexion 2.11.0 depends on PyYAML<6 and >=5.1 + apache-airflow 2.7.0 depends on jsonschema>=4.18.0 + flask-appbuilder 4.3.3 depends on jsonschema<5 and >=3 + connexion 2.10.0 depends on jsonschema<4 and >=2.5.1 + +.. code-block:: bash + +ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. + +The conflict is caused by: + dbt-core 1.10.0 depends on pydantic<2 + apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 + + + +How to reproduce +---------------- + +The table was created by running `nox `__ with the following ``noxfile.py``: + +.. code-block:: python + + import nox + + nox.options.sessions = ["compatibility"] + nox.options.reuse_existing_virtualenvs = True + + + @nox.session(python=["3.10"]) + @nox.parametrize( + "dbt_version", + ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9", "1.10"], + ) + @nox.parametrize( + "airflow_version", + ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "3.0"], + ) + def compatibility(session: nox.Session, airflow_version, dbt_version) -> None: + """Run both unit and integration tests.""" + session.run( + "pip3", + "install", + "--pre", + f"apache-airflow=={airflow_version}", + f"dbt-core=={dbt_version}", + ) diff --git a/docs/configuration/index.rst b/docs/configuration/index.rst index a6042327b0..d699e6189e 100644 --- a/docs/configuration/index.rst +++ b/docs/configuration/index.rst @@ -6,31 +6,88 @@ Configuration Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. .. toctree:: - :caption: Contents: + :maxdepth: 1 + :hidden: + :caption: Translating dbt into Airflow + + Source Nodes Rendering + Post-rendering DAG customization + +.. toctree:: + :maxdepth: 3 + :hidden: + :caption: How Cosmos runs dbt + + execution-modes-local-conflicts + run-dbt/execution-modes + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configure tests + + configure-tests/testing-behavior + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Callbacks + + callbacks/callbacks + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Multi-project Setups - dbt Fusion Multi-Project Setups +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Operators + + Operator Args + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Documentation + + dbt-docs/generating-docs + dbt-docs/hosting-docs + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Optimizing Performance + + Memory Optimization + dbt Fusion + Selecting & Excluding + Parsing Methods + Partial Parsing + Caching + Render Config + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + Project Config Profile Config Execution Config - Render Config - Parsing Methods + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Customizing Airflow + Configuring in Airflow Configuring Lineage - Generating Docs - Hosting Docs Scheduling - Testing Behavior - Selecting & Excluding - Partial Parsing - Source Nodes Rendering - Post-rendering DAG customization - Operator Args Compiled SQL Logging - Caching - Task display name - Callbacks - Memory Optimization + Task display name \ No newline at end of file diff --git a/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst b/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst new file mode 100644 index 0000000000..55d6778abc --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst @@ -0,0 +1,247 @@ +.. _async-execution-mode: + +Airflow Async Execution Mode +============================ + +This execution mode can reduce the runtime by 35% in comparison to Cosmos LOCAL execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. + +It can be particularly useful for long-running transformations, since it leverages Airflow's `deferrable operators `__. + +In this mode, there is a ``SetupAsyncOperator`` that will pre-generate the SQL files for the dbt project and upload them to Airflow XCom or a remote location. A remote location will only be used if users set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator is run before the remaining pipeline. +All the pipeline dbt model transformations will be run using ``DbtRunAirflowAsyncOperator`` which, instead of running the ``dbt run`` command for each model. They will download the SQL files from the Airflow XCom or remote location and execute them directly leveraging the Airflow ``BigQueryInsertJobOperator``. + +Users can leverage other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. + + +Advantages of Airflow Async Mode +++++++++++++++++++++++++++++++++ + +- **Improved Task Throughput:** Async tasks free up Airflow workers by leveraging the Airflow Trigger framework. While long-running SQL transformations are executing in the data warehouse, the worker is released and can handle other tasks, increasing overall task throughput. +- **Better Resource Utilization:** By minimizing idle time on Airflow workers, async tasks allow more efficient use of compute resources. Workers aren't blocked waiting for external systems and can be reused for other work while waiting on async operations. +- **Faster Task Execution:** With Cosmos ``SetupAsyncOperator``, the SQL transformations are precompiled and uploaded to XCom (default behaviour) or a remote location. Instead of invoking a full dbt run during each dbt model task, the SQL files are downloaded from this XCom or remote path and executed directly. This eliminates unnecessary overhead from running the full dbt command, resulting in faster and more efficient task execution. + +We have `observed `_ the following performance improvements by running a dbt project with 129 models: + ++----------------------------------------------+--------------------------+ +| How the dbt pipeline was executed | Execution Time (seconds) | ++==============================================+==========================+ +| ``dbt run`` with dbt Core 1.10 | 13 | ++----------------------------------------------+--------------------------+ +| Cosmos 1.11 with ExecutionMode.LOCAL | 11 | ++----------------------------------------------+--------------------------+ +| Cosmos 1.11 with ExecutionMode.AIRFLOW_ASYNC | 7 | ++----------------------------------------------+--------------------------+ + + +Getting Started with Airflow Async Mode ++++++++++++++++++++++++++++++++++++++++ + +This guide walks you through setting up an Astro CLI project and running a Cosmos-based DAG with a deferrable operator, enabling asynchronous task execution in Apache Airflow. + +Prerequisites ++++++++++++++ + +- `Astro CLI `_ +- Airflow>=2.9 + +1. Create Astro-CLI Project ++++++++++++++++++++++++++++ + +Run the following command in your terminal: + +.. code-block:: bash + + astro dev init + +This will create an Astro project with the following structure: + +.. code-block:: bash + + . + ├── Dockerfile + ├── README.md + ├── airflow_settings.yaml + ├── dags/ + ├── include/ + ├── packages.txt + ├── plugins/ + ├── requirements.txt + └── tests/ + + +2. Update Dockerfile +++++++++++++++++++++ + +Edit your Dockerfile to ensure all necessary requirements are included. + +.. code-block:: bash + + FROM astrocrpublic.azurecr.io/runtime:3.0-2 + + +3. Add astronomer-cosmos Dependency ++++++++++++++++++++++++++++++++++++ + +In your ``requirements.txt``, add: + +.. code-block:: bash + + astronomer-cosmos[dbt-bigquery, google]>=1.9 + + +4. Create Airflow DAG ++++++++++++++++++++++ + +1. Create a new DAG file: ``dags/cosmos_async_dag.py`` + +- Update the ``dataset`` and ``project`` + +.. code-block:: python + + import os + from datetime import datetime + from pathlib import Path + + from cosmos import ( + DbtDag, + ExecutionConfig, + ExecutionMode, + ProfileConfig, + ProjectConfig, + ) + from cosmos.constants import TestBehavior + from cosmos.profiles import GoogleCloudServiceAccountDictProfileMapping + + DEFAULT_DBT_ROOT_PATH = Path(__file__).resolve().parent / "dbt" + DBT_ROOT_PATH = Path(os.getenv("DBT_ROOT_PATH", DEFAULT_DBT_ROOT_PATH)) + DBT_ADAPTER_VERSION = os.getenv("DBT_ADAPTER_VERSION", "1.9") + + cosmos_async_dag = DbtDag( + project_config=ProjectConfig( + DBT_ROOT_PATH / "jaffle_shop", + ), + profile_config=ProfileConfig( + profile_name="default", + target_name="dev", + profile_mapping=GoogleCloudServiceAccountDictProfileMapping( + conn_id="gcp_conn", + profile_args={ + "dataset": "cosmos_async_demo", + "project": "astronomer-**", + }, + ), + ), + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AIRFLOW_ASYNC, + async_py_requirements=[f"dbt-bigquery=={DBT_ADAPTER_VERSION}"], + ), + schedule=None, + start_date=datetime(2025, 1, 1), + catchup=False, + dag_id="cosmos_async_dag", + operator_args={ + "location": "US", + "install_deps": True, + "full_refresh": True, + "virtualenv_dir": "dbt_venv", + }, + ) + +2. Folder structure for dbt project + +- Add a valid dbt project inside your Airflow project under ``dags/dbt/``. + + +5. Start the Project +++++++++++++++++++++ + +Launch the Airflow project locally: + +.. code-block:: bash + + astro dev start + +This will: + +- Spin up the scheduler, webserver, and triggerer (needed for deferrable operators) +- Expose Airflow UI at http://localhost:8080 + +6. Create Airflow Connection +++++++++++++++++++++++++++++ + +Create an Airflow connection with following configurations + +- Connection ID: gcp_conn +- Connection Type: google_cloud_platform +- Extra Fields JSON: + +.. code-block:: bash + + { + "project": "astronomer-**", + "keyfile_dict": { + "type": "***", + "project_id": "***", + "private_key_id": "***", + "private_key": "***", + "client_email": "***", + "client_id": "***", + "auth_uri": "***", + "token_uri": "***", + "auth_provider_x509_cert_url": "***", + "client_x509_cert_url": "***", + "universe_domain": "***" + } + } + + +7. Execute the DAG +++++++++++++++++++ + +1. Visit the Airflow UI at ``http://localhost:8080`` +2. Enable the DAG: ``cosmos_async_dag`` +3. Trigger the DAG manually + +.. image:: /_static/jaffle_shop_async_execution_mode.png + :alt: Cosmos dbt Async DAG + :align: center + +The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks. + + +Control of where to upload the SQL files +++++++++++++++++++++++++++++++++++++++++ + +For optimal performance we encourage to keep Cosmos standard behaviour (introduced in 1.11), which is to upload the SQL files to XCom, instead of a remote object location. + +For the benchmark example described in a previous section, there was an overhead of ~500 seconds with remote SQL file upload/download, but only ~2 seconds using XCom, which can outweigh the performance improvements introduced by using deferrable operators. + +However, if you want to upload the SQL files to a remote object location instead of XCom, you can set the following environment variables: + +.. code-block:: bash + + AIRFLOW__COSMOS__REMOTE_TARGET_PATH=gs://cosmos_remote_target_demo + AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID=gcp_conn + + +Limitations ++++++++++++ + + +1. **Limited to dbt models**: Only dbt resource type models are run asynchronously using Airflow deferrable operators. Other resource types are executed synchronously, similar to the local execution mode. + +2. **BigQuery support only**: This mode only supports BigQuery as the target database. If a different target is specified, Cosmos will throw an error indicating the target database is unsupported in this mode. Adding support for other adapters is on the roadmap. + +3. **ProfileMapping parameter required**: You need to specify the ``ProfileMapping`` parameter in the ``ProfileConfig`` for your DAG. Refer to the example DAG below for details on setting this parameter. + +4. **Location parameter required**: You must specify the location of the BigQuery dataset in the ``operator_args`` of the ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. + +5. **async_py_requirements parameter required**: If you're using the default approach of having a setup task, you must specify the necessary dbt adapter Python requirements based on your profile type for the async execution mode in the ``ExecutionConfig`` of your ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. + +6. **Creation of new isolated virtual environment for each task run**: By default, the ``SetupAsyncOperator`` creates and executes within a new isolated virtual environment for each task run, which can cause performance issues. To reuse an existing virtual environment, use the ``virtualenv_dir`` parameter within the ``operator_args`` of the ``DbtDag``. We have observed that for ``dbt-bigquery``, the ``SetupAsyncOperator`` executes approximately 30% faster when reusing an existing virtual environment, particularly for transformations that take around 10 minutes to complete. + +7. **Performance degradation when uploading to remote object location**: Even though it is possible to upload the SQL files to a remote object location by setting environment variables, it is slow. We observed that this introduces a significant overhead in the execution time (500s for 129 models). + +8. **TeardownAsyncOperator limitation**: When using a remote object location, in addition to the ``SetupAsyncOperator``, a ``TeardownAsyncOperator`` is also added to the DAG. This task will delete the SQL files from the remote location by the end of the DAG Run. This is can lead to a limitation from a retry perspective, as described in the issue `#2066 `_. This can be avoided by setting the ``enable_teardown_async_task`` configuration to ``False``, as described in the :ref:`enable_teardown_async_task` section. + +For a comparison between different Cosmos execution modes, please, check the :ref:`execution-modes-comparison` section. diff --git a/docs/configuration/run-dbt/airflow-worker/index.rst b/docs/configuration/run-dbt/airflow-worker/index.rst new file mode 100644 index 0000000000..00cb281bc8 --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/index.rst @@ -0,0 +1,9 @@ +Run dbt in an Airflow worker +============================ + +.. toctree:: + :maxdepth: 1 + :caption: Run dbt in an Airflow worker + + async-execution-mode + watcher-execution-mode \ No newline at end of file diff --git a/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst new file mode 100644 index 0000000000..af7589650c --- /dev/null +++ b/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst @@ -0,0 +1,480 @@ +.. _watcher-execution-mode: + +Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Execution in Cosmos +============================================================================================ + +With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. + +Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_ and Airflow deferrable sensors, the performance gains will depend on three major factors: + +- The amount of dbt ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag +- The topology of the dbt pipeline +- The ``poke_interval`` and ``timeout`` settings of the ``DbtConsumerWatcherSensor`` operator, which determine the frequency and duration of the sensor's polling. + +------------------------------------------------------------------------------- + +Background: The Problem with the Local Execution Mode in Cosmos +--------------------------------------------------------------- + +When running dbt via Cosmos using the default ``ExecutionMode.LOCAL``, each dbt model is executed as a separate Airflow task. + +This provides strong observability and task-level retry control — but it comes at a cost. Each model runs a new dbt process, which introduces significant overhead. + +Consider the `google/fhir-dbt-analytics `_ project: + ++-------------------------------------------------------------+-----------------------------------+------------------+ +| Run Type | Description | Total Runtime | ++=============================================================+===================================+==================+ +| Single ``dbt run`` (dbt CLI) | Runs the whole DAG in one command | ~5m 30s | ++-------------------------------------------------------------+-----------------------------------+------------------+ +| One ``dbt run`` per model, totalling 184 commands (dbt CLI) | Each model is its own task | ~32m | ++-------------------------------------------------------------+-----------------------------------+------------------+ + +This difference motivated a rethinking of how Cosmos interacts with dbt. + +------------------------------------------------------------------------------- + +Concept: ``ExecutionMode.WATCHER`` +---------------------------------- + +``ExecutionMode.WATCHER`` combines the **speed of a single dbt run** with the **observability and task management of Airflow**. + +It is built on two operator types: + +* ``DbtProducerWatcherOperator`` (`#1982 `_) + Runs dbt **once** across the entire pipeline, register to `dbt event callbacks `_ and sends model progress updates via Airflow **XComs**. + +* ``DbtConsumerWatcherSensor`` (`#1998 `_) + Watches those XComs and marks individual Airflow tasks as complete when their corresponding dbt models finish. + +Together, these operators let you: + +* Run dbt as a single command (for speed) +* Retain model-level observability (for clarity) +* Retry specific models (for resilience) + +------------------------------------------------------------------------------- + +Performance Gains +----------------- + +We used a dbt project developed by Google, the `google/fhir-dbt-analytics `_ project, that interfaces with BigQuery. It contains: +* 2 seeds +* 52 sources +* 185 models + +Initial benchmarks, using illustrate significant improvements: + ++-----------------------------------------------+-----------+--------------------+ +| Environment | Threads | Execution Time (s) | ++===============================================+===========+====================+ +| dbt build (dbt CLI) | 4 | 6–7 | ++-----------------------------------------------+-----------+--------------------+ +| dbt run per model (dbt CLI) | — | 30 | +| similar to the Cosmos ``ExecutionMode.LOCAL`` | | | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.LOCAL`` (Astro CLI) | — | 10–15 | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.WATCHER`` (Astro CLI) | 1 | 26 | +| | 2 | 14 | +| | 4 | 7 | +| | 8 | 4 | +| | 16 | 2 | ++-----------------------------------------------+-----------+--------------------+ +| Cosmos ``ExecutionMode.WATCHER`` (Astro Cloud | 8 | ≈5 | +| Standard Deployment with A10 workers | | | ++-----------------------------------------------+-----------+--------------------+ + +The last line represents the performance improvement in a real-world Airflow deployment, using `Astro Cloud `_. + +Depending on the dbt workflow topology, if your dbt DAG previously took 5 minutes with ``ExecutionMode.LOCAL``, you can expect it to complete in roughly **1 minute** with ``ExecutionMode.WATCHER``. + +We plan to repeat these benchmarks and share the code with the community in the future. + + +.. note:: + ``ExecutionMode.WATCHER`` relies on the ``threads`` value defined in your dbt profile. Start with a conservative value that matches the CPU capacity of your Airflow workers, then gradually increase it to find the sweet spot between faster runs and acceptable memory/CPU usage. + +When we ran the `astronomer/cosmos-benchmark `_ project with ``ExecutionMode.WATCHER``, that same ``threads`` setting directly affected runtime: moving from 1 to 8 threads reduced the end-to-end ``dbt build`` duration from roughly 26 seconds to about 4 seconds (see table above), while 16 threads squeezed it to around 2 seconds at the cost of higher CPU usage. Use those numbers as a reference point when evaluating how thread counts scale in your own environment. + +To increase the number of threads, edit your dbt ``profiles.yml`` (or Helm values if you manage the profile there) and update the ``threads`` key for the target you use with Cosmos: + +.. code-block:: yaml + + your_dbt_project: + target: prod + outputs: + prod: + type: postgres + host: your-host + user: your-user + password: your-password + schema: analytics + threads: 8 # increase or decrease to match available resources + + +If you prefer to manage threads through Cosmos profile mappings instead of editing ``profiles.yml`` directly, pass ``profile_args={"threads": }`` to your ``ProfileConfig``. For example, using the built-in ``PostgresUserPasswordProfileMapping``: + +.. code-block:: python + + from cosmos.config import ProfileConfig + from cosmos.profiles import PostgresUserPasswordProfileMapping + + profile_config = ProfileConfig( + profile_name="jaffle_shop", + target_name="prod", + profile_mapping=PostgresUserPasswordProfileMapping( + conn_id="postgres_connection", + profile_args={"threads": 8}, + ), + ) + + +------------------------------------------------------------------------------- + +Example Usage of ``ExecutionMode.WATCHER`` +------------------------------------------ + +There are two main ways to use the new execution mode in Cosmos — directly within a ``DbtDag``, or embedded as part of a ``DbtTaskGroup`` inside a larger DAG. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can enable WATCHER mode directly in your ``DbtDag`` configuration. +This approach is best when your Airflow DAG is fully dedicated to a dbt project. + +.. literalinclude:: ../../dev/dags/example_watcher.py + :language: python + :start-after: [START example_watcher] + :end-before: [END example_watcher] + +As it can be observed, the only difference with the default ``ExecutionMode.LOCAL`` is the addition of the ``execution_config`` parameter with the ``execution_mode`` set to ``ExecutionMode.WATCHER``. The ``ExecutionMode`` enum can be imported from ``cosmos.constants``. For more information on the ``ExecutionMode.LOCAL``, please, check the `dedicated page `__ + +**How it works:** + +* Cosmos executes your dbt project once via a producer task. +* Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. +* The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). + +**How it looks like:** + +.. image:: /_static/jaffle_shop_watcher_dbt_dag_dag_run.png + :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` + :align: center + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Example 2 — Using ``DbtTaskGroup`` with ``ExecutionMode.WATCHER`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If your Airflow DAG includes multiple stages or integrations (e.g., data ingestion → dbt → reporting), use ``DbtTaskGroup`` to embed your dbt project into a larger DAG — still benefiting from WATCHER performance. + +.. code-block:: python + :caption: example_watcher_taskgroup.py + :name: example_watcher_taskgroup + + from airflow.models import DAG + from airflow.operators.empty import EmptyOperator + from cosmos import DbtTaskGroup + + with DAG( + dag_id="example_watcher_taskgroup", + schedule="@daily", + start_date=datetime(2023, 1, 1), + catchup=False, + ): + """ + The simplest example of using Cosmos to render a dbt project as a TaskGroup. + """ + pre_dbt = EmptyOperator(task_id="pre_dbt") + + first_dbt_task_group = DbtTaskGroup( + group_id="first_dbt_task_group", + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER, + ), + project_config=ProjectConfig(DBT_PROJECT_PATH), + profile_config=profile_config, + operator_args=operator_args, + ) + + pre_dbt >> first_dbt_task_group + +**Key advantages:** + +* Integrates seamlessly into complex Airflow DAGs. +* Uses the same high-performance producer/consumer execution model. +* Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. + +.. image:: /_static/jaffle_shop_watcher_dbt_taskgroup_dag_run.png + :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` + :align: center + +------------------------------------------------------------------------------- + +Additional details +------------------- + +~~~~~~~~~~~~~~~~ +How retries work +~~~~~~~~~~~~~~~~ + +When the ``dbt build`` command run by ``DbtProducerWatcherOperator`` fails, it will notify all the ``DbtConsumerWatcherSensor``. + +The individual watcher tasks that subclass ``DbtConsumerWatcherSensor`` can retry the dbt command themselves, using the same behavior as ``ExecutionMode.LOCAL``. + +If a branch of the DAG fails, users can clear the status of a failed consumer task, including its downstream tasks, via the Airflow UI, and each of them will run in ``ExecutionMode.LOCAL``. + +**Producer retry behavior** + +.. versionadded:: 1.12.2 + +When the ``DbtProducerWatcherOperator`` is triggered for a retry (try_number > 1), it will not re-run the dbt build command and will succeed. In previous versions of Cosmos, the producer task would fail during retries. +This behavior is designed to support TaskGroup-level retries, as reported in `#2282 `_. + +**Why this matters:** + +- In earlier versions, attempting to retry the producer task would raise an ``AirflowException``, causing the retry to fail immediately. +- Now, the producer gracefully skips execution on retries, logging an informational message explaining that the retry was skipped to avoid running a second ``dbt build``. +- This allows users to retry entire TaskGroups and/or DAGs without the producer task blocking the retry flow. + +**Important considerations:** + +- The producer task should still be configured with ``retries=0`` (which Cosmos enforces by default) to avoid unintended duplicate ``dbt build`` runs. + +- By default, Cosmos sets ``retries`` to ``0`` in``DbtProducerWatcherOperator``. Users can retry manually by clearing the status of the producer task and all its downstream tasks, keeping in mind that the producer task will not re-run the ``dbt build`` command and will succeed. + +The overall retry behavior will be further improved once `#1978 `_ is implemented. + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Watcher dbt Execution Queue +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. versionadded:: 1.14.0 + +In watcher execution mode, by default, consumer sensor tasks are lightweight sensors that wait for the producer task to complete. On their first attempt, they require minimal CPU and memory resources. However, when these tasks retry, they execute the dbt command for the node, which may require significantly more resources. + +The ``watcher_dbt_execution_queue`` configuration allows you to specify a different worker queue for retry attempts. This enables you to: + +- **Optimize resource allocation** — Use lightweight workers for initial sensor execution and high-resource workers for retries +- **Improve scheduling efficiency** — Prevent resource contention between initial sensor tasks and retry executions +- **Scale independently** — Scale retry queues separately based on retry workload patterns + +**Configuration:** + +Set the ``watcher_dbt_execution_queue`` in your Airflow configuration: + +.. code-block:: ini + + [cosmos] + watcher_dbt_execution_queue = high_memory_queue + +Or via environment variable: + +.. code-block:: bash + + export AIRFLOW__COSMOS__WATCHER_DBT_EXECUTION_QUEUE=high_memory_queue + +**How it works:** + +- For watcher producer tasks (``DbtProducerWatcherOperator``), the configured queue is used during their first execution +- For watcher consumer tasks (``DbtConsumerWatcherSensor``), from their first retry onwards, if ``watcher_dbt_execution_queue`` is configured, the task is automatically assigned to the specified queue +- This behavior is enforced by Cosmos via an `Airflow cluster policy `_ (``task_instance_mutation_hook``) that mutates ``task_instance.queue`` at runtime for retry attempts + +.. note:: + + For producer task execution, we encourage users to set the ``watcher_dbt_execution_queue`` configuration. If, for any reason, users prefer to use a different node pool for producer tasks without setting an Airflow Cluster Policy, they can set the ``queue`` argument via ``setup_operator_args``. This, however, would not solve the problem of assigning consumer retries to nodes that may have more memory and CPU available. + + The effective precedence is: + + ``watcher_dbt_execution_queue`` > explicit ``queue`` on the producer (from ``setup_operator_args``) > ``operator_args`` > your Airflow deployment’s default queue. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Installation of Airflow and dbt +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since Cosmos 1.12.0, ``ExecutionMode.WATCHER`` works well regardless of whether dbt and Airflow are installed in the same Python virtual environment. + +When dbt and Airflow are installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` uses dbt `callback features `_. + +When dbt and Airflow are not installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` consumes the dbt `structured logging `_ to update the consumer tasks. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Synchronous versus Asynchronous sensor execution +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In Cosmos 1.11.0, the ``DbtConsumerWatcherSensor`` operator is implemented as a synchronous XCom sensor, which continuously occupies the worker slot - even if they're just sleeping and checking periodically. + +Starting with Cosmos 1.12.0, the ``DbtConsumerWatcherSensor`` supports +`deferrable (asynchronous) execution `_. Deferrable execution frees up the Airflow worker slot, while task status monitoring is handled by the Airflow triggerer component, +which increases overall task throughput. By default, the sensor now runs in deferrable mode. + +------------------------------------------------------------------------------- + +Known Limitations +------------------- + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Producer task implementation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The producer task is implemented as a ``DbtProducerWatcherOperator`` and currently relies on dbt being installed alongside the Airflow deployment, as in the ``ExecutionMode.LOCAL`` implementation. + +The alternative to this implementation is to use ``ExecutionMode.WATCHER_KUBERNETES``, which is built on top of ``ExecutionMode.KUBERNETES``. Check :ref:`watcher-kubernetes-execution-mode` for more information. + +~~~~~~~~~~~~~~~~~~~~~~~~ +Individual dbt Operators +~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``ExecutionMode.WATCHER`` efficiently implements the following operators: +* ``DbtSeedWatcherOperator`` +* ``DbtSnapshotWatcherOperator`` +* ``DbtRunWatcherOperator`` + +However, other operators that are available in the ``ExecutionMode.LOCAL`` mode are not implemented. + +The ``DbtBuildWatcherOperator`` is not implemented, since the build command is executed by the producer ``DbtProducerWatcherOperator`` operator. + +Additionally, since the ``dbt build`` command does not run ``source`` nodes, the operator ``DbtSourceWatcherOperator`` is equivalent to the ``DbtSourceLocalOperator`` operator, from ``ExecutionMode.LOCAL``. + +Finally, the following features are not implemented as operators under ``ExecutionMode.WATCHER``: + +* ``dbt ls`` +* ``dbt run-operation`` +* ``dbt docs`` +* ``dbt clone`` + +You can still invoke these operators using the default ``ExecutionMode.LOCAL`` mode. + +~~~~~~~~~~~~~ +Test behavior +~~~~~~~~~~~~~ + +By default, the watcher mode runs tests alongside models via the ``dbt build`` command being executed by the producer ``DbtProducerWatcherOperator`` operator. + +As a starting point, this execution mode does not support the ``TestBehavior.AFTER_EACH`` behavior, since the tests are not run as individual tasks. Since this is the default ``TestBehavior`` in Cosmos, we are injecting ``EmptyOperator`` as a starting point to ensure a seamless transition to the new mode. + +The ``TestBehavior.BUILD`` behavior is embedded in the producer ``DbtProducerWatcherOperator`` operator. + +The ``TestBehavior.NONE`` and ``TestBehavior.AFTER_ALL`` behave similarly to ``ExecutionMode.LOCAL``. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Airflow Datasets and Assets +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +While the ``ExecutionMode.WATCHER`` supports the ``emit_datasets`` parameter, the Airflow Datasets and Assets are emitted from the ``DbtProducerWatcherOperator`` task instead of the consumer tasks, as done for other Cosmos' execution modes. + +~~~~~~~~~~~~~~~~~~~~~~ +Source freshness nodes +~~~~~~~~~~~~~~~~~~~~~~ + +Since Cosmos 1.6, it `supports the rendering of source nodes `_. + +We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. + +This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. + +We have a follow-up ticket to `further investigate this use case `_. + + +Advanced config +------------------- + +~~~~~~~~~~~~~~~~ +Callback support +~~~~~~~~~~~~~~~~ + +The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` will use the user-defined callback function similar to ``ExecutionMode.LOCAL`` mode. + +You can define different ``callback`` behaviors for producer and consumer nodes by using ``operator_args`` to configure the consumer callback and ``setup_operator_args`` to override the callback for the producer, as described below. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Overriding ``operator_args`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` operators handle ``operator_args`` similar to the ``ExecutionMode.LOCAL`` mode. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Using Custom Args for the Producer and Watcher +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. versionadded:: 1.12.0 + +If you need to override ``operator_args`` for the ``DbtProducerWatcherOperator``, you can do so using ``setup_operator_args``. + +When using ``ExecutionMode.WATCHER``, you may want to configure specific properties, such as ``retries`` specifically for the ``DbtProducerWatcherOperator`` task. This can be useful for several reasons: +- Improved resilience - transient issues (e.g., temporary database or network failures) can be automatically retried. +- Reduced manual intervention - failed producer runs can recover without requiring operator restarts. +- Better reliability - retry behavior can be tuned independently from sensor tasks. + +Example: Configure the producer task with custom retry settings. + +.. code-block:: python + + from datetime import timedelta + from cosmos.config import ExecutionConfig + from cosmos.constants import ExecutionMode + + execution_config = ExecutionConfig( + execution_mode=ExecutionMode.WATCHER, + setup_operator_args={ + "retries": 0, + "retry_delay": timedelta(minutes=5), + }, + ) + +This allows you to customize ``DbtProducerWatcherOperator`` retry behavior without affecting the arguments used by the other sensor tasks. + +If configuring queues, we suggest using the previously mentioned ``watcher_dbt_execution_queue`` configuration instead of the ``setup_operator_args``. + +.. note:: + Please note that ``setup_operator_args`` is specific to Cosmos and is not related to Airflow setup or teardown task. + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Sensor slot allocation and polling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each ``DbtDag`` or ``DbtTaskGroup`` root node will startup during DAG runs at - potentially - the same time as the DAG Run. This may not happen, since it is dependent on the +concurrency settings and available task slots in the Airflow deployment. + +The consequence is that tasks may take longer to be updated if they are not sensing at the moment that the transformation happens. + +We plan to review this behaviour and alternative approaches in the future. + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Asynchronous sensor execution +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Deferrable execution is currently supported only for dbt models, seeds and snapshots. +- Deferrable execution applies only to the first task attempt (try number 1). For subsequent retries, the sensor falls back to synchronous execution. + +To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. + +.. literalinclude:: ../../dev/dags/example_watcher.py + :language: python + :start-after: [START example_watcher_synchronous] + :end-before: [END example_watcher_synchronous] + +------------------------------------------------------------------------------- + +Troubleshooting +--------------- + +Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." +Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. Check the `dbt threading docs `_ for more information on how to set the number of threads. + + +Summary +------- + +``ExecutionMode.WATCHER`` represents a significant leap forward for running dbt in Airflow via Cosmos: + +* ✅ Up to **5× faster** dbt DAG runs +* ✅ Maintains **model-level visibility** in Airflow +* ✅ Enables **smarter resource allocation** +* ✅ Built on proven Cosmos rendering techniques + +This is an experimental feature, and we are looking for feedback from the community. + +Stay tuned for further documentation and base image support for the ``ExecutionMode.WATCHER`` in upcoming releases. diff --git a/docs/configuration/run-dbt/container/aws-container-run-job.rst b/docs/configuration/run-dbt/container/aws-container-run-job.rst new file mode 100644 index 0000000000..4321c8f346 --- /dev/null +++ b/docs/configuration/run-dbt/container/aws-container-run-job.rst @@ -0,0 +1,191 @@ +.. _aws-container-run-job: + +Getting Started with Astronomer Cosmos on AWS ECS +================================================== + +Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. +Schematically, the guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aws_ecs_schematic.png + :width: 800 + +Prerequisites ++++++++++++++ + +Before you begin, ensure you have the following: + +- An active **AWS Account** with permissions to create ECS clusters, register task definitions, and run tasks. +- The **AWS CLI** installed and configured with the proper credentials. +- **Docker** installed for building your container image. +- Access to your container registry (for example, **Amazon ECR**) where your job image is stored. +- Basic familiarity with AWS ECS concepts (clusters, task definitions, services, and Fargate). +- An existing installation of **Astronomer Cosmos** (refer to the `Cosmos documentation `_ for more details). + + + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: + +.. code-block:: bash + + python3 -m venv venv + source venv/bin/activate + python3 -m pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[amazon]" + pip install "aiobotocore[boto3]" +.. note:: + The package aiobotocore[boto3] is optional; you will need it if you plan to use **deferred tasks**. + +**Set up your ECR** + +1. **Set your secrets** + On the `cosmos-examples `_ repository, you can find a ready-to-use Docker image for the AWS ECS service. Just replace your secrets, or you can create your own. + +2. **AWS CLI login** + Before building and pushing your image, you first need to log in to the AWS service using the AWS CLI tool. + Use the following command: + + .. code-block:: bash + + aws ecr-public get-login-password --region | docker login --username AWS --password-stdin + +3. **Build and tag your image** + Once you have your image ready, run the following commands: + + .. code-block:: bash + + docker build -f Dockerfile.aws_ecs . --platform=linux/amd64 -t + docker tag + +4. **Push your image** + + .. code-block:: bash + + docker push + +**Configure Your AWS Environment** + +1. **Create an ECS Cluster** + + Create an ECS cluster to host your Cosmos jobs. You can do this from the AWS Console or using the AWS CLI: + + .. code-block:: bash + + aws ecs create-cluster --cluster-name my-cosmos-cluster + +2. **Set Up an IAM Role for ECS Tasks** + + Ensure you have an IAM role that your ECS tasks can assume. This role should include permissions for ECS, ECR, and CloudWatch (for logs). For example, you might create a role named ``ecsTaskExecutionRole`` with the managed policies: + + - ``AmazonECSTaskExecutionRolePolicy`` + - (Optional) Additional policies for custom resource access + +3. **Configure Networking** + + For Fargate tasks, make sure you have at least one subnet (preferably in multiple Availability Zones) and a security group that permits outbound internet access if needed. Note the subnet IDs for later use. + +**Prepare Your Cosmos Job Definition** + +Cosmos jobs are defined as container tasks. Create a task definition file (e.g., ``cosmos-task-definition.json``) with the configuration for your job. + +For example: + +.. code-block:: json + + { + "family": "cosmos-job", + "networkMode": "awsvpc", + "requiresCompatibilities": [ + "FARGATE" + ], + "cpu": "512", + "memory": "1024", + "executionRoleArn": "arn:aws:iam:::role/ecsTaskExecutionRole", + "containerDefinitions": [ + { + "name": "cosmos-job", + "image": "/your_image:latest", + "essential": true, + "environment": [ + { "name": "VAR1", "value": "value1" }, + { "name": "VAR2", "value": "value2" } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/cosmos-job", + "awslogs-region": "us-east-1", + "awslogs-stream-prefix": "ecs" + } + } + } + ] + } + +.. note:: + + Replace ````, ````, and adjust the CPU, memory, and environment variables as needed. + +**Deploy Your Cosmos Job on AWS ECS** + +1. **Register the Task Definition** + + Use the AWS CLI to register your task definition: + + .. code-block:: bash + + aws ecs register-task-definition --cli-input-json file://cosmos-task-definition.json + +2. **Run the Task** + + Run a test task on your ECS cluster. Specify the subnets and security groups in your network configuration. For example: + + .. code-block:: bash + + aws ecs run-task \ + --cluster my-cosmos-cluster \ + --launch-type FARGATE \ + --task-definition cosmos-job \ + --network-configuration "awsvpcConfiguration={subnets=[subnet-12345678,subnet-87654321],securityGroups=[sg-abcdef12],assignPublicIp=ENABLED}" + + Once the test is ok, we are able to run the dbt commands in our Cosmos DAG: + + .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run.png + :width: 800 + + .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run_logs.png + :width: 800 + + Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! + + +**Monitor and Debug Your Job** + +1. **Check Task Status** + + You can view the status of your task from the AWS Console under your ECS cluster or via the CLI: + + .. code-block:: bash + + aws ecs describe-tasks --cluster my-cosmos-cluster --tasks + +2. **View Logs** + + Since the task definition configures AWS CloudWatch Logs, you can view your job’s output in the CloudWatch Logs console. Look for log streams with the prefix you set (e.g., ``ecs/cosmos-job``). + +**Conclusion** + + +By following this guide, you can deploy Astronomer Cosmos jobs on AWS ECS using Fargate. This integration enables you to leverage the scalability and managed infrastructure of ECS while maintaining a consistent container orchestration experience with Cosmos. + +For more detailed information on AWS ECS, please refer to the `AWS ECS Developer Guide `_. + +Happy deploying! :rocket: + + +Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! diff --git a/docs/configuration/run-dbt/container/azure-container-instance.rst b/docs/configuration/run-dbt/container/azure-container-instance.rst new file mode 100644 index 0000000000..86ce3ab9ef --- /dev/null +++ b/docs/configuration/run-dbt/container/azure-container-instance.rst @@ -0,0 +1,138 @@ +.. _azure-container-instance: + +Azure Container Instance Execution Mode +======================================= +.. versionadded:: 1.4 + +This tutorial will guide you through the steps required to use Azure Container Instance as the Execution Mode for your dbt code with Astronomer Cosmos. Schematically, the guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aci_schematic.png + :width: 800 + +Prerequisites ++++++++++++++ +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Azure CLI (install guide here: `Azure CLI `_) +4. Astronomer-cosmos package containing the dbt Azure Container Instance operators +5. Azure account with: + 1. A resource group + 2. A service principal with `Contributor` permissions on the resource group + 3. A Container Registry + 4. A Postgres instance accessible from Azure. (we use an Azure Postgres instance in the example) +6. Docker image built with required dbt project and dbt DAG +7. dbt DAG with dbt Azure Container Instance operators in the Airflow DAGs directory to run in Airflow + +More information on how to achieve 2-6 is detailed below. + +Note that the steps below will walk you through an example, for which the code can be found HERE + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install apache airflow & astronomer-postgres + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-postgres,azure-container-instance]" + +**Setup Postgres database** + +You will need a postgres database running to be used as the database for the dbt project. In order to have it accessible from Azure Container Instance, the easiest way is to create an Azure Postgres instance. For this, run the following (assuming you are logged into your Azure account) + +.. code-block:: bash + + az postgres server create -l westeurope -g <<>> -n <<>> -u dbadmin -p <<>> --sku-name B_Gen5_1 --ssl-enforcement Enabled + + +**Setup Azure Container Registry** +In order to run a container in Azure Container Instance, it needs access to the container image. In our setup, we will use Azure Container Registry for this. To set an Azure Container Registry up, you can use the following bash command: + +.. code-block:: bash + + az acr create --name <<>> --resource-group <<>> --sku Basic --admin-enabled + +**Build the dbt Docker image** + +For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. + +Clone the `cosmos-example `_ repo + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. + +.. code-block:: bash + + docker build -t <<>:1.0.0 -f Dockerfile.azure_container_instance . + +After this, the image needs to be pushed to the registry of your choice. Note that your image name should contain the name of your registry: +.. code-block:: bash + + docker push <<>>:1.0.0 + +.. note:: + + You may need to ensure docker knows to use the right credentials. If using Azure Container Registry, this can be done by running the following command: + .. code-block:: bash + + az acr login + +.. note:: + + If running on M1, you may need to set the following envvars for running the docker build command in case it fails + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The `dbt profile `_ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Setup Airflow Connections** +Now you have the required Azure infrastructure, you still need to add configuration to Airflow to ensure the infrastructure can be used. You'll need 3 connections: + +1. ``aci_db``: a Postgres connection to your Azure Postgres instance. +2. ``aci``: an Azure Container Instance connection configured with a Service Principal with sufficient permissions (i.e. ``Contributor`` on the resource group in which you will use Azure Container Instances). +3. ``acr``: an Azure Container Registry connection configured for your Azure Container Registry. + +Check out the ``airflow-settings.yml`` file `here `_ for an example. If you are using Astro CLI, filling in the right values here will be enough for this to work. + +**Setup and Trigger the DAG with Airflow** + +Copy the dags directory from cosmos-example repo to your Airflow home + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_azure_container_instance `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_azure_container_instance.png + :width: 800 diff --git a/docs/configuration/run-dbt/container/docker.rst b/docs/configuration/run-dbt/container/docker.rst new file mode 100644 index 0000000000..0005914886 --- /dev/null +++ b/docs/configuration/run-dbt/container/docker.rst @@ -0,0 +1,111 @@ +.. _docker: + +Docker Execution Mode +======================================== + +The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. + +Requirements +++++++++++++ + +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Astronomer-cosmos package containing the dbt Docker operators +4. Postgres docker container +5. Docker image built with required dbt project and dbt DAG +6. dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow + +More information on how to achieve 2-6 is detailed below. + +Step-by-step instructions ++++++++++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install `Apache Airflow® `_ & astronomer-postgres + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-postgres]" + +**Setup Postgres database** + +You will need a postgres database running to be used as the database for the dbt project. Run the following command to run and expose a postgres database + +.. code-block:: bash + + docker run --name some-postgres -e POSTGRES_PASSWORD="" -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p5432:5432 -d postgres + +**Build the dbt Docker image** + +For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. + +Clone the `cosmos-example `_ repo + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. + +.. code-block:: bash + + docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . + +.. note:: + + If running on M1, you may need to set the following envvars for running the docker build command in case it fails + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The `dbt profile `_ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Setup and Trigger the DAG with Airflow** + +Copy the dags directory from cosmos-example repo to your Airflow home + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_docker `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_docker_dag_run.png + :width: 800 + + +Specifying ProfileConfig ++++++++++++++++++++++++++ + +Starting with Cosmos 1.8.0, you can use the ``profile_config`` argument in your Dbt DAG Docker operators to reference +profiles for your dbt project defined in a profiles.yml file. To do so, provide the file’s path via the +``profiles_yml_path`` parameter in ``profile_config``. + +Note that in ``ExecutionMode.DOCKER``, the ``profile_config`` is only compatible with the ``profiles_yml_path`` +approach. The ``profile_mapping`` method will not work because the required Airflow connections cannot be accessed +within the Docker container to map them to the dbt profile. diff --git a/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst b/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst new file mode 100644 index 0000000000..fa4d0c60c4 --- /dev/null +++ b/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst @@ -0,0 +1,265 @@ +.. _gcp-cloud-run-job: + +GCP Cloud Run Job Execution Mode +======================================= +.. versionadded:: 1.7 + +This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_gcp_crj_schematic.png + :width: 600 + +Prerequisites ++++++++++++++ +1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. +2. Airflow +3. Google Cloud SDK (`install guide `_) +4. Astronomer-cosmos package containing the dbt Cloud Run Job operators +5. GCP account with: + 1. A GCP project (`setup guide `_) + 2. IAM roles: + * Basic Role: `Owner `_ (control over whole project) or + * Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) + 3. Enabled service APIs: + * Artifact Registry API + * Cloud Run Admin API + * BigQuery API + 4. A service account with BigQuery roles: `JobUser `_ and `DataEditor `_ +6. Docker image built with required dbt project and dbt DAG +7. dbt DAG with Cloud Run Job operators in the Airflow DAGs directory to run in Airflow + +.. note:: + + Google Cloud Platform provides free tier on many resources, as well as Free Trial with $300 in credit. Learn more `here `_. + +More information on how to achieve 2-6 is detailed below. + + +Step-by-step guide +++++++++++++++++++ + +**Install Airflow and Cosmos** + +Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: + +.. code-block:: bash + + python3 -m venv venv + source venv/bin/activate + python3 -m pip install --upgrade pip + pip install apache-airflow + pip install "astronomer-cosmos[dbt-bigquery,gcp-cloud-run-job]" + +**Setup gcloud and environment variables** + +Set environment variables that will be used to create cloud infrastructure. Replace placeholders with your unique GCP ``project id`` and ``region`` of the project: + +.. code-block:: bash + + export PROJECT_ID=<<>> + export REGION=<<>> + export REPO_NAME="astronomer-cosmos-dbt" + export IMAGE_NAME="$REGION-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/cosmos-example" + export SERVICE_ACCOUNT_NAME="cloud-run-job-sa" + export DATASET_NAME="astronomer_cosmos_example" + export CLOUD_RUN_JOB_NAME="astronomer-cosmos-example" + +Before we do anything in the GCP project, we first need to authorize gcloud to access the Cloud Platform with Google user credentials: + +.. code-block:: bash + + gcloud auth login + +You'll receive a link to sign into Google Cloud SDK using a Google Account. + +Next, set default ``project id`` using below command: + +.. code-block:: bash + + gcloud config set project $PROJECT_ID + +In case BigQuery has never been used before in the project, run below command to enable BigQuery API: + +.. code-block:: bash + + gcloud services enable bigquery.googleapis.com + +**Setup Artifact Registry** + +In order to run a container in Cloud Run Job, it needs access to the container image. In our setup, we will use Artifact Registry repository that stores images. +To use Artifact Registry, you need to enable the API first: + +.. code-block:: bash + + gcloud services enable artifactregistry.googleapis.com + +To set an Artifact Registry repository up, you can use the following bash command: + +.. code-block:: bash + + gcloud artifacts repositories create $REPO_NAME \ + --repository-format=docker \ + --location=$REGION \ + --project $PROJECT_ID + +**Setup Service Account** + +In order to use dbt and make transformations in BigQuery, Cloud Run Job needs some BigQuery permissions. One way to achieve that is to set up a separate ``Service Account`` with needed permissions: + +.. code-block:: bash + + # create a service account + gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME + +.. code-block:: bash + + # grant JobUser role + gcloud projects add-iam-policy-binding $PROJECT_ID \ + --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ + --role="roles/bigquery.jobUser" + +.. code-block:: bash + + # grant DataEditor role + gcloud projects add-iam-policy-binding $PROJECT_ID \ + --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ + --role="roles/bigquery.dataEditor" + +**Build the dbt Docker image** + +Now, we are going to download an example dbt project and build a Docker image with it. + +.. important:: + + You need to ensure Docker is using the right credentials to push images. For Artifact Registry, this can be done by running the following command: + + .. code-block:: bash + + gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://$REGION-docker.pkg.dev + + The token will be valid for 1 hour. After that, you need to create another one, if still needed. + +Clone the `cosmos-example `_ repo: + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example + +Open `Dockerfile `_ located in ``gcp_cloud_run_job_example`` folder and change environments variables ``GCP_PROJECT_ID`` and ``GCP_REGION`` to your GCP project id and project region. + +Build a Docker image using previously modified ``Dockerfile``, which will be used by Cloud Run Job: + +.. code-block:: bash + + docker build -t $IMAGE_NAME -f gcp_cloud_run_job_example/Dockerfile.gcp_cloud_run_job . + +.. important:: + + Make sure to stay in ``cosmos-example`` directory when running ``docker build`` command. + +After this, the image needs to be pushed to the Artifact Registry: + +.. code-block:: bash + + docker push $IMAGE_NAME + +Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. + + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The `bigquery dbt profile `_ file is added to the image + - The dbt_project.yml is replaced with `bigquery_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +**Create Cloud Run Job instance** + +When the image is pushed to Artifact Registry, you can finally create Cloud Run Job with the image and previously created service account. + +First, enable Cloud Run Admin API using below command: + +.. code-block:: bash + + gcloud services enable run.googleapis.com + + +Next, set default Cloud Run region to your GCP region: + +.. code-block:: bash + + gcloud config set run/region $REGION + +Then, run below command to create Cloud Run Job instance: + +.. code-block:: bash + + gcloud run jobs create $CLOUD_RUN_JOB_NAME \ + --image=$IMAGE_NAME \ + --task-timeout=180s \ + --max-retries=0 \ + --cpu=1 \ + --memory=512Mi \ + --service-account=$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com + +**Setup Airflow Connections** + +Now, when you have the required Google Cloud infrastructure, you still need to check Airflow configuration to ensure the infrastructure can be used. You'll need a ``google_cloud_default`` connection in order to work on GCP resources. + +Check out an `example `_ of the ``airflow-settings.yml`` file. If you are using Astro CLI, filling in the right values here will be enough for this to work. + +**Setup and Trigger the DAG with Airflow** + +Open `jaffle_shop_gcp_cloud_run_job `_ DAG file and update ``GCP_PROJECT_ID`` and ``GCP_LOCATION`` constants with your GCP project id and project region. + +When the DAG is configured, copy the ``dags`` directory from ``cosmos-example`` repo to your Airflow home: + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow: + +.. code-block:: bash + + airflow standalone + +.. note:: + + You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_gcp_cloud_run_job `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_gcp_cloud_run_job.png + :width: 800 + + +You can also verify the tables that were created using dbt in BigQuery Studio: + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_big_query.png + :width: 800 + + +**Delete resources** + +After the successful tests, don't forget to delete Google Cloud resources to save up costs: + +.. code-block:: bash + + # Delete Cloud Run Job instance + + gcloud run jobs delete $CLOUD_RUN_JOB_NAME + +.. code-block:: bash + + # Delete BigQuery main and custom dataset specified in dbt schema.yml with all tables included + + bq rm -r -f -d $PROJECT_ID:$DATASET_NAME + + bq rm -r -f -d $PROJECT_ID:dbt_dev + +.. code-block:: bash + + # Delete Artifact Registry repository with all images included + + gcloud artifacts repositories delete $REPO_NAME \ + --location=$REGION diff --git a/docs/configuration/run-dbt/container/index.rst b/docs/configuration/run-dbt/container/index.rst new file mode 100644 index 0000000000..634e9e8eb4 --- /dev/null +++ b/docs/configuration/run-dbt/container/index.rst @@ -0,0 +1,13 @@ +Run dbt in a container +====================== + +.. toctree:: + :maxdepth: 1 + :caption: Run dbt in a container + + aws-container-run-job + azure-container-instance + docker + gcp-cloud-run-job + kubernetes + watcher-kubernetes-execution-mode \ No newline at end of file diff --git a/docs/configuration/run-dbt/container/kubernetes.rst b/docs/configuration/run-dbt/container/kubernetes.rst new file mode 100644 index 0000000000..607ba07bd7 --- /dev/null +++ b/docs/configuration/run-dbt/container/kubernetes.rst @@ -0,0 +1,167 @@ +.. _kubernetes: + +Kubernetes Execution Mode +============================================== + +The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: + +- Postgres is run in the Kubernetes (K8s) cluster as a container +- Airflow is run locally, and it triggers a K8s Pod which runs dbt + +Requirements +++++++++++++ + +To test the DbtKubernetesOperators locally, we encourage you to install the following: + +- Local Airflow (either standalone or using Astro CLI) +- `Kind `_ to run K8s locally +- `Helm `_ to install Postgres in K8s +- `Docker `_ to create the dbt container image, which will allow Airflow to create a K8s pod which will run dbt + +At the moment, the user is expected to add to the Docker image both: + +- The dbt project files +- The dbt Profile, which contains the information for dbt to access the database while parsing the project from Apache Airflow nodes +- Handle secrets + +Additional KubernetesPodOperator parameters can be added to the ``operator_args`` parameter of the ``DbtKubernetesOperator``. + +For instance, + +.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py + :language: python + :start-after: [START kubernetes_tg_example] + :end-before: [END kubernetes_tg_example] + +Step-by-step instructions ++++++++++++++++++++++++++ + +Using installed `Kind `_, you can setup a local kubernetes cluster + +.. code-block:: bash + + kind create cluster + +Deploy a Postgres pod to Kind using `Helm `_ + +.. code-block:: bash + + helm repo add bitnami https://charts.bitnami.com/bitnami + helm repo update + helm install postgres bitnami/postgresql + +Retrieve the Postgres password and set it as an environment variable. + +.. code-block:: bash + + export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgres-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d) + +Check that the environment variable was set and that it is not empty + +.. code-block:: bash + + echo $POSTGRES_PASSWORD + +Expose the Postgres to the host running Docker/Kind. + +.. code-block:: bash + + kubectl port-forward --namespace default postgres-postgresql-0 5432:5432 + +Check that you're able to connect to the exposed pod. + +.. code-block:: bash + + PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432 + + postgres=# \dt + \q + +Create a K8s secret which contains the credentials to access Postgres. + +.. code-block:: bash + + kubectl create secret generic postgres-secrets --from-literal=host=postgres-postgresql.default.svc.cluster.local --from-literal=password=$POSTGRES_PASSWORD + +Clone the example repo that contains the Airflow DAG and dbt project files. + +.. code-block:: bash + + git clone https://github.com/astronomer/cosmos-example.git + cd cosmos-example/ + +Create a Docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be run in K8s. + +.. code-block:: bash + + docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . + +.. note:: + + If running on M1, you may need to set the following environment variables to run the Docker build command in case it fails. + + .. code-block:: bash + + export DOCKER_BUILDKIT=0 + export COMPOSE_DOCKER_CLI_BUILD=0 + export DOCKER_DEFAULT_PLATFORM=linux/amd64 + +Take a look at the Dockerfile to understand its purpose so that you can use it as a reference in your project. + + - The `dbt profile `__ file is added to the image + - The dags directory containing the `dbt project jaffle_shop `_ is added to the image + - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. + +Make the build image available in the Kind K8s cluster. + +.. code-block:: bash + + kind load docker-image dbt-jaffle-shop:1.0.0 + +Create a Python virtual environment and install the latest version of Astronomer Cosmos, which contains the K8s Operator. + +.. code-block:: bash + + python -m venv venv + source venv/bin/activate + pip install --upgrade pip + pip install "astronomer-cosmos[dbt-postgres]" apache-airflow-providers-cncf-kubernetes + +Make the `jaffle_shop_kubernetes.py `__ file at your Airflow DAG home: + +.. code-block:: bash + + cp -r dags $AIRFLOW_HOME/ + +Run Airflow + +.. code-block:: bash + + airflow standalone + +.. note:: + + You may need to run Airflow standalone with ``sudo`` if your Airflow user is unable to access the Docker socket URL or pull images in the Kind cluster. + +Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. + +Enable and trigger a run of the `jaffle_shop_k8s `_ DAG. You will be able to see the following successful DAG run. + +.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_k8s_dag_run.png + :width: 800 + +.. _kubernetes-known-limitations: + +Known Limitations ++++++++++++++++++ + +The Kubernetes execution mode has the following limitations: + +- Does not emit OpenLineage events (there is an `open ticket #496 `__ to address this) +- Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) +- Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) +- Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) +- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) +- Does not expose Compiled SQL as a `templated field `_ +- Does not benefit from `Cosmos caching mechanisms `_ +- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst new file mode 100644 index 0000000000..16dbbffd0a --- /dev/null +++ b/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst @@ -0,0 +1,214 @@ +.. _watcher-kubernetes-execution-mode: + +``ExecutionMode.WATCHER_KUBERNETES``: High-Performance dbt Execution in Kubernetes +=================================================================================== + +.. versionadded:: 1.13.0 + +The ``ExecutionMode.WATCHER_KUBERNETES`` combines the **speed of the** :ref:`watcher-execution-mode` **with the isolation of** :ref:`kubernetes`. + +This execution mode is ideal for users who: + +* Want to leverage the performance benefits of the watcher execution mode +* Need to run dbt in isolated Kubernetes pods +* Prefer not to install dbt in their Airflow deployment + +------------------------------------------------------------------------------- + +Background +---------- + +The :ref:`watcher-execution-mode` introduced in Cosmos 1.11.0 significantly reduces dbt pipeline run times by running dbt as a single command while maintaining model-level observability in Airflow. + +However, the original ``ExecutionMode.WATCHER`` requires dbt to be installed alongside Airflow. The ``ExecutionMode.WATCHER_KUBERNETES`` removes this limitation by running the dbt command inside Kubernetes pods, similar to ``ExecutionMode.KUBERNETES``. + +For more details on the watcher concept and how it works, please refer to the :ref:`watcher-execution-mode` documentation. + +------------------------------------------------------------------------------- + +How to Use +---------- + +Users previously using ``ExecutionMode.KUBERNETES`` can simply replace the ``execution_mode`` to use ``ExecutionMode.WATCHER_KUBERNETES``. + +The following example shows how to configure a ``DbtDag`` with ``ExecutionMode.WATCHER_KUBERNETES``: + +.. code-block:: python + + from cosmos import DbtDag + from cosmos.config import ExecutionConfig + from cosmos.constants import ExecutionMode + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + "image": DBT_IMAGE, + "get_logs": True, + "log_events_on_failure": True, + }, + ) + +**Key differences from** ``ExecutionMode.KUBERNETES``: + +* The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` +* The producer task runs the entire ``dbt build`` command in a single Kubernetes pod +* Consumer tasks (sensors) watch for the completion of their corresponding dbt models + +For the complete setup including Kubernetes secrets, Docker image configuration, and profile setup, refer to the :ref:`kubernetes` documentation. + +------------------------------------------------------------------------------- + +Performance Gains +----------------- + +Early benchmarks using the ``jaffle_shop_watcher_kubernetes`` DAG show significant improvements: + ++-----------------------------------------------+------------------+ +| Execution Mode | Total Runtime | ++===============================================+==================+ +| ``ExecutionMode.KUBERNETES`` | 00:00:32.155 | ++-----------------------------------------------+------------------+ +| ``ExecutionMode.WATCHER_KUBERNETES`` | 00:00:11.783 | ++-----------------------------------------------+------------------+ + +This represents approximately a **63% reduction** in total DAG runtime. + +The performance improvement comes from: + +* Running dbt as a single command (reducing Kubernetes pod startup overhead) +* Leveraging dbt's native threading capabilities +* Eliminating repeated dbt initialization for each model + +------------------------------------------------------------------------------- + +Known Limitations +----------------- + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Kubernetes Provider Version Compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``ExecutionMode.WATCHER_KUBERNETES`` does not work with older versions of the ``apache-airflow-providers-cncf-kubernetes`` provider (<=10.7.0). + +Please ensure you have a compatible version installed: + +.. code-block:: bash + + pip install "apache-airflow-providers-cncf-kubernetes>10.7.0" + +We successfully tested against the most recent release of the provider (`10.12.2 `_). + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Support for KPO deferrable mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The producer node created by the ``ExecutionMode.WATCHER_KUBERNETES`` producer task can be set to deferrable mode as long as: + +- The correct version of Airflow Kubernetes is installed (``>=10.12.2``). This version fixed a bug (`PR `_) that prevented setting callbacks and parsing the logs when the Kubernetes Operator run using ``deferrable``. The experience should be further improved once `this other PR is merged `_. + +.. code-block:: bash + + pip install "apache-airflow-providers-cncf-kubernetes>=10.12.2" + +- The arguments ``deferrable=True`` and ``is_delete_operator_pod=True`` are set: + +.. code-block:: python + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + "deferrable": True, + "is_delete_operator_pod": True, + "image": DBT_IMAGE, + "get_logs": True, + "log_events_on_failure": True, + }, + ) + +Conversely, the consumer tasks that subclass ``DbtConsumerWatcherKubernetesSensor`` run in deferrable mode by default when operating as a sensor. They can also operate in deferrable mode if they are running dbt themselves upon retry. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Mandatory ``operator_args`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``operator_args`` must define ``get_logs`` and ``log_events_on_failure``: + +.. code-block: python + + dag = DbtDag( + dag_id="jaffle_shop_watcher_kubernetes", + # ... other DAG parameters ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.WATCHER_KUBERNETES, + dbt_project_path=K8S_PROJECT_DIR, + ), + operator_args={ + # ... other KPO mandatory args ... + "get_logs": True, + "log_events_on_failure": True, + }, + ) + + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Other Inherited Limitations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following limitations from ``ExecutionMode.WATCHER`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``: + +* **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. + +* **Test behavior**: The ``TestBehavior.AFTER_EACH`` is not supported. Tests are run as part of the ``dbt build`` command by the producer task. + +* **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. + +For more details on these limitations, refer to the :ref:`watcher-execution-mode` documentation. + +Additionally, the limitations from ``ExecutionMode.KUBERNETES`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``. For details, refer to the :ref:`kubernetes-known-limitations` documentation. + +------------------------------------------------------------------------------- + +Example DAG +----------- + +Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: + +.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py + :language: python + +------------------------------------------------------------------------------- + +Prerequisites +------------- + +Before using ``ExecutionMode.WATCHER_KUBERNETES``, ensure you have: + +1. A Kubernetes cluster configured and accessible from your Airflow deployment +2. A Docker image containing your dbt project and profile +3. The ``apache-airflow-providers-cncf-kubernetes`` provider installed (version >10.7.0) + +For detailed setup instructions, refer to the :ref:`kubernetes` documentation. + +------------------------------------------------------------------------------- + +Summary +------- + +``ExecutionMode.WATCHER_KUBERNETES`` provides: + +* ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` +* ✅ **Isolation** between dbt and Airflow dependencies +* ✅ **Model-level visibility** in Airflow +* ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` + +This execution mode is ideal for teams who want the performance benefits of the watcher mode while maintaining the isolation provided by Kubernetes execution. diff --git a/docs/configuration/run-dbt/execution-modes.rst b/docs/configuration/run-dbt/execution-modes.rst new file mode 100644 index 0000000000..a9bd3f1e2b --- /dev/null +++ b/docs/configuration/run-dbt/execution-modes.rst @@ -0,0 +1,387 @@ +.. _execution-modes: + +Execution Modes +=============== + +.. toctree:: + :maxdepth: 3 + :caption: Run dbt in the Airflow worker + + airflow-worker/index + +.. toctree:: + :maxdepth: 3 + :caption: Run dbt in a container + + container/index + + +Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: + +1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) +2. **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos +3. **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) +4. **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) +5. **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) +6. **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) +7. **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) +8. **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) +9. **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ +10. **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. +11. **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. + +The choice of the ``execution mode`` can vary based on each user's needs and concerns. For more details, check each execution mode described below. + +.. _execution-modes-comparison: + +.. list-table:: Execution Modes Comparison + :widths: 25 25 25 25 + :header-rows: 1 + + * - Execution Mode + - Task Duration + - Environment Isolation + - Cosmos Profile Management + * - Local + - Fast + - None + - Yes + * - Virtualenv + - Medium + - Lightweight + - Yes + * - Docker + - Slow + - Medium + - No + * - Kubernetes + - Slow + - High + - No + * - AWS_EKS + - Slow + - High + - No + * - Azure Container Instance + - Slow + - High + - No + * - GCP Cloud Run Job Instance + - Slow + - High + - No + * - AWS ECS + - Slow + - High + - No + * - Airflow Async + - Very Fast + - Medium + - Yes + * - Watcher + - Very Fast + - None + - Yes + * - Watcher Kubernetes + - Fast + - High + - No + +Local +----- + +By default, Cosmos uses the ``local`` execution mode. + +The ``local`` execution mode is the fastest way to run Cosmos operators since they don't install ``dbt`` nor build docker containers. However, it may not be an option for users using managed Airflow services such as +Google Cloud Composer, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`), the user may not be able to install ``dbt`` in a custom path. + +The ``local`` execution mode assumes a ``dbt`` binary is reachable within the Airflow worker node. + +If ``dbt`` was not installed as part of the Cosmos packages, +users can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. + +.. note:: + Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. + This feature is bound to `dbt partial parsing limitations `_. + Learn more: :ref:`partial-parsing`. + +When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). + +Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: + +.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py + :language: python + :start-after: [START local_example] + :end-before: [END local_example] + + +Virtualenv +---------- + +If you're using managed Airflow on GCP (Cloud Composer), for instance, we recommend you use the ``virtualenv`` execution mode. + +The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. + +In this case, users are responsible for declaring which version of ``dbt`` they want to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. + +Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). +Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. + +Some drawbacks of this approach: + +- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. +- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. +- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. + +Example of how to use: + +.. literalinclude:: ../../dev/dags/example_virtualenv.py + :language: python + :start-after: [START virtualenv_example] + :end-before: [END virtualenv_example] + +Docker +------ + +The ``docker`` approach assumes users have a previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml``, managed by the user. + +The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). + +The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. + +This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. +If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. + +Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. + +Example DAG: + +.. code-block:: python + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.DOCKER, + ), + operator_args={ + "image": "dbt-jaffle-shop:1.0.0", + "network_mode": "bridge", + }, + ) + + +Kubernetes +---------- + +The ``kubernetes`` approach is a very isolated way of running ``dbt`` since the ``dbt`` run commands from within a Kubernetes Pod, usually in a separate host. + +It assumes the user has a Kubernetes cluster. It also expects the user to ensure the Docker container has up-to-date ``dbt`` pipelines and profiles, potentially leading the user to declare secrets in two places (Airflow and Docker container). + +The ``Kubernetes`` deployment may be slower than ``Docker`` and ``Virtualenv`` assuming that the container image is built (which is slower than creating a Python ``virtualenv`` and installing ``dbt-core``) and the Airflow task needs to spin up a new ``Pod`` in Kubernetes. + +Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref:`kubernetes`. + +Example DAG: + +.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py + :language: python + :start-after: [START kubernetes_seed_example] + :end-before: [END kubernetes_seed_example] + +AWS_EKS +---------- + +The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. +It uses the `EKSPodOperator `_ +to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. + + +Example DAG: + +.. code-block:: python + + postgres_password_secret = Secret( + deploy_type="env", + deploy_target="POSTGRES_PASSWORD", + secret="postgres-secrets", + key="password", + ) + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AWS_EKS, + ), + operator_args={ + "image": "dbt-jaffle-shop:1.0.0", + "cluster_name": CLUSTER_NAME, + "get_logs": True, + "is_delete_operator_pod": False, + "secrets": [postgres_password_secret], + }, + ) + +Azure Container Instance +------------------------ +.. versionadded:: 1.4 + +Similar to the ``kubernetes`` approach, using ``Azure Container Instances`` as the execution mode gives a very isolated way of running ``dbt``, since the ``dbt`` run itself is run within a container running in an Azure Container Instance. + +This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. Please checkout the step-by-step guide for using Azure Container Instance as the execution mode + + +.. code-block:: python + + docker_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.AZURE_CONTAINER_INSTANCE + ), + operator_args={ + "ci_conn_id": "aci", + "registry_conn_id": "acr", + "resource_group": "my-rg", + "name": "my-aci-{{ ti.task_id.replace('.','-').replace('_','-') }}", + "region": "West Europe", + "image": "dbt-jaffle-shop:1.0.0", + }, + ) + +GCP Cloud Run Job +------------------------ +.. versionadded:: 1.7 + +The ``gcp_cloud_run_job`` execution mode is particularly useful for users who prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. + +For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . + +This execution mode allows users to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. + + +.. code-block:: python + + gcp_cloud_run_job_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig(execution_mode=ExecutionMode.GCP_CLOUD_RUN_JOB), + operator_args={ + "project_id": "my-gcp-project-id", + "region": "europe-west1", + "job_name": "my-crj-{{ ti.task_id.replace('.','-').replace('_','-') }}", + }, + ) + + +AWS ECS +--------- +.. versionadded:: 1.9.0 + +Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. + +This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. + +Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. For users who require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. + +Please refer to the step-by-step guide for using AWS ECS as the execution mode. + +.. code-block:: python + + aws_ecs_cosmos_dag = DbtDag( + # ... + execution_config=ExecutionConfig(execution_mode=ExecutionMode.AWS_ECS), + operator_args={ + "aws_conn_id": "aws_default", + "cluster": "my-ecs-cluster", + "task_definition": "my-dbt-task", + "container_name": "dbt-container", + "launch_type": "FARGATE", + "deferrable": True, + "network_configuration": { + "awsvpcConfiguration": { + "subnets": ["<<>>"], + "assignPublicIp": "ENABLED", + }, + }, + "environment_variables": {"DBT_PROFILE_NAME": "default"}, + }, + ) + +.. _airflow-async-execution-mode: + +Airflow Async +------------- + +.. versionadded:: 1.9.0 + +Although this execution mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. +In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. + +The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's +`Deferrable operators `__. +This execution mode could be preferred when you've long running resources and you want to run them asynchronously by +leveraging Airflow's deferrable operators. With that, you would be able to potentially observe higher throughput of tasks +as more dbt nodes will be run in parallel since they won't be blocking Airflow's worker slots. + +Example DAG: + +.. literalinclude:: ../../dev/dags/simple_dag_async.py + :language: python + :start-after: [START airflow_async_execution_mode_example] + :end-before: [END airflow_async_execution_mode_example] + +For a full step-by-step guide and limitations, check the :ref:`async-execution-mode` page. + + +Watcher Execution Mode (Experimental) +------------------------------------- + +.. versionadded:: 1.11.0 + +The ``watcher`` execution mode is an experimental execution mode that runs a single ``dbt build`` command from a producer task and has sensor tasks to watch the progress of the producer. +It is designed to improve DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. + +Check the :ref:`watcher-execution-mode` for more details. + + +Watcher Kubernetes Execution Mode (Experimental) +------------------------------------------------ + +.. versionadded:: 1.13.0 + +The ``watcher_kubernetes`` execution mode combines the speed of the ``watcher`` execution mode with the isolation of the ``kubernetes`` execution mode. It runs a single ``dbt build`` command from a producer task inside a Kubernetes pod and has sensor tasks to watch the progress of the producer. + +Check the :ref:`watcher-kubernetes-execution-mode` for more details. + + +.. _invocation_modes: + +Invocation Modes +================ +.. versionadded:: 1.4 + +For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: + +1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. + +2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ + In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ + This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. + +The invocation mode can be set in the ``ExecutionConfig`` as shown below: + +.. code-block:: python + + from cosmos.constants import InvocationMode + + dag = DbtDag( + # ... + execution_config=ExecutionConfig( + execution_mode=ExecutionMode.LOCAL, + invocation_mode=InvocationMode.DBT_RUNNER, + ), + ) + +If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. From 7561367b5f781114003e26800885bc344eecac16 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:53:56 -0500 Subject: [PATCH 30/48] move execution methods --- docs/getting_started/astro.rst | 4 +- docs/getting_started/async-execution-mode.rst | 249 --------- .../getting_started/aws-container-run-job.rst | 193 ------- .../azure-container-instance.rst | 138 ----- docs/getting_started/dbt-airflow-concepts.rst | 4 +- docs/getting_started/docker.rst | 111 ---- .../execution-modes-local-conflicts.rst | 133 ----- docs/getting_started/execution-modes.rst | 374 -------------- docs/getting_started/gcp-cloud-run-job.rst | 265 ---------- docs/getting_started/index.rst | 44 +- docs/getting_started/kubernetes.rst | 167 ------ docs/getting_started/mwaa.rst | 4 +- docs/getting_started/open-source.rst | 4 +- .../watcher-execution-mode.rst | 480 ------------------ .../watcher-kubernetes-execution-mode.rst | 214 -------- 15 files changed, 32 insertions(+), 2352 deletions(-) delete mode 100644 docs/getting_started/async-execution-mode.rst delete mode 100644 docs/getting_started/aws-container-run-job.rst delete mode 100644 docs/getting_started/azure-container-instance.rst delete mode 100644 docs/getting_started/docker.rst delete mode 100644 docs/getting_started/execution-modes-local-conflicts.rst delete mode 100644 docs/getting_started/execution-modes.rst delete mode 100644 docs/getting_started/gcp-cloud-run-job.rst delete mode 100644 docs/getting_started/kubernetes.rst delete mode 100644 docs/getting_started/watcher-execution-mode.rst delete mode 100644 docs/getting_started/watcher-kubernetes-execution-mode.rst diff --git a/docs/getting_started/astro.rst b/docs/getting_started/astro.rst index b590575f2e..56e9fa0d53 100644 --- a/docs/getting_started/astro.rst +++ b/docs/getting_started/astro.rst @@ -1,7 +1,7 @@ .. _astro: -Getting Started on Astro -======================== +Getting Started with Cosmos on Astro +==================================== While it is possible to use Cosmos on Astro with all :ref:`Execution Modes `, we recommend using the ``local`` execution mode. It's the simplest to set up and use. diff --git a/docs/getting_started/async-execution-mode.rst b/docs/getting_started/async-execution-mode.rst deleted file mode 100644 index 6d61bcf22b..0000000000 --- a/docs/getting_started/async-execution-mode.rst +++ /dev/null @@ -1,249 +0,0 @@ -.. _async-execution-mode: - -.. title:: Getting Started with Deferrable Operator - -Airflow Async Execution Mode -============================ - -This execution mode can reduce the runtime by 35% in comparison to Cosmos LOCAL execution mode, but is currently only available for BigQuery. While this mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. - -It can be particularly useful for long-running transformations, since it leverages Airflow's `deferrable operators `__. - -In this mode, there is a ``SetupAsyncOperator`` that will pre-generate the SQL files for the dbt project and upload them to Airflow XCom or a remote location. A remote location will only be used if users set ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH`` and ``AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID``. This operator is run before the remaining pipeline. -All the pipeline dbt model transformations will be run using ``DbtRunAirflowAsyncOperator`` which, instead of running the ``dbt run`` command for each model. They will download the SQL files from the Airflow XCom or remote location and execute them directly leveraging the Airflow ``BigQueryInsertJobOperator``. - -Users can leverage other existing ``BigQueryInsertJobOperator`` features, such as the UI controls to link to the job in the BigQuery UI. - - -Advantages of Airflow Async Mode -++++++++++++++++++++++++++++++++ - -- **Improved Task Throughput:** Async tasks free up Airflow workers by leveraging the Airflow Trigger framework. While long-running SQL transformations are executing in the data warehouse, the worker is released and can handle other tasks, increasing overall task throughput. -- **Better Resource Utilization:** By minimizing idle time on Airflow workers, async tasks allow more efficient use of compute resources. Workers aren't blocked waiting for external systems and can be reused for other work while waiting on async operations. -- **Faster Task Execution:** With Cosmos ``SetupAsyncOperator``, the SQL transformations are precompiled and uploaded to XCom (default behaviour) or a remote location. Instead of invoking a full dbt run during each dbt model task, the SQL files are downloaded from this XCom or remote path and executed directly. This eliminates unnecessary overhead from running the full dbt command, resulting in faster and more efficient task execution. - -We have `observed `_ the following performance improvements by running a dbt project with 129 models: - -+----------------------------------------------+--------------------------+ -| How the dbt pipeline was executed | Execution Time (seconds) | -+==============================================+==========================+ -| ``dbt run`` with dbt Core 1.10 | 13 | -+----------------------------------------------+--------------------------+ -| Cosmos 1.11 with ExecutionMode.LOCAL | 11 | -+----------------------------------------------+--------------------------+ -| Cosmos 1.11 with ExecutionMode.AIRFLOW_ASYNC | 7 | -+----------------------------------------------+--------------------------+ - - -Getting Started with Airflow Async Mode -+++++++++++++++++++++++++++++++++++++++ - -This guide walks you through setting up an Astro CLI project and running a Cosmos-based DAG with a deferrable operator, enabling asynchronous task execution in Apache Airflow. - -Prerequisites -+++++++++++++ - -- `Astro CLI `_ -- Airflow>=2.9 - -1. Create Astro-CLI Project -+++++++++++++++++++++++++++ - -Run the following command in your terminal: - -.. code-block:: bash - - astro dev init - -This will create an Astro project with the following structure: - -.. code-block:: bash - - . - ├── Dockerfile - ├── README.md - ├── airflow_settings.yaml - ├── dags/ - ├── include/ - ├── packages.txt - ├── plugins/ - ├── requirements.txt - └── tests/ - - -2. Update Dockerfile -++++++++++++++++++++ - -Edit your Dockerfile to ensure all necessary requirements are included. - -.. code-block:: bash - - FROM astrocrpublic.azurecr.io/runtime:3.0-2 - - -3. Add astronomer-cosmos Dependency -+++++++++++++++++++++++++++++++++++ - -In your ``requirements.txt``, add: - -.. code-block:: bash - - astronomer-cosmos[dbt-bigquery, google]>=1.9 - - -4. Create Airflow DAG -+++++++++++++++++++++ - -1. Create a new DAG file: ``dags/cosmos_async_dag.py`` - -- Update the ``dataset`` and ``project`` - -.. code-block:: python - - import os - from datetime import datetime - from pathlib import Path - - from cosmos import ( - DbtDag, - ExecutionConfig, - ExecutionMode, - ProfileConfig, - ProjectConfig, - ) - from cosmos.constants import TestBehavior - from cosmos.profiles import GoogleCloudServiceAccountDictProfileMapping - - DEFAULT_DBT_ROOT_PATH = Path(__file__).resolve().parent / "dbt" - DBT_ROOT_PATH = Path(os.getenv("DBT_ROOT_PATH", DEFAULT_DBT_ROOT_PATH)) - DBT_ADAPTER_VERSION = os.getenv("DBT_ADAPTER_VERSION", "1.9") - - cosmos_async_dag = DbtDag( - project_config=ProjectConfig( - DBT_ROOT_PATH / "jaffle_shop", - ), - profile_config=ProfileConfig( - profile_name="default", - target_name="dev", - profile_mapping=GoogleCloudServiceAccountDictProfileMapping( - conn_id="gcp_conn", - profile_args={ - "dataset": "cosmos_async_demo", - "project": "astronomer-**", - }, - ), - ), - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AIRFLOW_ASYNC, - async_py_requirements=[f"dbt-bigquery=={DBT_ADAPTER_VERSION}"], - ), - schedule=None, - start_date=datetime(2025, 1, 1), - catchup=False, - dag_id="cosmos_async_dag", - operator_args={ - "location": "US", - "install_deps": True, - "full_refresh": True, - "virtualenv_dir": "dbt_venv", - }, - ) - -2. Folder structure for dbt project - -- Add a valid dbt project inside your Airflow project under ``dags/dbt/``. - - -5. Start the Project -++++++++++++++++++++ - -Launch the Airflow project locally: - -.. code-block:: bash - - astro dev start - -This will: - -- Spin up the scheduler, webserver, and triggerer (needed for deferrable operators) -- Expose Airflow UI at http://localhost:8080 - -6. Create Airflow Connection -++++++++++++++++++++++++++++ - -Create an Airflow connection with following configurations - -- Connection ID: gcp_conn -- Connection Type: google_cloud_platform -- Extra Fields JSON: - -.. code-block:: bash - - { - "project": "astronomer-**", - "keyfile_dict": { - "type": "***", - "project_id": "***", - "private_key_id": "***", - "private_key": "***", - "client_email": "***", - "client_id": "***", - "auth_uri": "***", - "token_uri": "***", - "auth_provider_x509_cert_url": "***", - "client_x509_cert_url": "***", - "universe_domain": "***" - } - } - - -7. Execute the DAG -++++++++++++++++++ - -1. Visit the Airflow UI at ``http://localhost:8080`` -2. Enable the DAG: ``cosmos_async_dag`` -3. Trigger the DAG manually - -.. image:: /_static/jaffle_shop_async_execution_mode.png - :alt: Cosmos dbt Async DAG - :align: center - -The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks. - - -Control of where to upload the SQL files -++++++++++++++++++++++++++++++++++++++++ - -For optimal performance we encourage to keep Cosmos standard behaviour (introduced in 1.11), which is to upload the SQL files to XCom, instead of a remote object location. - -For the benchmark example described in a previous section, there was an overhead of ~500 seconds with remote SQL file upload/download, but only ~2 seconds using XCom, which can outweigh the performance improvements introduced by using deferrable operators. - -However, if you want to upload the SQL files to a remote object location instead of XCom, you can set the following environment variables: - -.. code-block:: bash - - AIRFLOW__COSMOS__REMOTE_TARGET_PATH=gs://cosmos_remote_target_demo - AIRFLOW__COSMOS__REMOTE_TARGET_PATH_CONN_ID=gcp_conn - - -Limitations -+++++++++++ - - -1. **Limited to dbt models**: Only dbt resource type models are run asynchronously using Airflow deferrable operators. Other resource types are executed synchronously, similar to the local execution mode. - -2. **BigQuery support only**: This mode only supports BigQuery as the target database. If a different target is specified, Cosmos will throw an error indicating the target database is unsupported in this mode. Adding support for other adapters is on the roadmap. - -3. **ProfileMapping parameter required**: You need to specify the ``ProfileMapping`` parameter in the ``ProfileConfig`` for your DAG. Refer to the example DAG below for details on setting this parameter. - -4. **Location parameter required**: You must specify the location of the BigQuery dataset in the ``operator_args`` of the ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. - -5. **async_py_requirements parameter required**: If you're using the default approach of having a setup task, you must specify the necessary dbt adapter Python requirements based on your profile type for the async execution mode in the ``ExecutionConfig`` of your ``DbtDag`` or ``DbtTaskGroup``. The example DAG below provides guidance on this. - -6. **Creation of new isolated virtual environment for each task run**: By default, the ``SetupAsyncOperator`` creates and executes within a new isolated virtual environment for each task run, which can cause performance issues. To reuse an existing virtual environment, use the ``virtualenv_dir`` parameter within the ``operator_args`` of the ``DbtDag``. We have observed that for ``dbt-bigquery``, the ``SetupAsyncOperator`` executes approximately 30% faster when reusing an existing virtual environment, particularly for transformations that take around 10 minutes to complete. - -7. **Performance degradation when uploading to remote object location**: Even though it is possible to upload the SQL files to a remote object location by setting environment variables, it is slow. We observed that this introduces a significant overhead in the execution time (500s for 129 models). - -8. **TeardownAsyncOperator limitation**: When using a remote object location, in addition to the ``SetupAsyncOperator``, a ``TeardownAsyncOperator`` is also added to the DAG. This task will delete the SQL files from the remote location by the end of the DAG Run. This is can lead to a limitation from a retry perspective, as described in the issue `#2066 `_. This can be avoided by setting the ``enable_teardown_async_task`` configuration to ``False``, as described in the :ref:`enable_teardown_async_task` section. - -For a comparison between different Cosmos execution modes, please, check the :ref:`execution-modes-comparison` section. diff --git a/docs/getting_started/aws-container-run-job.rst b/docs/getting_started/aws-container-run-job.rst deleted file mode 100644 index db00fc8c3c..0000000000 --- a/docs/getting_started/aws-container-run-job.rst +++ /dev/null @@ -1,193 +0,0 @@ -.. _aws-container-run-job: - -.. title:: Getting Started with Astronomer Cosmos on AWS ECS - -Getting Started with Astronomer Cosmos on AWS ECS -================================================== - -Astronomer Cosmos provides a unified way to run containerized workloads across multiple cloud providers. In this guide, you’ll learn how to deploy and run a Cosmos job on AWS Elastic Container Service (ECS) using Fargate. -Schematically, the guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aws_ecs_schematic.png - :width: 800 - -Prerequisites -+++++++++++++ - -Before you begin, ensure you have the following: - -- An active **AWS Account** with permissions to create ECS clusters, register task definitions, and run tasks. -- The **AWS CLI** installed and configured with the proper credentials. -- **Docker** installed for building your container image. -- Access to your container registry (for example, **Amazon ECR**) where your job image is stored. -- Basic familiarity with AWS ECS concepts (clusters, task definitions, services, and Fargate). -- An existing installation of **Astronomer Cosmos** (refer to the `Cosmos documentation `_ for more details). - - - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: - -.. code-block:: bash - - python3 -m venv venv - source venv/bin/activate - python3 -m pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[amazon]" - pip install "aiobotocore[boto3]" -.. note:: - The package aiobotocore[boto3] is optional; you will need it if you plan to use **deferred tasks**. - -**Set up your ECR** - -1. **Set your secrets** - On the `cosmos-examples `_ repository, you can find a ready-to-use Docker image for the AWS ECS service. Just replace your secrets, or you can create your own. - -2. **AWS CLI login** - Before building and pushing your image, you first need to log in to the AWS service using the AWS CLI tool. - Use the following command: - - .. code-block:: bash - - aws ecr-public get-login-password --region | docker login --username AWS --password-stdin - -3. **Build and tag your image** - Once you have your image ready, run the following commands: - - .. code-block:: bash - - docker build -f Dockerfile.aws_ecs . --platform=linux/amd64 -t - docker tag - -4. **Push your image** - - .. code-block:: bash - - docker push - -**Configure Your AWS Environment** - -1. **Create an ECS Cluster** - - Create an ECS cluster to host your Cosmos jobs. You can do this from the AWS Console or using the AWS CLI: - - .. code-block:: bash - - aws ecs create-cluster --cluster-name my-cosmos-cluster - -2. **Set Up an IAM Role for ECS Tasks** - - Ensure you have an IAM role that your ECS tasks can assume. This role should include permissions for ECS, ECR, and CloudWatch (for logs). For example, you might create a role named ``ecsTaskExecutionRole`` with the managed policies: - - - ``AmazonECSTaskExecutionRolePolicy`` - - (Optional) Additional policies for custom resource access - -3. **Configure Networking** - - For Fargate tasks, make sure you have at least one subnet (preferably in multiple Availability Zones) and a security group that permits outbound internet access if needed. Note the subnet IDs for later use. - -**Prepare Your Cosmos Job Definition** - -Cosmos jobs are defined as container tasks. Create a task definition file (e.g., ``cosmos-task-definition.json``) with the configuration for your job. - -For example: - -.. code-block:: json - - { - "family": "cosmos-job", - "networkMode": "awsvpc", - "requiresCompatibilities": [ - "FARGATE" - ], - "cpu": "512", - "memory": "1024", - "executionRoleArn": "arn:aws:iam:::role/ecsTaskExecutionRole", - "containerDefinitions": [ - { - "name": "cosmos-job", - "image": "/your_image:latest", - "essential": true, - "environment": [ - { "name": "VAR1", "value": "value1" }, - { "name": "VAR2", "value": "value2" } - ], - "logConfiguration": { - "logDriver": "awslogs", - "options": { - "awslogs-group": "/ecs/cosmos-job", - "awslogs-region": "us-east-1", - "awslogs-stream-prefix": "ecs" - } - } - } - ] - } - -.. note:: - - Replace ````, ````, and adjust the CPU, memory, and environment variables as needed. - -**Deploy Your Cosmos Job on AWS ECS** - -1. **Register the Task Definition** - - Use the AWS CLI to register your task definition: - - .. code-block:: bash - - aws ecs register-task-definition --cli-input-json file://cosmos-task-definition.json - -2. **Run the Task** - - Run a test task on your ECS cluster. Specify the subnets and security groups in your network configuration. For example: - - .. code-block:: bash - - aws ecs run-task \ - --cluster my-cosmos-cluster \ - --launch-type FARGATE \ - --task-definition cosmos-job \ - --network-configuration "awsvpcConfiguration={subnets=[subnet-12345678,subnet-87654321],securityGroups=[sg-abcdef12],assignPublicIp=ENABLED}" - - Once the test is ok, we are able to run the dbt commands in our Cosmos DAG: - - .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run.png - :width: 800 - - .. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_aws_ecs_dag_run_logs.png - :width: 800 - - Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! - - -**Monitor and Debug Your Job** - -1. **Check Task Status** - - You can view the status of your task from the AWS Console under your ECS cluster or via the CLI: - - .. code-block:: bash - - aws ecs describe-tasks --cluster my-cosmos-cluster --tasks - -2. **View Logs** - - Since the task definition configures AWS CloudWatch Logs, you can view your job’s output in the CloudWatch Logs console. Look for log streams with the prefix you set (e.g., ``ecs/cosmos-job``). - -**Conclusion** - - -By following this guide, you can deploy Astronomer Cosmos jobs on AWS ECS using Fargate. This integration enables you to leverage the scalability and managed infrastructure of ECS while maintaining a consistent container orchestration experience with Cosmos. - -For more detailed information on AWS ECS, please refer to the `AWS ECS Developer Guide `_. - -Happy deploying! :rocket: - - -Remember to config your DAG for connecting to AWS ECS and the database connection where you are performing your SQL queries! diff --git a/docs/getting_started/azure-container-instance.rst b/docs/getting_started/azure-container-instance.rst deleted file mode 100644 index 86ce3ab9ef..0000000000 --- a/docs/getting_started/azure-container-instance.rst +++ /dev/null @@ -1,138 +0,0 @@ -.. _azure-container-instance: - -Azure Container Instance Execution Mode -======================================= -.. versionadded:: 1.4 - -This tutorial will guide you through the steps required to use Azure Container Instance as the Execution Mode for your dbt code with Astronomer Cosmos. Schematically, the guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_aci_schematic.png - :width: 800 - -Prerequisites -+++++++++++++ -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Azure CLI (install guide here: `Azure CLI `_) -4. Astronomer-cosmos package containing the dbt Azure Container Instance operators -5. Azure account with: - 1. A resource group - 2. A service principal with `Contributor` permissions on the resource group - 3. A Container Registry - 4. A Postgres instance accessible from Azure. (we use an Azure Postgres instance in the example) -6. Docker image built with required dbt project and dbt DAG -7. dbt DAG with dbt Azure Container Instance operators in the Airflow DAGs directory to run in Airflow - -More information on how to achieve 2-6 is detailed below. - -Note that the steps below will walk you through an example, for which the code can be found HERE - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install apache airflow & astronomer-postgres - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-postgres,azure-container-instance]" - -**Setup Postgres database** - -You will need a postgres database running to be used as the database for the dbt project. In order to have it accessible from Azure Container Instance, the easiest way is to create an Azure Postgres instance. For this, run the following (assuming you are logged into your Azure account) - -.. code-block:: bash - - az postgres server create -l westeurope -g <<>> -n <<>> -u dbadmin -p <<>> --sku-name B_Gen5_1 --ssl-enforcement Enabled - - -**Setup Azure Container Registry** -In order to run a container in Azure Container Instance, it needs access to the container image. In our setup, we will use Azure Container Registry for this. To set an Azure Container Registry up, you can use the following bash command: - -.. code-block:: bash - - az acr create --name <<>> --resource-group <<>> --sku Basic --admin-enabled - -**Build the dbt Docker image** - -For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. - -Clone the `cosmos-example `_ repo - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. - -.. code-block:: bash - - docker build -t <<>:1.0.0 -f Dockerfile.azure_container_instance . - -After this, the image needs to be pushed to the registry of your choice. Note that your image name should contain the name of your registry: -.. code-block:: bash - - docker push <<>>:1.0.0 - -.. note:: - - You may need to ensure docker knows to use the right credentials. If using Azure Container Registry, this can be done by running the following command: - .. code-block:: bash - - az acr login - -.. note:: - - If running on M1, you may need to set the following envvars for running the docker build command in case it fails - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The `dbt profile `_ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Setup Airflow Connections** -Now you have the required Azure infrastructure, you still need to add configuration to Airflow to ensure the infrastructure can be used. You'll need 3 connections: - -1. ``aci_db``: a Postgres connection to your Azure Postgres instance. -2. ``aci``: an Azure Container Instance connection configured with a Service Principal with sufficient permissions (i.e. ``Contributor`` on the resource group in which you will use Azure Container Instances). -3. ``acr``: an Azure Container Registry connection configured for your Azure Container Registry. - -Check out the ``airflow-settings.yml`` file `here `_ for an example. If you are using Astro CLI, filling in the right values here will be enough for this to work. - -**Setup and Trigger the DAG with Airflow** - -Copy the dags directory from cosmos-example repo to your Airflow home - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_azure_container_instance `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_azure_container_instance.png - :width: 800 diff --git a/docs/getting_started/dbt-airflow-concepts.rst b/docs/getting_started/dbt-airflow-concepts.rst index 70c4feae8d..ee55abe694 100644 --- a/docs/getting_started/dbt-airflow-concepts.rst +++ b/docs/getting_started/dbt-airflow-concepts.rst @@ -1,7 +1,7 @@ .. _dbt-airflow-concepts: -Similar dbt & Airflow concepts -============================== +Similar dbt and Airflow concepts +================================ While dbt is an open source tool for data transformations and analysis, using SQL, Airflow focuses on being a platform for the development, scheduling and monitoring of batch-oriented workflows, using Python. Although both tools have many diff --git a/docs/getting_started/docker.rst b/docs/getting_started/docker.rst deleted file mode 100644 index 0005914886..0000000000 --- a/docs/getting_started/docker.rst +++ /dev/null @@ -1,111 +0,0 @@ -.. _docker: - -Docker Execution Mode -======================================== - -The following tutorial illustrates how to run the Cosmos dbt Docker Operators and the required setup for them. - -Requirements -++++++++++++ - -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Astronomer-cosmos package containing the dbt Docker operators -4. Postgres docker container -5. Docker image built with required dbt project and dbt DAG -6. dbt DAG with dbt docker operators in the Airflow DAGs directory to run in Airflow - -More information on how to achieve 2-6 is detailed below. - -Step-by-step instructions -+++++++++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install `Apache Airflow® `_ & astronomer-postgres - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-postgres]" - -**Setup Postgres database** - -You will need a postgres database running to be used as the database for the dbt project. Run the following command to run and expose a postgres database - -.. code-block:: bash - - docker run --name some-postgres -e POSTGRES_PASSWORD="" -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p5432:5432 -d postgres - -**Build the dbt Docker image** - -For the Docker operators to work, you need to create a docker image that will be supplied as image parameter to the dbt docker operators used in the DAG. - -Clone the `cosmos-example `_ repo - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Create a docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be supplied to the Docker operators. - -.. code-block:: bash - - docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . - -.. note:: - - If running on M1, you may need to set the following envvars for running the docker build command in case it fails - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The `dbt profile `_ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Setup and Trigger the DAG with Airflow** - -Copy the dags directory from cosmos-example repo to your Airflow home - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_docker `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_docker_dag_run.png - :width: 800 - - -Specifying ProfileConfig -+++++++++++++++++++++++++ - -Starting with Cosmos 1.8.0, you can use the ``profile_config`` argument in your Dbt DAG Docker operators to reference -profiles for your dbt project defined in a profiles.yml file. To do so, provide the file’s path via the -``profiles_yml_path`` parameter in ``profile_config``. - -Note that in ``ExecutionMode.DOCKER``, the ``profile_config`` is only compatible with the ``profiles_yml_path`` -approach. The ``profile_mapping`` method will not work because the required Airflow connections cannot be accessed -within the Docker container to map them to the dbt profile. diff --git a/docs/getting_started/execution-modes-local-conflicts.rst b/docs/getting_started/execution-modes-local-conflicts.rst deleted file mode 100644 index 9fec173751..0000000000 --- a/docs/getting_started/execution-modes-local-conflicts.rst +++ /dev/null @@ -1,133 +0,0 @@ -:orphan: - -.. _execution-modes-local-conflicts: - -Airflow and dbt dependencies conflicts -====================================== - -When using the `Local Execution Mode `__, users may face dependency conflicts between -`Apache Airflow® `_ and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used. - -If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. -With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. - -The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. - -In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): - -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| Airflow / DBT | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | -+===============+=====+=====+=====+=====+=====+=====+=====+=====+=====+=====+======+ -| 2.2 | | | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.3 | x | x | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.4 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.5 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.6 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.7 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.8 | x | x | x | x | x | | x | | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.9 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.10 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.11 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 3.0 | x | x | x | x | x | x | x | x | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ - -Examples of errors ------------------------------------ - -.. code-block:: bash - - The conflict is caused by: - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10 - - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.2.4 depends on jinja2<3.1 and >=2.10.1 - dbt-core 1.5.0 depends on Jinja2==3.1.2 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.6.0 and dbt-core because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.6.0 depends on importlib-metadata<5.0.0 and >=1.7; python_version < "3.9" - dbt-semantic-interfaces 0.1.0.dev7 depends on importlib-metadata==6.6.0 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow, apache-airflow==2.7.0 and dbt-core==1.4.0 because these package versions have conflicting dependencies. - - The conflict is caused by: - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.12.0 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.2 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.1 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.0 depends on PyYAML<6 and >=5.1 - apache-airflow 2.7.0 depends on jsonschema>=4.18.0 - flask-appbuilder 4.3.3 depends on jsonschema<5 and >=3 - connexion 2.10.0 depends on jsonschema<4 and >=2.5.1 - -.. code-block:: bash - -ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. - -The conflict is caused by: - dbt-core 1.10.0 depends on pydantic<2 - apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 - - - -How to reproduce ----------------- - -The table was created by running `nox `__ with the following ``noxfile.py``: - -.. code-block:: python - - import nox - - nox.options.sessions = ["compatibility"] - nox.options.reuse_existing_virtualenvs = True - - - @nox.session(python=["3.10"]) - @nox.parametrize( - "dbt_version", - ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9", "1.10"], - ) - @nox.parametrize( - "airflow_version", - ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "3.0"], - ) - def compatibility(session: nox.Session, airflow_version, dbt_version) -> None: - """Run both unit and integration tests.""" - session.run( - "pip3", - "install", - "--pre", - f"apache-airflow=={airflow_version}", - f"dbt-core=={dbt_version}", - ) diff --git a/docs/getting_started/execution-modes.rst b/docs/getting_started/execution-modes.rst deleted file mode 100644 index ea6a03f283..0000000000 --- a/docs/getting_started/execution-modes.rst +++ /dev/null @@ -1,374 +0,0 @@ -.. _execution-modes: - -Execution Modes -=============== - -Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: - -1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) -2. **virtualenv**: Run ``dbt`` commands from Python virtual environments managed by Cosmos -3. **docker**: Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) -4. **kubernetes**: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) -5. **aws_eks**: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) -6. **azure_container_instance**: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) -7. **gcp_cloud_run_job**: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) -8. **aws_ecs**: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) -9. **airflow_async**: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ -10. **watcher**: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. Check the :ref:`watcher-execution-mode` for more details. -11. **watcher_kubernetes**: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. - -The choice of the ``execution mode`` can vary based on each user's needs and concerns. For more details, check each execution mode described below. - -.. _execution-modes-comparison: - -.. list-table:: Execution Modes Comparison - :widths: 25 25 25 25 - :header-rows: 1 - - * - Execution Mode - - Task Duration - - Environment Isolation - - Cosmos Profile Management - * - Local - - Fast - - None - - Yes - * - Virtualenv - - Medium - - Lightweight - - Yes - * - Docker - - Slow - - Medium - - No - * - Kubernetes - - Slow - - High - - No - * - AWS_EKS - - Slow - - High - - No - * - Azure Container Instance - - Slow - - High - - No - * - GCP Cloud Run Job Instance - - Slow - - High - - No - * - AWS ECS - - Slow - - High - - No - * - Airflow Async - - Very Fast - - Medium - - Yes - * - Watcher - - Very Fast - - None - - Yes - * - Watcher Kubernetes - - Fast - - High - - No - -Local ------ - -By default, Cosmos uses the ``local`` execution mode. - -The ``local`` execution mode is the fastest way to run Cosmos operators since they don't install ``dbt`` nor build docker containers. However, it may not be an option for users using managed Airflow services such as -Google Cloud Composer, since Airflow and ``dbt`` dependencies can conflict (:ref:`execution-modes-local-conflicts`), the user may not be able to install ``dbt`` in a custom path. - -The ``local`` execution mode assumes a ``dbt`` binary is reachable within the Airflow worker node. - -If ``dbt`` was not installed as part of the Cosmos packages, -users can define a custom path to ``dbt`` by declaring the argument ``dbt_executable_path``. - -.. note:: - Starting in the 1.4 version, Cosmos tries to leverage the dbt partial parsing (``partial_parse.msgpack``) to speed up task execution. - This feature is bound to `dbt partial parsing limitations `_. - Learn more: :ref:`partial-parsing`. - -When using the ``local`` execution mode, Cosmos converts Airflow Connections into a native ``dbt`` profiles file (``profiles.yml``). - -Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: - -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py - :language: python - :start-after: [START local_example] - :end-before: [END local_example] - - -Virtualenv ----------- - -If you're using managed Airflow on GCP (Cloud Composer), for instance, we recommend you use the ``virtualenv`` execution mode. - -The ``virtualenv`` mode isolates the Airflow worker dependencies from ``dbt`` by managing a Python virtual environment created during task execution and deleted afterwards. - -In this case, users are responsible for declaring which version of ``dbt`` they want to use by giving the argument ``py_requirements``. This argument can be set directly in operator instances or when instantiating ``DbtDag`` and ``DbtTaskGroup`` as part of ``operator_args``. - -Similar to the ``local`` execution mode, Cosmos converts Airflow Connections into a way ``dbt`` understands them by creating a ``dbt`` profile file (``profiles.yml``). -Also similar to the ``local`` execution mode, Cosmos will by default attempt to use a ``partial_parse.msgpack`` if one exists to speed up parsing. - -Some drawbacks of this approach: - -- It is slower than ``local`` because it creates a new Python virtual environment for each Cosmos dbt task run. -- If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. -- Only ``InvocationMode.SUBPROCESS`` is supported currently, attempt to use ``InvocationMode.DBT_RUNNER`` will raise error. - -Example of how to use: - -.. literalinclude:: ../../dev/dags/example_virtualenv.py - :language: python - :start-after: [START virtualenv_example] - :end-before: [END virtualenv_example] - -Docker ------- - -The ``docker`` approach assumes users have a previously created Docker image, which should contain all the ``dbt`` pipelines and a ``profiles.yml``, managed by the user. - -The user has better environment isolation than when using ``local`` or ``virtualenv`` modes, but also more responsibility (ensuring the Docker container used has up-to-date files and managing secrets potentially in multiple places). - -The other challenge with the ``docker`` approach is if the Airflow worker is already running in Docker, which sometimes can lead to challenges running `Docker in Docker `__. - -This approach can be significantly slower than ``virtualenv`` since it may have to build the ``Docker`` container, which is slower than creating a Virtualenv with ``dbt-core``. -If dbt is unavailable in the Airflow scheduler, the default ``LoadMode.DBT_LS`` will not work. In this scenario, users must use a :ref:`parsing-methods` that does not rely on dbt, such as ``LoadMode.MANIFEST``. - -Check the step-by-step guide on using the ``docker`` execution mode at :ref:`docker`. - -Example DAG: - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.DOCKER, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "network_mode": "bridge", - }, - ) - - -Kubernetes ----------- - -The ``kubernetes`` approach is a very isolated way of running ``dbt`` since the ``dbt`` run commands from within a Kubernetes Pod, usually in a separate host. - -It assumes the user has a Kubernetes cluster. It also expects the user to ensure the Docker container has up-to-date ``dbt`` pipelines and profiles, potentially leading the user to declare secrets in two places (Airflow and Docker container). - -The ``Kubernetes`` deployment may be slower than ``Docker`` and ``Virtualenv`` assuming that the container image is built (which is slower than creating a Python ``virtualenv`` and installing ``dbt-core``) and the Airflow task needs to spin up a new ``Pod`` in Kubernetes. - -Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref:`kubernetes`. - -Example DAG: - -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py - :language: python - :start-after: [START kubernetes_seed_example] - :end-before: [END kubernetes_seed_example] - -AWS_EKS ----------- - -The ``aws_eks`` approach is very similar to the ``kubernetes`` approach, but it is specifically designed to run on AWS EKS clusters. -It uses the `EKSPodOperator `_ -to run the dbt commands. You need to provide the ``cluster_name`` in your operator_args to connect to the AWS EKS cluster. - - -Example DAG: - -.. code-block:: python - - postgres_password_secret = Secret( - deploy_type="env", - deploy_target="POSTGRES_PASSWORD", - secret="postgres-secrets", - key="password", - ) - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AWS_EKS, - ), - operator_args={ - "image": "dbt-jaffle-shop:1.0.0", - "cluster_name": CLUSTER_NAME, - "get_logs": True, - "is_delete_operator_pod": False, - "secrets": [postgres_password_secret], - }, - ) - -Azure Container Instance ------------------------- -.. versionadded:: 1.4 - -Similar to the ``kubernetes`` approach, using ``Azure Container Instances`` as the execution mode gives a very isolated way of running ``dbt``, since the ``dbt`` run itself is run within a container running in an Azure Container Instance. - -This execution mode requires the user has an Azure environment that can be used to run Azure Container Groups in (see :ref:`azure-container-instance` for more details on the exact requirements). Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new container on Azure, giving full isolation. This, however, comes at the cost of speed, as this separation of tasks introduces some overhead. Please checkout the step-by-step guide for using Azure Container Instance as the execution mode - - -.. code-block:: python - - docker_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.AZURE_CONTAINER_INSTANCE - ), - operator_args={ - "ci_conn_id": "aci", - "registry_conn_id": "acr", - "resource_group": "my-rg", - "name": "my-aci-{{ ti.task_id.replace('.','-').replace('_','-') }}", - "region": "West Europe", - "image": "dbt-jaffle-shop:1.0.0", - }, - ) - -GCP Cloud Run Job ------------------------- -.. versionadded:: 1.7 - -The ``gcp_cloud_run_job`` execution mode is particularly useful for users who prefer to run their ``dbt`` commands on Google Cloud infrastructure, taking advantage of Cloud Run's scalability, isolation, and managed service capabilities. - -For the ``gcp_cloud_run_job`` execution mode to work, a Cloud Run Job instance must first be created using a previously built Docker container. This container should include the latest ``dbt`` pipelines and profiles. You can find more details in the `Cloud Run Job creation guide `__ . - -This execution mode allows users to run ``dbt`` core CLI commands in a Google Cloud Run Job instance. This mode leverages the ``CloudRunExecuteJobOperator`` from the Google Cloud Airflow provider to execute commands within a Cloud Run Job instance, where ``dbt`` is already installed. Similarly to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new Cloud Run Job execution, giving full isolation. The separation of tasks adds extra overhead; however, that can be mitigated by using the ``concurrency`` parameter in ``DbtDag``, which will result in parallelized execution of ``dbt`` models. - - -.. code-block:: python - - gcp_cloud_run_job_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.GCP_CLOUD_RUN_JOB), - operator_args={ - "project_id": "my-gcp-project-id", - "region": "europe-west1", - "job_name": "my-crj-{{ ti.task_id.replace('.','-').replace('_','-') }}", - }, - ) - - -AWS ECS ---------- -.. versionadded:: 1.9.0 - -Using ``AWS Elastic Container Service (ECS)`` as the execution mode provides an isolated and scalable way to run ``dbt`` tasks within an AWS ECS service. This execution mode ensures that each ``dbt`` run is performed inside a dedicated container running in an ECS task. - -This execution mode requires the user to have an AWS environment configured to run ECS tasks (see :ref:``aws-ecs`` for more details on the exact requirements). Similar to the ``Docker`` and ``Kubernetes`` execution modes, a Docker container should be available, containing the up-to-date ``dbt`` pipelines and profiles. - -Each task will create a new ECS task execution, providing full isolation. However, this separation introduces some overhead in execution time due to container startup and provisioning. For users who require faster execution times, configuring appropriate ECS task definitions and cluster optimizations can help mitigate these delays. - -Please refer to the step-by-step guide for using AWS ECS as the execution mode. - -.. code-block:: python - - aws_ecs_cosmos_dag = DbtDag( - # ... - execution_config=ExecutionConfig(execution_mode=ExecutionMode.AWS_ECS), - operator_args={ - "aws_conn_id": "aws_default", - "cluster": "my-ecs-cluster", - "task_definition": "my-dbt-task", - "container_name": "dbt-container", - "launch_type": "FARGATE", - "deferrable": True, - "network_configuration": { - "awsvpcConfiguration": { - "subnets": ["<<>>"], - "assignPublicIp": "ENABLED", - }, - }, - "environment_variables": {"DBT_PROFILE_NAME": "default"}, - }, - ) - -.. _airflow-async-execution-mode: - -Airflow Async -------------- - -.. versionadded:: 1.9.0 - -Although this execution mode was introduced in Cosmos 1.9, we strongly encourage users to use Cosmos 1.11, which has significant performance improvements. -In comparison to the ``local``, the ``airflow_async`` execution mode can reduce the execution time of a dbt project by up to 36%. - -The ``airflow_async`` execution mode is a way to run the dbt resources from your dbt project using Apache Airflow's -`Deferrable operators `__. -This execution mode could be preferred when you've long running resources and you want to run them asynchronously by -leveraging Airflow's deferrable operators. With that, you would be able to potentially observe higher throughput of tasks -as more dbt nodes will be run in parallel since they won't be blocking Airflow's worker slots. - -Example DAG: - -.. literalinclude:: ../../dev/dags/simple_dag_async.py - :language: python - :start-after: [START airflow_async_execution_mode_example] - :end-before: [END airflow_async_execution_mode_example] - -For a full step-by-step guide and limitations, check the :ref:`async-execution-mode` page. - - -Watcher Execution Mode (Experimental) -------------------------------------- - -.. versionadded:: 1.11.0 - -The ``watcher`` execution mode is an experimental execution mode that runs a single ``dbt build`` command from a producer task and has sensor tasks to watch the progress of the producer. -It is designed to improve DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. - -Check the :ref:`watcher-execution-mode` for more details. - - -Watcher Kubernetes Execution Mode (Experimental) ------------------------------------------------- - -.. versionadded:: 1.13.0 - -The ``watcher_kubernetes`` execution mode combines the speed of the ``watcher`` execution mode with the isolation of the ``kubernetes`` execution mode. It runs a single ``dbt build`` command from a producer task inside a Kubernetes pod and has sensor tasks to watch the progress of the producer. - -Check the :ref:`watcher-kubernetes-execution-mode` for more details. - - -.. _invocation_modes: - -Invocation Modes -================ -.. versionadded:: 1.4 - -For ``ExecutionMode.LOCAL`` execution mode, Cosmos supports two invocation modes for running dbt: - -1. ``InvocationMode.SUBPROCESS``: In this mode, Cosmos runs dbt cli commands using the Python ``subprocess`` module and parses the output to capture logs and to raise exceptions. - -2. ``InvocationMode.DBT_RUNNER``: In this mode, Cosmos uses the ``dbtRunner`` available for `dbt programmatic invocations `__ to run dbt commands. \ - In order to use this mode, dbt must be installed in the same local environment. This mode does not have the overhead of spawning new subprocesses or parsing the output of dbt commands and is faster than ``InvocationMode.SUBPROCESS``. \ - This mode requires dbt version 1.5.0 or higher. It is up to the user to resolve :ref:`execution-modes-local-conflicts` when using this mode. - -The invocation mode can be set in the ``ExecutionConfig`` as shown below: - -.. code-block:: python - - from cosmos.constants import InvocationMode - - dag = DbtDag( - # ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.LOCAL, - invocation_mode=InvocationMode.DBT_RUNNER, - ), - ) - -If the invocation mode is not set, Cosmos will attempt to use ``InvocationMode.DBT_RUNNER`` if dbt is installed in the same environment as the worker, otherwise it will fall back to ``InvocationMode.SUBPROCESS``. diff --git a/docs/getting_started/gcp-cloud-run-job.rst b/docs/getting_started/gcp-cloud-run-job.rst deleted file mode 100644 index fa4d0c60c4..0000000000 --- a/docs/getting_started/gcp-cloud-run-job.rst +++ /dev/null @@ -1,265 +0,0 @@ -.. _gcp-cloud-run-job: - -GCP Cloud Run Job Execution Mode -======================================= -.. versionadded:: 1.7 - -This tutorial will guide you through the steps required to use Cloud Run Job instance as the Execution Mode for your dbt code with Astronomer Cosmos. This guide will walk you through the steps required to build the following architecture: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/cosmos_gcp_crj_schematic.png - :width: 600 - -Prerequisites -+++++++++++++ -1. Docker with docker daemon (Docker Desktop on MacOS). Follow the `Docker installation guide `_. -2. Airflow -3. Google Cloud SDK (`install guide `_) -4. Astronomer-cosmos package containing the dbt Cloud Run Job operators -5. GCP account with: - 1. A GCP project (`setup guide `_) - 2. IAM roles: - * Basic Role: `Owner `_ (control over whole project) or - * Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) - 3. Enabled service APIs: - * Artifact Registry API - * Cloud Run Admin API - * BigQuery API - 4. A service account with BigQuery roles: `JobUser `_ and `DataEditor `_ -6. Docker image built with required dbt project and dbt DAG -7. dbt DAG with Cloud Run Job operators in the Airflow DAGs directory to run in Airflow - -.. note:: - - Google Cloud Platform provides free tier on many resources, as well as Free Trial with $300 in credit. Learn more `here `_. - -More information on how to achieve 2-6 is detailed below. - - -Step-by-step guide -++++++++++++++++++ - -**Install Airflow and Cosmos** - -Create a python virtualenv, activate it, upgrade pip to the latest version and install ``apache airflow`` & ``astronomer cosmos``: - -.. code-block:: bash - - python3 -m venv venv - source venv/bin/activate - python3 -m pip install --upgrade pip - pip install apache-airflow - pip install "astronomer-cosmos[dbt-bigquery,gcp-cloud-run-job]" - -**Setup gcloud and environment variables** - -Set environment variables that will be used to create cloud infrastructure. Replace placeholders with your unique GCP ``project id`` and ``region`` of the project: - -.. code-block:: bash - - export PROJECT_ID=<<>> - export REGION=<<>> - export REPO_NAME="astronomer-cosmos-dbt" - export IMAGE_NAME="$REGION-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/cosmos-example" - export SERVICE_ACCOUNT_NAME="cloud-run-job-sa" - export DATASET_NAME="astronomer_cosmos_example" - export CLOUD_RUN_JOB_NAME="astronomer-cosmos-example" - -Before we do anything in the GCP project, we first need to authorize gcloud to access the Cloud Platform with Google user credentials: - -.. code-block:: bash - - gcloud auth login - -You'll receive a link to sign into Google Cloud SDK using a Google Account. - -Next, set default ``project id`` using below command: - -.. code-block:: bash - - gcloud config set project $PROJECT_ID - -In case BigQuery has never been used before in the project, run below command to enable BigQuery API: - -.. code-block:: bash - - gcloud services enable bigquery.googleapis.com - -**Setup Artifact Registry** - -In order to run a container in Cloud Run Job, it needs access to the container image. In our setup, we will use Artifact Registry repository that stores images. -To use Artifact Registry, you need to enable the API first: - -.. code-block:: bash - - gcloud services enable artifactregistry.googleapis.com - -To set an Artifact Registry repository up, you can use the following bash command: - -.. code-block:: bash - - gcloud artifacts repositories create $REPO_NAME \ - --repository-format=docker \ - --location=$REGION \ - --project $PROJECT_ID - -**Setup Service Account** - -In order to use dbt and make transformations in BigQuery, Cloud Run Job needs some BigQuery permissions. One way to achieve that is to set up a separate ``Service Account`` with needed permissions: - -.. code-block:: bash - - # create a service account - gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME - -.. code-block:: bash - - # grant JobUser role - gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/bigquery.jobUser" - -.. code-block:: bash - - # grant DataEditor role - gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/bigquery.dataEditor" - -**Build the dbt Docker image** - -Now, we are going to download an example dbt project and build a Docker image with it. - -.. important:: - - You need to ensure Docker is using the right credentials to push images. For Artifact Registry, this can be done by running the following command: - - .. code-block:: bash - - gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://$REGION-docker.pkg.dev - - The token will be valid for 1 hour. After that, you need to create another one, if still needed. - -Clone the `cosmos-example `_ repo: - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example - -Open `Dockerfile `_ located in ``gcp_cloud_run_job_example`` folder and change environments variables ``GCP_PROJECT_ID`` and ``GCP_REGION`` to your GCP project id and project region. - -Build a Docker image using previously modified ``Dockerfile``, which will be used by Cloud Run Job: - -.. code-block:: bash - - docker build -t $IMAGE_NAME -f gcp_cloud_run_job_example/Dockerfile.gcp_cloud_run_job . - -.. important:: - - Make sure to stay in ``cosmos-example`` directory when running ``docker build`` command. - -After this, the image needs to be pushed to the Artifact Registry: - -.. code-block:: bash - - docker push $IMAGE_NAME - -Take a read of the Dockerfile to understand what it does so that you could use it as a reference in your project. - - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The `bigquery dbt profile `_ file is added to the image - - The dbt_project.yml is replaced with `bigquery_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -**Create Cloud Run Job instance** - -When the image is pushed to Artifact Registry, you can finally create Cloud Run Job with the image and previously created service account. - -First, enable Cloud Run Admin API using below command: - -.. code-block:: bash - - gcloud services enable run.googleapis.com - - -Next, set default Cloud Run region to your GCP region: - -.. code-block:: bash - - gcloud config set run/region $REGION - -Then, run below command to create Cloud Run Job instance: - -.. code-block:: bash - - gcloud run jobs create $CLOUD_RUN_JOB_NAME \ - --image=$IMAGE_NAME \ - --task-timeout=180s \ - --max-retries=0 \ - --cpu=1 \ - --memory=512Mi \ - --service-account=$SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com - -**Setup Airflow Connections** - -Now, when you have the required Google Cloud infrastructure, you still need to check Airflow configuration to ensure the infrastructure can be used. You'll need a ``google_cloud_default`` connection in order to work on GCP resources. - -Check out an `example `_ of the ``airflow-settings.yml`` file. If you are using Astro CLI, filling in the right values here will be enough for this to work. - -**Setup and Trigger the DAG with Airflow** - -Open `jaffle_shop_gcp_cloud_run_job `_ DAG file and update ``GCP_PROJECT_ID`` and ``GCP_LOCATION`` constants with your GCP project id and project region. - -When the DAG is configured, copy the ``dags`` directory from ``cosmos-example`` repo to your Airflow home: - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow: - -.. code-block:: bash - - airflow standalone - -.. note:: - - You might need to run airflow standalone with ``sudo`` if your Airflow user is not able to access the docker socket URL or pull the images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_gcp_cloud_run_job `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_gcp_cloud_run_job.png - :width: 800 - - -You can also verify the tables that were created using dbt in BigQuery Studio: - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_big_query.png - :width: 800 - - -**Delete resources** - -After the successful tests, don't forget to delete Google Cloud resources to save up costs: - -.. code-block:: bash - - # Delete Cloud Run Job instance - - gcloud run jobs delete $CLOUD_RUN_JOB_NAME - -.. code-block:: bash - - # Delete BigQuery main and custom dataset specified in dbt schema.yml with all tables included - - bq rm -r -f -d $PROJECT_ID:$DATASET_NAME - - bq rm -r -f -d $PROJECT_ID:dbt_dev - -.. code-block:: bash - - # Delete Artifact Registry repository with all images included - - gcloud artifacts repositories delete $REPO_NAME \ - --location=$REGION diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index eb71d10221..083e03fc48 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -1,28 +1,32 @@ .. _getting-started: .. toctree:: + :maxdepth: 1 :hidden: - :caption: Contents: + :caption: Cosmos Fundamentals Astro CLI quickstart + Similar dbt and Airflow concepts + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Get started with Cosmos + + Open-source Airflow Astro - MWAA - GCC - Open-Source - Execution Modes - Docker Execution Mode - Kubernetes Execution Mode - Azure Container Instance Execution Mode - AWS Container Run Job Execution Mode - GCP Cloud Run Job Execution Mode - Airflow Async Execution Mode - Watcher Execution Mode - Watcher Kubernetes Execution Mode - dbt and Airflow Similar Concepts + Google Cloud Composer (GCC) + Amazon Managed Workflows for Apache Airflow (MWAA) + + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Operators + Operators Custom Airflow Properties - Getting Started =============== @@ -46,11 +50,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/getting_started/kubernetes.rst b/docs/getting_started/kubernetes.rst deleted file mode 100644 index 607ba07bd7..0000000000 --- a/docs/getting_started/kubernetes.rst +++ /dev/null @@ -1,167 +0,0 @@ -.. _kubernetes: - -Kubernetes Execution Mode -============================================== - -The following tutorial illustrates how to run the Cosmos dbt Kubernetes Operator using a local Kubernetes (K8s) cluster. It assumes the following: - -- Postgres is run in the Kubernetes (K8s) cluster as a container -- Airflow is run locally, and it triggers a K8s Pod which runs dbt - -Requirements -++++++++++++ - -To test the DbtKubernetesOperators locally, we encourage you to install the following: - -- Local Airflow (either standalone or using Astro CLI) -- `Kind `_ to run K8s locally -- `Helm `_ to install Postgres in K8s -- `Docker `_ to create the dbt container image, which will allow Airflow to create a K8s pod which will run dbt - -At the moment, the user is expected to add to the Docker image both: - -- The dbt project files -- The dbt Profile, which contains the information for dbt to access the database while parsing the project from Apache Airflow nodes -- Handle secrets - -Additional KubernetesPodOperator parameters can be added to the ``operator_args`` parameter of the ``DbtKubernetesOperator``. - -For instance, - -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py - :language: python - :start-after: [START kubernetes_tg_example] - :end-before: [END kubernetes_tg_example] - -Step-by-step instructions -+++++++++++++++++++++++++ - -Using installed `Kind `_, you can setup a local kubernetes cluster - -.. code-block:: bash - - kind create cluster - -Deploy a Postgres pod to Kind using `Helm `_ - -.. code-block:: bash - - helm repo add bitnami https://charts.bitnami.com/bitnami - helm repo update - helm install postgres bitnami/postgresql - -Retrieve the Postgres password and set it as an environment variable. - -.. code-block:: bash - - export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgres-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d) - -Check that the environment variable was set and that it is not empty - -.. code-block:: bash - - echo $POSTGRES_PASSWORD - -Expose the Postgres to the host running Docker/Kind. - -.. code-block:: bash - - kubectl port-forward --namespace default postgres-postgresql-0 5432:5432 - -Check that you're able to connect to the exposed pod. - -.. code-block:: bash - - PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432 - - postgres=# \dt - \q - -Create a K8s secret which contains the credentials to access Postgres. - -.. code-block:: bash - - kubectl create secret generic postgres-secrets --from-literal=host=postgres-postgresql.default.svc.cluster.local --from-literal=password=$POSTGRES_PASSWORD - -Clone the example repo that contains the Airflow DAG and dbt project files. - -.. code-block:: bash - - git clone https://github.com/astronomer/cosmos-example.git - cd cosmos-example/ - -Create a Docker image containing the dbt project files and dbt profile by using the `Dockerfile `_, which will be run in K8s. - -.. code-block:: bash - - docker build -t dbt-jaffle-shop:1.0.0 -f Dockerfile.postgres_profile_docker_k8s . - -.. note:: - - If running on M1, you may need to set the following environment variables to run the Docker build command in case it fails. - - .. code-block:: bash - - export DOCKER_BUILDKIT=0 - export COMPOSE_DOCKER_CLI_BUILD=0 - export DOCKER_DEFAULT_PLATFORM=linux/amd64 - -Take a look at the Dockerfile to understand its purpose so that you can use it as a reference in your project. - - - The `dbt profile `__ file is added to the image - - The dags directory containing the `dbt project jaffle_shop `_ is added to the image - - The dbt_project.yml is replaced with `postgres_profile_dbt_project.yml `_ which contains the profile key pointing to postgres_profile as profile creation is not handled at the moment for K8s operators like in local mode. - -Make the build image available in the Kind K8s cluster. - -.. code-block:: bash - - kind load docker-image dbt-jaffle-shop:1.0.0 - -Create a Python virtual environment and install the latest version of Astronomer Cosmos, which contains the K8s Operator. - -.. code-block:: bash - - python -m venv venv - source venv/bin/activate - pip install --upgrade pip - pip install "astronomer-cosmos[dbt-postgres]" apache-airflow-providers-cncf-kubernetes - -Make the `jaffle_shop_kubernetes.py `__ file at your Airflow DAG home: - -.. code-block:: bash - - cp -r dags $AIRFLOW_HOME/ - -Run Airflow - -.. code-block:: bash - - airflow standalone - -.. note:: - - You may need to run Airflow standalone with ``sudo`` if your Airflow user is unable to access the Docker socket URL or pull images in the Kind cluster. - -Log in to Airflow through a web browser ``http://localhost:8080/``, using the user ``airflow`` and the password described in the ``standalone_admin_password.txt`` file. - -Enable and trigger a run of the `jaffle_shop_k8s `_ DAG. You will be able to see the following successful DAG run. - -.. figure:: https://github.com/astronomer/astronomer-cosmos/raw/main/docs/_static/jaffle_shop_k8s_dag_run.png - :width: 800 - -.. _kubernetes-known-limitations: - -Known Limitations -+++++++++++++++++ - -The Kubernetes execution mode has the following limitations: - -- Does not emit OpenLineage events (there is an `open ticket #496 `__ to address this) -- Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) -- Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) -- Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) -- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) -- Does not expose Compiled SQL as a `templated field `_ -- Does not benefit from `Cosmos caching mechanisms `_ -- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/getting_started/mwaa.rst b/docs/getting_started/mwaa.rst index 5b7c41bde5..5b1da23439 100644 --- a/docs/getting_started/mwaa.rst +++ b/docs/getting_started/mwaa.rst @@ -1,7 +1,7 @@ .. _mwaa: -Getting Started on MWAA -======================= +Getting Started with Cosmos on Amazon Managed Workflows +======================================================= Users can face Python dependency issues when trying to use the Cosmos `Local Execution Mode `_ in Amazon Managed Workflows for `Apache Airflow® `_ (MWAA). diff --git a/docs/getting_started/open-source.rst b/docs/getting_started/open-source.rst index ba9bbdb15c..f5d1db832b 100644 --- a/docs/getting_started/open-source.rst +++ b/docs/getting_started/open-source.rst @@ -1,7 +1,7 @@ .. _open-source: -Getting Started on Open Source Airflow -====================================== +Getting Started with Cosmos on Open-source Airflow +================================================== When running open-source Airflow, your setup may vary. This guide assumes you have access to edit the underlying image. diff --git a/docs/getting_started/watcher-execution-mode.rst b/docs/getting_started/watcher-execution-mode.rst deleted file mode 100644 index af7589650c..0000000000 --- a/docs/getting_started/watcher-execution-mode.rst +++ /dev/null @@ -1,480 +0,0 @@ -.. _watcher-execution-mode: - -Introducing ``ExecutionMode.WATCHER``: Experimental High-Performance dbt Execution in Cosmos -============================================================================================ - -With the release of **Cosmos 1.11.0**, we are introducing a powerful new experimental execution mode — ``ExecutionMode.WATCHER`` — designed to drastically reduce dbt pipeline run times in Airflow. - -Early benchmarks show that ``ExecutionMode.WATCHER`` can cut total DAG runtime **by up to 80%**, bringing performance **on par with running dbt CLI locally**. Since this execution mode improves the performance by leveraging `dbt threading `_ and Airflow deferrable sensors, the performance gains will depend on three major factors: - -- The amount of dbt ``threads`` set either via the dbt profile configuration or the dbt ``--threads`` flag -- The topology of the dbt pipeline -- The ``poke_interval`` and ``timeout`` settings of the ``DbtConsumerWatcherSensor`` operator, which determine the frequency and duration of the sensor's polling. - -------------------------------------------------------------------------------- - -Background: The Problem with the Local Execution Mode in Cosmos ---------------------------------------------------------------- - -When running dbt via Cosmos using the default ``ExecutionMode.LOCAL``, each dbt model is executed as a separate Airflow task. - -This provides strong observability and task-level retry control — but it comes at a cost. Each model runs a new dbt process, which introduces significant overhead. - -Consider the `google/fhir-dbt-analytics `_ project: - -+-------------------------------------------------------------+-----------------------------------+------------------+ -| Run Type | Description | Total Runtime | -+=============================================================+===================================+==================+ -| Single ``dbt run`` (dbt CLI) | Runs the whole DAG in one command | ~5m 30s | -+-------------------------------------------------------------+-----------------------------------+------------------+ -| One ``dbt run`` per model, totalling 184 commands (dbt CLI) | Each model is its own task | ~32m | -+-------------------------------------------------------------+-----------------------------------+------------------+ - -This difference motivated a rethinking of how Cosmos interacts with dbt. - -------------------------------------------------------------------------------- - -Concept: ``ExecutionMode.WATCHER`` ----------------------------------- - -``ExecutionMode.WATCHER`` combines the **speed of a single dbt run** with the **observability and task management of Airflow**. - -It is built on two operator types: - -* ``DbtProducerWatcherOperator`` (`#1982 `_) - Runs dbt **once** across the entire pipeline, register to `dbt event callbacks `_ and sends model progress updates via Airflow **XComs**. - -* ``DbtConsumerWatcherSensor`` (`#1998 `_) - Watches those XComs and marks individual Airflow tasks as complete when their corresponding dbt models finish. - -Together, these operators let you: - -* Run dbt as a single command (for speed) -* Retain model-level observability (for clarity) -* Retry specific models (for resilience) - -------------------------------------------------------------------------------- - -Performance Gains ------------------ - -We used a dbt project developed by Google, the `google/fhir-dbt-analytics `_ project, that interfaces with BigQuery. It contains: -* 2 seeds -* 52 sources -* 185 models - -Initial benchmarks, using illustrate significant improvements: - -+-----------------------------------------------+-----------+--------------------+ -| Environment | Threads | Execution Time (s) | -+===============================================+===========+====================+ -| dbt build (dbt CLI) | 4 | 6–7 | -+-----------------------------------------------+-----------+--------------------+ -| dbt run per model (dbt CLI) | — | 30 | -| similar to the Cosmos ``ExecutionMode.LOCAL`` | | | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.LOCAL`` (Astro CLI) | — | 10–15 | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.WATCHER`` (Astro CLI) | 1 | 26 | -| | 2 | 14 | -| | 4 | 7 | -| | 8 | 4 | -| | 16 | 2 | -+-----------------------------------------------+-----------+--------------------+ -| Cosmos ``ExecutionMode.WATCHER`` (Astro Cloud | 8 | ≈5 | -| Standard Deployment with A10 workers | | | -+-----------------------------------------------+-----------+--------------------+ - -The last line represents the performance improvement in a real-world Airflow deployment, using `Astro Cloud `_. - -Depending on the dbt workflow topology, if your dbt DAG previously took 5 minutes with ``ExecutionMode.LOCAL``, you can expect it to complete in roughly **1 minute** with ``ExecutionMode.WATCHER``. - -We plan to repeat these benchmarks and share the code with the community in the future. - - -.. note:: - ``ExecutionMode.WATCHER`` relies on the ``threads`` value defined in your dbt profile. Start with a conservative value that matches the CPU capacity of your Airflow workers, then gradually increase it to find the sweet spot between faster runs and acceptable memory/CPU usage. - -When we ran the `astronomer/cosmos-benchmark `_ project with ``ExecutionMode.WATCHER``, that same ``threads`` setting directly affected runtime: moving from 1 to 8 threads reduced the end-to-end ``dbt build`` duration from roughly 26 seconds to about 4 seconds (see table above), while 16 threads squeezed it to around 2 seconds at the cost of higher CPU usage. Use those numbers as a reference point when evaluating how thread counts scale in your own environment. - -To increase the number of threads, edit your dbt ``profiles.yml`` (or Helm values if you manage the profile there) and update the ``threads`` key for the target you use with Cosmos: - -.. code-block:: yaml - - your_dbt_project: - target: prod - outputs: - prod: - type: postgres - host: your-host - user: your-user - password: your-password - schema: analytics - threads: 8 # increase or decrease to match available resources - - -If you prefer to manage threads through Cosmos profile mappings instead of editing ``profiles.yml`` directly, pass ``profile_args={"threads": }`` to your ``ProfileConfig``. For example, using the built-in ``PostgresUserPasswordProfileMapping``: - -.. code-block:: python - - from cosmos.config import ProfileConfig - from cosmos.profiles import PostgresUserPasswordProfileMapping - - profile_config = ProfileConfig( - profile_name="jaffle_shop", - target_name="prod", - profile_mapping=PostgresUserPasswordProfileMapping( - conn_id="postgres_connection", - profile_args={"threads": 8}, - ), - ) - - -------------------------------------------------------------------------------- - -Example Usage of ``ExecutionMode.WATCHER`` ------------------------------------------- - -There are two main ways to use the new execution mode in Cosmos — directly within a ``DbtDag``, or embedded as part of a ``DbtTaskGroup`` inside a larger DAG. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can enable WATCHER mode directly in your ``DbtDag`` configuration. -This approach is best when your Airflow DAG is fully dedicated to a dbt project. - -.. literalinclude:: ../../dev/dags/example_watcher.py - :language: python - :start-after: [START example_watcher] - :end-before: [END example_watcher] - -As it can be observed, the only difference with the default ``ExecutionMode.LOCAL`` is the addition of the ``execution_config`` parameter with the ``execution_mode`` set to ``ExecutionMode.WATCHER``. The ``ExecutionMode`` enum can be imported from ``cosmos.constants``. For more information on the ``ExecutionMode.LOCAL``, please, check the `dedicated page `__ - -**How it works:** - -* Cosmos executes your dbt project once via a producer task. -* Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. -* The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). - -**How it looks like:** - -.. image:: /_static/jaffle_shop_watcher_dbt_dag_dag_run.png - :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` - :align: center - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Example 2 — Using ``DbtTaskGroup`` with ``ExecutionMode.WATCHER`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If your Airflow DAG includes multiple stages or integrations (e.g., data ingestion → dbt → reporting), use ``DbtTaskGroup`` to embed your dbt project into a larger DAG — still benefiting from WATCHER performance. - -.. code-block:: python - :caption: example_watcher_taskgroup.py - :name: example_watcher_taskgroup - - from airflow.models import DAG - from airflow.operators.empty import EmptyOperator - from cosmos import DbtTaskGroup - - with DAG( - dag_id="example_watcher_taskgroup", - schedule="@daily", - start_date=datetime(2023, 1, 1), - catchup=False, - ): - """ - The simplest example of using Cosmos to render a dbt project as a TaskGroup. - """ - pre_dbt = EmptyOperator(task_id="pre_dbt") - - first_dbt_task_group = DbtTaskGroup( - group_id="first_dbt_task_group", - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER, - ), - project_config=ProjectConfig(DBT_PROJECT_PATH), - profile_config=profile_config, - operator_args=operator_args, - ) - - pre_dbt >> first_dbt_task_group - -**Key advantages:** - -* Integrates seamlessly into complex Airflow DAGs. -* Uses the same high-performance producer/consumer execution model. -* Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. - -.. image:: /_static/jaffle_shop_watcher_dbt_taskgroup_dag_run.png - :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` - :align: center - -------------------------------------------------------------------------------- - -Additional details -------------------- - -~~~~~~~~~~~~~~~~ -How retries work -~~~~~~~~~~~~~~~~ - -When the ``dbt build`` command run by ``DbtProducerWatcherOperator`` fails, it will notify all the ``DbtConsumerWatcherSensor``. - -The individual watcher tasks that subclass ``DbtConsumerWatcherSensor`` can retry the dbt command themselves, using the same behavior as ``ExecutionMode.LOCAL``. - -If a branch of the DAG fails, users can clear the status of a failed consumer task, including its downstream tasks, via the Airflow UI, and each of them will run in ``ExecutionMode.LOCAL``. - -**Producer retry behavior** - -.. versionadded:: 1.12.2 - -When the ``DbtProducerWatcherOperator`` is triggered for a retry (try_number > 1), it will not re-run the dbt build command and will succeed. In previous versions of Cosmos, the producer task would fail during retries. -This behavior is designed to support TaskGroup-level retries, as reported in `#2282 `_. - -**Why this matters:** - -- In earlier versions, attempting to retry the producer task would raise an ``AirflowException``, causing the retry to fail immediately. -- Now, the producer gracefully skips execution on retries, logging an informational message explaining that the retry was skipped to avoid running a second ``dbt build``. -- This allows users to retry entire TaskGroups and/or DAGs without the producer task blocking the retry flow. - -**Important considerations:** - -- The producer task should still be configured with ``retries=0`` (which Cosmos enforces by default) to avoid unintended duplicate ``dbt build`` runs. - -- By default, Cosmos sets ``retries`` to ``0`` in``DbtProducerWatcherOperator``. Users can retry manually by clearing the status of the producer task and all its downstream tasks, keeping in mind that the producer task will not re-run the ``dbt build`` command and will succeed. - -The overall retry behavior will be further improved once `#1978 `_ is implemented. - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Watcher dbt Execution Queue -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. versionadded:: 1.14.0 - -In watcher execution mode, by default, consumer sensor tasks are lightweight sensors that wait for the producer task to complete. On their first attempt, they require minimal CPU and memory resources. However, when these tasks retry, they execute the dbt command for the node, which may require significantly more resources. - -The ``watcher_dbt_execution_queue`` configuration allows you to specify a different worker queue for retry attempts. This enables you to: - -- **Optimize resource allocation** — Use lightweight workers for initial sensor execution and high-resource workers for retries -- **Improve scheduling efficiency** — Prevent resource contention between initial sensor tasks and retry executions -- **Scale independently** — Scale retry queues separately based on retry workload patterns - -**Configuration:** - -Set the ``watcher_dbt_execution_queue`` in your Airflow configuration: - -.. code-block:: ini - - [cosmos] - watcher_dbt_execution_queue = high_memory_queue - -Or via environment variable: - -.. code-block:: bash - - export AIRFLOW__COSMOS__WATCHER_DBT_EXECUTION_QUEUE=high_memory_queue - -**How it works:** - -- For watcher producer tasks (``DbtProducerWatcherOperator``), the configured queue is used during their first execution -- For watcher consumer tasks (``DbtConsumerWatcherSensor``), from their first retry onwards, if ``watcher_dbt_execution_queue`` is configured, the task is automatically assigned to the specified queue -- This behavior is enforced by Cosmos via an `Airflow cluster policy `_ (``task_instance_mutation_hook``) that mutates ``task_instance.queue`` at runtime for retry attempts - -.. note:: - - For producer task execution, we encourage users to set the ``watcher_dbt_execution_queue`` configuration. If, for any reason, users prefer to use a different node pool for producer tasks without setting an Airflow Cluster Policy, they can set the ``queue`` argument via ``setup_operator_args``. This, however, would not solve the problem of assigning consumer retries to nodes that may have more memory and CPU available. - - The effective precedence is: - - ``watcher_dbt_execution_queue`` > explicit ``queue`` on the producer (from ``setup_operator_args``) > ``operator_args`` > your Airflow deployment’s default queue. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Installation of Airflow and dbt -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Since Cosmos 1.12.0, ``ExecutionMode.WATCHER`` works well regardless of whether dbt and Airflow are installed in the same Python virtual environment. - -When dbt and Airflow are installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` uses dbt `callback features `_. - -When dbt and Airflow are not installed in the same Python virtual environment, the ``ExecutionMode.WATCHER`` consumes the dbt `structured logging `_ to update the consumer tasks. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Synchronous versus Asynchronous sensor execution -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In Cosmos 1.11.0, the ``DbtConsumerWatcherSensor`` operator is implemented as a synchronous XCom sensor, which continuously occupies the worker slot - even if they're just sleeping and checking periodically. - -Starting with Cosmos 1.12.0, the ``DbtConsumerWatcherSensor`` supports -`deferrable (asynchronous) execution `_. Deferrable execution frees up the Airflow worker slot, while task status monitoring is handled by the Airflow triggerer component, -which increases overall task throughput. By default, the sensor now runs in deferrable mode. - -------------------------------------------------------------------------------- - -Known Limitations -------------------- - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Producer task implementation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The producer task is implemented as a ``DbtProducerWatcherOperator`` and currently relies on dbt being installed alongside the Airflow deployment, as in the ``ExecutionMode.LOCAL`` implementation. - -The alternative to this implementation is to use ``ExecutionMode.WATCHER_KUBERNETES``, which is built on top of ``ExecutionMode.KUBERNETES``. Check :ref:`watcher-kubernetes-execution-mode` for more information. - -~~~~~~~~~~~~~~~~~~~~~~~~ -Individual dbt Operators -~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``ExecutionMode.WATCHER`` efficiently implements the following operators: -* ``DbtSeedWatcherOperator`` -* ``DbtSnapshotWatcherOperator`` -* ``DbtRunWatcherOperator`` - -However, other operators that are available in the ``ExecutionMode.LOCAL`` mode are not implemented. - -The ``DbtBuildWatcherOperator`` is not implemented, since the build command is executed by the producer ``DbtProducerWatcherOperator`` operator. - -Additionally, since the ``dbt build`` command does not run ``source`` nodes, the operator ``DbtSourceWatcherOperator`` is equivalent to the ``DbtSourceLocalOperator`` operator, from ``ExecutionMode.LOCAL``. - -Finally, the following features are not implemented as operators under ``ExecutionMode.WATCHER``: - -* ``dbt ls`` -* ``dbt run-operation`` -* ``dbt docs`` -* ``dbt clone`` - -You can still invoke these operators using the default ``ExecutionMode.LOCAL`` mode. - -~~~~~~~~~~~~~ -Test behavior -~~~~~~~~~~~~~ - -By default, the watcher mode runs tests alongside models via the ``dbt build`` command being executed by the producer ``DbtProducerWatcherOperator`` operator. - -As a starting point, this execution mode does not support the ``TestBehavior.AFTER_EACH`` behavior, since the tests are not run as individual tasks. Since this is the default ``TestBehavior`` in Cosmos, we are injecting ``EmptyOperator`` as a starting point to ensure a seamless transition to the new mode. - -The ``TestBehavior.BUILD`` behavior is embedded in the producer ``DbtProducerWatcherOperator`` operator. - -The ``TestBehavior.NONE`` and ``TestBehavior.AFTER_ALL`` behave similarly to ``ExecutionMode.LOCAL``. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Airflow Datasets and Assets -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -While the ``ExecutionMode.WATCHER`` supports the ``emit_datasets`` parameter, the Airflow Datasets and Assets are emitted from the ``DbtProducerWatcherOperator`` task instead of the consumer tasks, as done for other Cosmos' execution modes. - -~~~~~~~~~~~~~~~~~~~~~~ -Source freshness nodes -~~~~~~~~~~~~~~~~~~~~~~ - -Since Cosmos 1.6, it `supports the rendering of source nodes `_. - -We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. - -This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. - -We have a follow-up ticket to `further investigate this use case `_. - - -Advanced config -------------------- - -~~~~~~~~~~~~~~~~ -Callback support -~~~~~~~~~~~~~~~~ - -The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` will use the user-defined callback function similar to ``ExecutionMode.LOCAL`` mode. - -You can define different ``callback`` behaviors for producer and consumer nodes by using ``operator_args`` to configure the consumer callback and ``setup_operator_args`` to override the callback for the producer, as described below. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Overriding ``operator_args`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``DbtProducerWatcherOperator`` and ``DbtConsumerWatcherSensor`` operators handle ``operator_args`` similar to the ``ExecutionMode.LOCAL`` mode. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Using Custom Args for the Producer and Watcher -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. versionadded:: 1.12.0 - -If you need to override ``operator_args`` for the ``DbtProducerWatcherOperator``, you can do so using ``setup_operator_args``. - -When using ``ExecutionMode.WATCHER``, you may want to configure specific properties, such as ``retries`` specifically for the ``DbtProducerWatcherOperator`` task. This can be useful for several reasons: -- Improved resilience - transient issues (e.g., temporary database or network failures) can be automatically retried. -- Reduced manual intervention - failed producer runs can recover without requiring operator restarts. -- Better reliability - retry behavior can be tuned independently from sensor tasks. - -Example: Configure the producer task with custom retry settings. - -.. code-block:: python - - from datetime import timedelta - from cosmos.config import ExecutionConfig - from cosmos.constants import ExecutionMode - - execution_config = ExecutionConfig( - execution_mode=ExecutionMode.WATCHER, - setup_operator_args={ - "retries": 0, - "retry_delay": timedelta(minutes=5), - }, - ) - -This allows you to customize ``DbtProducerWatcherOperator`` retry behavior without affecting the arguments used by the other sensor tasks. - -If configuring queues, we suggest using the previously mentioned ``watcher_dbt_execution_queue`` configuration instead of the ``setup_operator_args``. - -.. note:: - Please note that ``setup_operator_args`` is specific to Cosmos and is not related to Airflow setup or teardown task. - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Sensor slot allocation and polling -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Each ``DbtDag`` or ``DbtTaskGroup`` root node will startup during DAG runs at - potentially - the same time as the DAG Run. This may not happen, since it is dependent on the -concurrency settings and available task slots in the Airflow deployment. - -The consequence is that tasks may take longer to be updated if they are not sensing at the moment that the transformation happens. - -We plan to review this behaviour and alternative approaches in the future. - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Asynchronous sensor execution -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Deferrable execution is currently supported only for dbt models, seeds and snapshots. -- Deferrable execution applies only to the first task attempt (try number 1). For subsequent retries, the sensor falls back to synchronous execution. - -To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. - -.. literalinclude:: ../../dev/dags/example_watcher.py - :language: python - :start-after: [START example_watcher_synchronous] - :end-before: [END example_watcher_synchronous] - -------------------------------------------------------------------------------- - -Troubleshooting ---------------- - -Problem: "I changed from ``ExecutionMode.LOCAL`` to ``ExecutionMode.WATCHER``, but my DAG is running slower." -Answer: Please, check the number of threads that are being used by searching the producer task logs for a message similar to ``Concurrency: 1 threads (target='DEV')``. To leverage the Watcher mode, you should have a high number of threads, at least dbt's default of 4. Check the `dbt threading docs `_ for more information on how to set the number of threads. - - -Summary -------- - -``ExecutionMode.WATCHER`` represents a significant leap forward for running dbt in Airflow via Cosmos: - -* ✅ Up to **5× faster** dbt DAG runs -* ✅ Maintains **model-level visibility** in Airflow -* ✅ Enables **smarter resource allocation** -* ✅ Built on proven Cosmos rendering techniques - -This is an experimental feature, and we are looking for feedback from the community. - -Stay tuned for further documentation and base image support for the ``ExecutionMode.WATCHER`` in upcoming releases. diff --git a/docs/getting_started/watcher-kubernetes-execution-mode.rst b/docs/getting_started/watcher-kubernetes-execution-mode.rst deleted file mode 100644 index 16dbbffd0a..0000000000 --- a/docs/getting_started/watcher-kubernetes-execution-mode.rst +++ /dev/null @@ -1,214 +0,0 @@ -.. _watcher-kubernetes-execution-mode: - -``ExecutionMode.WATCHER_KUBERNETES``: High-Performance dbt Execution in Kubernetes -=================================================================================== - -.. versionadded:: 1.13.0 - -The ``ExecutionMode.WATCHER_KUBERNETES`` combines the **speed of the** :ref:`watcher-execution-mode` **with the isolation of** :ref:`kubernetes`. - -This execution mode is ideal for users who: - -* Want to leverage the performance benefits of the watcher execution mode -* Need to run dbt in isolated Kubernetes pods -* Prefer not to install dbt in their Airflow deployment - -------------------------------------------------------------------------------- - -Background ----------- - -The :ref:`watcher-execution-mode` introduced in Cosmos 1.11.0 significantly reduces dbt pipeline run times by running dbt as a single command while maintaining model-level observability in Airflow. - -However, the original ``ExecutionMode.WATCHER`` requires dbt to be installed alongside Airflow. The ``ExecutionMode.WATCHER_KUBERNETES`` removes this limitation by running the dbt command inside Kubernetes pods, similar to ``ExecutionMode.KUBERNETES``. - -For more details on the watcher concept and how it works, please refer to the :ref:`watcher-execution-mode` documentation. - -------------------------------------------------------------------------------- - -How to Use ----------- - -Users previously using ``ExecutionMode.KUBERNETES`` can simply replace the ``execution_mode`` to use ``ExecutionMode.WATCHER_KUBERNETES``. - -The following example shows how to configure a ``DbtDag`` with ``ExecutionMode.WATCHER_KUBERNETES``: - -.. code-block:: python - - from cosmos import DbtDag - from cosmos.config import ExecutionConfig - from cosmos.constants import ExecutionMode - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - "image": DBT_IMAGE, - "get_logs": True, - "log_events_on_failure": True, - }, - ) - -**Key differences from** ``ExecutionMode.KUBERNETES``: - -* The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` -* The producer task runs the entire ``dbt build`` command in a single Kubernetes pod -* Consumer tasks (sensors) watch for the completion of their corresponding dbt models - -For the complete setup including Kubernetes secrets, Docker image configuration, and profile setup, refer to the :ref:`kubernetes` documentation. - -------------------------------------------------------------------------------- - -Performance Gains ------------------ - -Early benchmarks using the ``jaffle_shop_watcher_kubernetes`` DAG show significant improvements: - -+-----------------------------------------------+------------------+ -| Execution Mode | Total Runtime | -+===============================================+==================+ -| ``ExecutionMode.KUBERNETES`` | 00:00:32.155 | -+-----------------------------------------------+------------------+ -| ``ExecutionMode.WATCHER_KUBERNETES`` | 00:00:11.783 | -+-----------------------------------------------+------------------+ - -This represents approximately a **63% reduction** in total DAG runtime. - -The performance improvement comes from: - -* Running dbt as a single command (reducing Kubernetes pod startup overhead) -* Leveraging dbt's native threading capabilities -* Eliminating repeated dbt initialization for each model - -------------------------------------------------------------------------------- - -Known Limitations ------------------ - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Kubernetes Provider Version Compatibility -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -``ExecutionMode.WATCHER_KUBERNETES`` does not work with older versions of the ``apache-airflow-providers-cncf-kubernetes`` provider (<=10.7.0). - -Please ensure you have a compatible version installed: - -.. code-block:: bash - - pip install "apache-airflow-providers-cncf-kubernetes>10.7.0" - -We successfully tested against the most recent release of the provider (`10.12.2 `_). - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Support for KPO deferrable mode -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The producer node created by the ``ExecutionMode.WATCHER_KUBERNETES`` producer task can be set to deferrable mode as long as: - -- The correct version of Airflow Kubernetes is installed (``>=10.12.2``). This version fixed a bug (`PR `_) that prevented setting callbacks and parsing the logs when the Kubernetes Operator run using ``deferrable``. The experience should be further improved once `this other PR is merged `_. - -.. code-block:: bash - - pip install "apache-airflow-providers-cncf-kubernetes>=10.12.2" - -- The arguments ``deferrable=True`` and ``is_delete_operator_pod=True`` are set: - -.. code-block:: python - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - "deferrable": True, - "is_delete_operator_pod": True, - "image": DBT_IMAGE, - "get_logs": True, - "log_events_on_failure": True, - }, - ) - -Conversely, the consumer tasks that subclass ``DbtConsumerWatcherKubernetesSensor`` run in deferrable mode by default when operating as a sensor. They can also operate in deferrable mode if they are running dbt themselves upon retry. - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Mandatory ``operator_args`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``operator_args`` must define ``get_logs`` and ``log_events_on_failure``: - -.. code-block: python - - dag = DbtDag( - dag_id="jaffle_shop_watcher_kubernetes", - # ... other DAG parameters ... - execution_config=ExecutionConfig( - execution_mode=ExecutionMode.WATCHER_KUBERNETES, - dbt_project_path=K8S_PROJECT_DIR, - ), - operator_args={ - # ... other KPO mandatory args ... - "get_logs": True, - "log_events_on_failure": True, - }, - ) - - -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Other Inherited Limitations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The following limitations from ``ExecutionMode.WATCHER`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``: - -* **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. - -* **Test behavior**: The ``TestBehavior.AFTER_EACH`` is not supported. Tests are run as part of the ``dbt build`` command by the producer task. - -* **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. - -For more details on these limitations, refer to the :ref:`watcher-execution-mode` documentation. - -Additionally, the limitations from ``ExecutionMode.KUBERNETES`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``. For details, refer to the :ref:`kubernetes-known-limitations` documentation. - -------------------------------------------------------------------------------- - -Example DAG ------------ - -Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: - -.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py - :language: python - -------------------------------------------------------------------------------- - -Prerequisites -------------- - -Before using ``ExecutionMode.WATCHER_KUBERNETES``, ensure you have: - -1. A Kubernetes cluster configured and accessible from your Airflow deployment -2. A Docker image containing your dbt project and profile -3. The ``apache-airflow-providers-cncf-kubernetes`` provider installed (version >10.7.0) - -For detailed setup instructions, refer to the :ref:`kubernetes` documentation. - -------------------------------------------------------------------------------- - -Summary -------- - -``ExecutionMode.WATCHER_KUBERNETES`` provides: - -* ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` -* ✅ **Isolation** between dbt and Airflow dependencies -* ✅ **Model-level visibility** in Airflow -* ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` - -This execution mode is ideal for teams who want the performance benefits of the watcher mode while maintaining the isolation provided by Kubernetes execution. From 0851001739a9ce202413f181127d3099d1d71c7f Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 16:57:44 -0500 Subject: [PATCH 31/48] rename to guides --- docs/{configuration => guides}/caching.rst | 0 docs/{configuration => guides}/callbacks/callbacks.rst | 0 docs/{configuration => guides}/compiled-sql.rst | 0 .../configure-tests/testing-behavior.rst | 0 docs/{configuration => guides}/cosmos-conf.rst | 0 docs/{configuration => guides}/dag-customization.rst | 0 docs/{configuration => guides}/dbt-docs/generating-docs.rst | 0 docs/{configuration => guides}/dbt-docs/hosting-docs.rst | 0 docs/{configuration => guides}/dbt-fusion.rst | 0 docs/{configuration => guides}/execution-config.rst | 0 .../{configuration => guides}/execution-modes-local-conflicts.rst | 0 docs/{configuration => guides}/index.rst | 0 docs/{configuration => guides}/lineage.rst | 0 docs/{configuration => guides}/logging.rst | 0 docs/{configuration => guides}/memory_optimization.rst | 0 docs/{configuration => guides}/multi-project.rst | 0 docs/{configuration => guides}/operator-args.rst | 0 docs/{configuration => guides}/parsing-methods.rst | 0 docs/{configuration => guides}/partial-parsing.rst | 0 docs/{configuration => guides}/profile-config.rst | 0 docs/{configuration => guides}/project-config.rst | 0 docs/{configuration => guides}/render-config.rst | 0 .../run-dbt/airflow-worker/async-execution-mode.rst | 0 docs/{configuration => guides}/run-dbt/airflow-worker/index.rst | 0 .../run-dbt/airflow-worker/watcher-execution-mode.rst | 0 .../run-dbt/container/aws-container-run-job.rst | 0 .../run-dbt/container/azure-container-instance.rst | 0 docs/{configuration => guides}/run-dbt/container/docker.rst | 0 .../run-dbt/container/gcp-cloud-run-job.rst | 0 docs/{configuration => guides}/run-dbt/container/index.rst | 0 docs/{configuration => guides}/run-dbt/container/kubernetes.rst | 0 .../run-dbt/container/watcher-kubernetes-execution-mode.rst | 0 docs/{configuration => guides}/run-dbt/execution-modes.rst | 0 docs/{configuration => guides}/scheduling.rst | 0 docs/{configuration => guides}/selecting-excluding.rst | 0 docs/{configuration => guides}/source-nodes-rendering.rst | 0 docs/{configuration => guides}/task-display-name.rst | 0 37 files changed, 0 insertions(+), 0 deletions(-) rename docs/{configuration => guides}/caching.rst (100%) rename docs/{configuration => guides}/callbacks/callbacks.rst (100%) rename docs/{configuration => guides}/compiled-sql.rst (100%) rename docs/{configuration => guides}/configure-tests/testing-behavior.rst (100%) rename docs/{configuration => guides}/cosmos-conf.rst (100%) rename docs/{configuration => guides}/dag-customization.rst (100%) rename docs/{configuration => guides}/dbt-docs/generating-docs.rst (100%) rename docs/{configuration => guides}/dbt-docs/hosting-docs.rst (100%) rename docs/{configuration => guides}/dbt-fusion.rst (100%) rename docs/{configuration => guides}/execution-config.rst (100%) rename docs/{configuration => guides}/execution-modes-local-conflicts.rst (100%) rename docs/{configuration => guides}/index.rst (100%) rename docs/{configuration => guides}/lineage.rst (100%) rename docs/{configuration => guides}/logging.rst (100%) rename docs/{configuration => guides}/memory_optimization.rst (100%) rename docs/{configuration => guides}/multi-project.rst (100%) rename docs/{configuration => guides}/operator-args.rst (100%) rename docs/{configuration => guides}/parsing-methods.rst (100%) rename docs/{configuration => guides}/partial-parsing.rst (100%) rename docs/{configuration => guides}/profile-config.rst (100%) rename docs/{configuration => guides}/project-config.rst (100%) rename docs/{configuration => guides}/render-config.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/async-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/index.rst (100%) rename docs/{configuration => guides}/run-dbt/airflow-worker/watcher-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/container/aws-container-run-job.rst (100%) rename docs/{configuration => guides}/run-dbt/container/azure-container-instance.rst (100%) rename docs/{configuration => guides}/run-dbt/container/docker.rst (100%) rename docs/{configuration => guides}/run-dbt/container/gcp-cloud-run-job.rst (100%) rename docs/{configuration => guides}/run-dbt/container/index.rst (100%) rename docs/{configuration => guides}/run-dbt/container/kubernetes.rst (100%) rename docs/{configuration => guides}/run-dbt/container/watcher-kubernetes-execution-mode.rst (100%) rename docs/{configuration => guides}/run-dbt/execution-modes.rst (100%) rename docs/{configuration => guides}/scheduling.rst (100%) rename docs/{configuration => guides}/selecting-excluding.rst (100%) rename docs/{configuration => guides}/source-nodes-rendering.rst (100%) rename docs/{configuration => guides}/task-display-name.rst (100%) diff --git a/docs/configuration/caching.rst b/docs/guides/caching.rst similarity index 100% rename from docs/configuration/caching.rst rename to docs/guides/caching.rst diff --git a/docs/configuration/callbacks/callbacks.rst b/docs/guides/callbacks/callbacks.rst similarity index 100% rename from docs/configuration/callbacks/callbacks.rst rename to docs/guides/callbacks/callbacks.rst diff --git a/docs/configuration/compiled-sql.rst b/docs/guides/compiled-sql.rst similarity index 100% rename from docs/configuration/compiled-sql.rst rename to docs/guides/compiled-sql.rst diff --git a/docs/configuration/configure-tests/testing-behavior.rst b/docs/guides/configure-tests/testing-behavior.rst similarity index 100% rename from docs/configuration/configure-tests/testing-behavior.rst rename to docs/guides/configure-tests/testing-behavior.rst diff --git a/docs/configuration/cosmos-conf.rst b/docs/guides/cosmos-conf.rst similarity index 100% rename from docs/configuration/cosmos-conf.rst rename to docs/guides/cosmos-conf.rst diff --git a/docs/configuration/dag-customization.rst b/docs/guides/dag-customization.rst similarity index 100% rename from docs/configuration/dag-customization.rst rename to docs/guides/dag-customization.rst diff --git a/docs/configuration/dbt-docs/generating-docs.rst b/docs/guides/dbt-docs/generating-docs.rst similarity index 100% rename from docs/configuration/dbt-docs/generating-docs.rst rename to docs/guides/dbt-docs/generating-docs.rst diff --git a/docs/configuration/dbt-docs/hosting-docs.rst b/docs/guides/dbt-docs/hosting-docs.rst similarity index 100% rename from docs/configuration/dbt-docs/hosting-docs.rst rename to docs/guides/dbt-docs/hosting-docs.rst diff --git a/docs/configuration/dbt-fusion.rst b/docs/guides/dbt-fusion.rst similarity index 100% rename from docs/configuration/dbt-fusion.rst rename to docs/guides/dbt-fusion.rst diff --git a/docs/configuration/execution-config.rst b/docs/guides/execution-config.rst similarity index 100% rename from docs/configuration/execution-config.rst rename to docs/guides/execution-config.rst diff --git a/docs/configuration/execution-modes-local-conflicts.rst b/docs/guides/execution-modes-local-conflicts.rst similarity index 100% rename from docs/configuration/execution-modes-local-conflicts.rst rename to docs/guides/execution-modes-local-conflicts.rst diff --git a/docs/configuration/index.rst b/docs/guides/index.rst similarity index 100% rename from docs/configuration/index.rst rename to docs/guides/index.rst diff --git a/docs/configuration/lineage.rst b/docs/guides/lineage.rst similarity index 100% rename from docs/configuration/lineage.rst rename to docs/guides/lineage.rst diff --git a/docs/configuration/logging.rst b/docs/guides/logging.rst similarity index 100% rename from docs/configuration/logging.rst rename to docs/guides/logging.rst diff --git a/docs/configuration/memory_optimization.rst b/docs/guides/memory_optimization.rst similarity index 100% rename from docs/configuration/memory_optimization.rst rename to docs/guides/memory_optimization.rst diff --git a/docs/configuration/multi-project.rst b/docs/guides/multi-project.rst similarity index 100% rename from docs/configuration/multi-project.rst rename to docs/guides/multi-project.rst diff --git a/docs/configuration/operator-args.rst b/docs/guides/operator-args.rst similarity index 100% rename from docs/configuration/operator-args.rst rename to docs/guides/operator-args.rst diff --git a/docs/configuration/parsing-methods.rst b/docs/guides/parsing-methods.rst similarity index 100% rename from docs/configuration/parsing-methods.rst rename to docs/guides/parsing-methods.rst diff --git a/docs/configuration/partial-parsing.rst b/docs/guides/partial-parsing.rst similarity index 100% rename from docs/configuration/partial-parsing.rst rename to docs/guides/partial-parsing.rst diff --git a/docs/configuration/profile-config.rst b/docs/guides/profile-config.rst similarity index 100% rename from docs/configuration/profile-config.rst rename to docs/guides/profile-config.rst diff --git a/docs/configuration/project-config.rst b/docs/guides/project-config.rst similarity index 100% rename from docs/configuration/project-config.rst rename to docs/guides/project-config.rst diff --git a/docs/configuration/render-config.rst b/docs/guides/render-config.rst similarity index 100% rename from docs/configuration/render-config.rst rename to docs/guides/render-config.rst diff --git a/docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/async-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/async-execution-mode.rst rename to docs/guides/run-dbt/airflow-worker/async-execution-mode.rst diff --git a/docs/configuration/run-dbt/airflow-worker/index.rst b/docs/guides/run-dbt/airflow-worker/index.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/index.rst rename to docs/guides/run-dbt/airflow-worker/index.rst diff --git a/docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/airflow-worker/watcher-execution-mode.rst rename to docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst diff --git a/docs/configuration/run-dbt/container/aws-container-run-job.rst b/docs/guides/run-dbt/container/aws-container-run-job.rst similarity index 100% rename from docs/configuration/run-dbt/container/aws-container-run-job.rst rename to docs/guides/run-dbt/container/aws-container-run-job.rst diff --git a/docs/configuration/run-dbt/container/azure-container-instance.rst b/docs/guides/run-dbt/container/azure-container-instance.rst similarity index 100% rename from docs/configuration/run-dbt/container/azure-container-instance.rst rename to docs/guides/run-dbt/container/azure-container-instance.rst diff --git a/docs/configuration/run-dbt/container/docker.rst b/docs/guides/run-dbt/container/docker.rst similarity index 100% rename from docs/configuration/run-dbt/container/docker.rst rename to docs/guides/run-dbt/container/docker.rst diff --git a/docs/configuration/run-dbt/container/gcp-cloud-run-job.rst b/docs/guides/run-dbt/container/gcp-cloud-run-job.rst similarity index 100% rename from docs/configuration/run-dbt/container/gcp-cloud-run-job.rst rename to docs/guides/run-dbt/container/gcp-cloud-run-job.rst diff --git a/docs/configuration/run-dbt/container/index.rst b/docs/guides/run-dbt/container/index.rst similarity index 100% rename from docs/configuration/run-dbt/container/index.rst rename to docs/guides/run-dbt/container/index.rst diff --git a/docs/configuration/run-dbt/container/kubernetes.rst b/docs/guides/run-dbt/container/kubernetes.rst similarity index 100% rename from docs/configuration/run-dbt/container/kubernetes.rst rename to docs/guides/run-dbt/container/kubernetes.rst diff --git a/docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst similarity index 100% rename from docs/configuration/run-dbt/container/watcher-kubernetes-execution-mode.rst rename to docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst diff --git a/docs/configuration/run-dbt/execution-modes.rst b/docs/guides/run-dbt/execution-modes.rst similarity index 100% rename from docs/configuration/run-dbt/execution-modes.rst rename to docs/guides/run-dbt/execution-modes.rst diff --git a/docs/configuration/scheduling.rst b/docs/guides/scheduling.rst similarity index 100% rename from docs/configuration/scheduling.rst rename to docs/guides/scheduling.rst diff --git a/docs/configuration/selecting-excluding.rst b/docs/guides/selecting-excluding.rst similarity index 100% rename from docs/configuration/selecting-excluding.rst rename to docs/guides/selecting-excluding.rst diff --git a/docs/configuration/source-nodes-rendering.rst b/docs/guides/source-nodes-rendering.rst similarity index 100% rename from docs/configuration/source-nodes-rendering.rst rename to docs/guides/source-nodes-rendering.rst diff --git a/docs/configuration/task-display-name.rst b/docs/guides/task-display-name.rst similarity index 100% rename from docs/configuration/task-display-name.rst rename to docs/guides/task-display-name.rst From 0769ac0051639d80d6d235cf980f3e2e0a6156a1 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 17:26:26 -0500 Subject: [PATCH 32/48] update hradcoded URLs --- docs/getting_started/index.rst | 10 +++++----- docs/guides/execution-modes-local-conflicts.rst | 4 ++-- docs/guides/index.rst | 6 +++--- .../run-dbt/airflow-worker/watcher-execution-mode.rst | 2 +- docs/guides/run-dbt/container/kubernetes.rst | 8 ++++---- docs/index.rst | 2 +- 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 083e03fc48..20c689e0b9 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -50,11 +50,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/guides/execution-modes-local-conflicts.rst b/docs/guides/execution-modes-local-conflicts.rst index 9fec173751..0f9120127c 100644 --- a/docs/guides/execution-modes-local-conflicts.rst +++ b/docs/guides/execution-modes-local-conflicts.rst @@ -10,8 +10,8 @@ When using the `Local Execution Mode `__, users may If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. With the `Local Execution Mode `__, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../configuration/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../configuration/render-config.html>`_ parameters. +Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../guides/execution-config.html>`_ and +`RenderConfig.dbt_executable_path <../guides/render-config.html>`_ parameters. The page `execution modes `__ describes many other methods that support isolating dbt from Airflow. diff --git a/docs/guides/index.rst b/docs/guides/index.rst index d699e6189e..df227dea90 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -1,7 +1,7 @@ -.. _configuration: +.. _guides: -Configuration -============= +Guides +====== Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. diff --git a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst index af7589650c..f33ef15900 100644 --- a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst @@ -370,7 +370,7 @@ Source freshness nodes Since Cosmos 1.6, it `supports the rendering of source nodes `_. -We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. +We noticed some Cosmos users use this feature alongside `overriding Cosmos source nodes `_ as sensors or another operator that allows them to skip the following branch of the DAG if the source is not fresh. This use case is not currently supported by the ``ExecutionMode.WATCHER``, since the ``dbt build`` command does not run `source freshness checks `_. diff --git a/docs/guides/run-dbt/container/kubernetes.rst b/docs/guides/run-dbt/container/kubernetes.rst index 607ba07bd7..4ea8ccd4b9 100644 --- a/docs/guides/run-dbt/container/kubernetes.rst +++ b/docs/guides/run-dbt/container/kubernetes.rst @@ -161,7 +161,7 @@ The Kubernetes execution mode has the following limitations: - Does not emit Airflow datasets, assets, and dataset aliases (there is an `open ticket #2329 `__ to address this) - Does not handle installing dbt deps for users (there is an `open ticket #679 `__ to address this) - Does not support `ProfileMapping `_ (there is an `open ticket #749 `__ to address this) -- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) -- Does not expose Compiled SQL as a `templated field `_ -- Does not benefit from `Cosmos caching mechanisms `_ -- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) +- Does not support `Callbacks `_ (there is an `open ticket #1575 `__ to address this) +- Does not expose Compiled SQL as a `templated field `_ +- Does not benefit from `Cosmos caching mechanisms `_ +- Does not support `generating dbt docs & uploading to an object store `_ (there is a `PR `_ to solve this for S3) diff --git a/docs/index.rst b/docs/index.rst index beee4f40bb..8926b5c198 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,7 +7,7 @@ Home Getting Started - Configuration + Guides Profiles Contributing Airflow 3 compatibility From 0018a461df6d7e494b8a9c1fc88500cd723e721c Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Fri, 27 Feb 2026 18:08:47 -0500 Subject: [PATCH 33/48] redistribute files --- docs/contributing.rst | 4 +++- docs/guides/{ => cosmos_devex}/compiled-sql.rst | 0 docs/guides/{ => cosmos_devex}/lineage.rst | 0 docs/guides/{ => cosmos_devex}/logging.rst | 0 docs/guides/{dbt-docs => dbt_docs}/generating-docs.rst | 0 docs/guides/{dbt-docs => dbt_docs}/hosting-docs.rst | 0 docs/guides/{ => dbt_setup}/dbt-fusion.rst | 0 docs/guides/{ => multi_project}/multi-project.rst | 0 docs/guides/{ => optimize_performance}/caching.rst | 0 .../{ => optimize_performance}/memory_optimization.rst | 0 .../{ => optimize_performance}/partial-parsing.rst | 0 .../{ => optimize_performance}/selecting-excluding.rst | 0 .../airflow-worker/async-execution-mode.rst | 0 .../airflow-worker}/execution-modes-local-conflicts.rst | 0 .../guides/{run-dbt => run_dbt}/airflow-worker/index.rst | 0 .../airflow-worker/watcher-execution-mode.rst | 0 docs/guides/{ => run_dbt}/callbacks/callbacks.rst | 0 .../container/aws-container-run-job.rst | 0 .../container/azure-container-instance.rst | 0 docs/guides/{run-dbt => run_dbt}/container/docker.rst | 0 .../{run-dbt => run_dbt}/container/gcp-cloud-run-job.rst | 0 docs/guides/{run-dbt => run_dbt}/container/index.rst | 0 .../guides/{run-dbt => run_dbt}/container/kubernetes.rst | 0 .../container/watcher-kubernetes-execution-mode.rst | 0 docs/guides/run_dbt/customization/index.rst | 9 +++++++++ .../guides/{ => run_dbt/customization}/operator-args.rst | 0 docs/guides/{ => run_dbt/customization}/scheduling.rst | 0 .../{ => run_dbt/customization}/task-display-name.rst | 0 .../{run-dbt/execution-modes.rst => run_dbt/index.rst} | 3 +++ .../run_dbt/operators}/operators.rst | 0 .../configure-tests/testing-behavior.rst | 0 .../custom-airflow-properties.rst | 0 .../{ => translate_dbt_to_airflow}/dag-customization.rst | 0 .../{ => translate_dbt_to_airflow}/parsing-methods.rst | 0 .../{ => translate_dbt_to_airflow}/render-config.rst | 0 .../source-nodes-rendering.rst | 0 docs/index.rst | 2 +- 37 files changed, 16 insertions(+), 2 deletions(-) rename docs/guides/{ => cosmos_devex}/compiled-sql.rst (100%) rename docs/guides/{ => cosmos_devex}/lineage.rst (100%) rename docs/guides/{ => cosmos_devex}/logging.rst (100%) rename docs/guides/{dbt-docs => dbt_docs}/generating-docs.rst (100%) rename docs/guides/{dbt-docs => dbt_docs}/hosting-docs.rst (100%) rename docs/guides/{ => dbt_setup}/dbt-fusion.rst (100%) rename docs/guides/{ => multi_project}/multi-project.rst (100%) rename docs/guides/{ => optimize_performance}/caching.rst (100%) rename docs/guides/{ => optimize_performance}/memory_optimization.rst (100%) rename docs/guides/{ => optimize_performance}/partial-parsing.rst (100%) rename docs/guides/{ => optimize_performance}/selecting-excluding.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/async-execution-mode.rst (100%) rename docs/guides/{ => run_dbt/airflow-worker}/execution-modes-local-conflicts.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/index.rst (100%) rename docs/guides/{run-dbt => run_dbt}/airflow-worker/watcher-execution-mode.rst (100%) rename docs/guides/{ => run_dbt}/callbacks/callbacks.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/aws-container-run-job.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/azure-container-instance.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/docker.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/gcp-cloud-run-job.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/index.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/kubernetes.rst (100%) rename docs/guides/{run-dbt => run_dbt}/container/watcher-kubernetes-execution-mode.rst (100%) create mode 100644 docs/guides/run_dbt/customization/index.rst rename docs/guides/{ => run_dbt/customization}/operator-args.rst (100%) rename docs/guides/{ => run_dbt/customization}/scheduling.rst (100%) rename docs/guides/{ => run_dbt/customization}/task-display-name.rst (100%) rename docs/guides/{run-dbt/execution-modes.rst => run_dbt/index.rst} (99%) rename docs/{getting_started => guides/run_dbt/operators}/operators.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/configure-tests/testing-behavior.rst (100%) rename docs/{getting_started => guides/translate_dbt_to_airflow}/custom-airflow-properties.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/dag-customization.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/parsing-methods.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/render-config.rst (100%) rename docs/guides/{ => translate_dbt_to_airflow}/source-nodes-rendering.rst (100%) diff --git a/docs/contributing.rst b/docs/contributing.rst index 006149faac..d50c120398 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -155,17 +155,19 @@ To run the checks manually, run: Writing Docs ____________ -`Hatch `_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatchto declare the dependencies required for the project itself, as well as for tests and documentation builds. +`Hatch `_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatch to declare the dependencies required for the project itself, as well as for tests and documentation builds. If you don’t already have Hatch installed, please `install it `_ before proceeding. As an example, on macOS, you can do so with: .. code-block:: bash + brew install hatch You can run the docs locally by running the following: .. code-block:: bash + hatch run docs:serve diff --git a/docs/guides/compiled-sql.rst b/docs/guides/cosmos_devex/compiled-sql.rst similarity index 100% rename from docs/guides/compiled-sql.rst rename to docs/guides/cosmos_devex/compiled-sql.rst diff --git a/docs/guides/lineage.rst b/docs/guides/cosmos_devex/lineage.rst similarity index 100% rename from docs/guides/lineage.rst rename to docs/guides/cosmos_devex/lineage.rst diff --git a/docs/guides/logging.rst b/docs/guides/cosmos_devex/logging.rst similarity index 100% rename from docs/guides/logging.rst rename to docs/guides/cosmos_devex/logging.rst diff --git a/docs/guides/dbt-docs/generating-docs.rst b/docs/guides/dbt_docs/generating-docs.rst similarity index 100% rename from docs/guides/dbt-docs/generating-docs.rst rename to docs/guides/dbt_docs/generating-docs.rst diff --git a/docs/guides/dbt-docs/hosting-docs.rst b/docs/guides/dbt_docs/hosting-docs.rst similarity index 100% rename from docs/guides/dbt-docs/hosting-docs.rst rename to docs/guides/dbt_docs/hosting-docs.rst diff --git a/docs/guides/dbt-fusion.rst b/docs/guides/dbt_setup/dbt-fusion.rst similarity index 100% rename from docs/guides/dbt-fusion.rst rename to docs/guides/dbt_setup/dbt-fusion.rst diff --git a/docs/guides/multi-project.rst b/docs/guides/multi_project/multi-project.rst similarity index 100% rename from docs/guides/multi-project.rst rename to docs/guides/multi_project/multi-project.rst diff --git a/docs/guides/caching.rst b/docs/guides/optimize_performance/caching.rst similarity index 100% rename from docs/guides/caching.rst rename to docs/guides/optimize_performance/caching.rst diff --git a/docs/guides/memory_optimization.rst b/docs/guides/optimize_performance/memory_optimization.rst similarity index 100% rename from docs/guides/memory_optimization.rst rename to docs/guides/optimize_performance/memory_optimization.rst diff --git a/docs/guides/partial-parsing.rst b/docs/guides/optimize_performance/partial-parsing.rst similarity index 100% rename from docs/guides/partial-parsing.rst rename to docs/guides/optimize_performance/partial-parsing.rst diff --git a/docs/guides/selecting-excluding.rst b/docs/guides/optimize_performance/selecting-excluding.rst similarity index 100% rename from docs/guides/selecting-excluding.rst rename to docs/guides/optimize_performance/selecting-excluding.rst diff --git a/docs/guides/run-dbt/airflow-worker/async-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/async-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/async-execution-mode.rst rename to docs/guides/run_dbt/airflow-worker/async-execution-mode.rst diff --git a/docs/guides/execution-modes-local-conflicts.rst b/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst similarity index 100% rename from docs/guides/execution-modes-local-conflicts.rst rename to docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst diff --git a/docs/guides/run-dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/index.rst rename to docs/guides/run_dbt/airflow-worker/index.rst diff --git a/docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/airflow-worker/watcher-execution-mode.rst rename to docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst diff --git a/docs/guides/callbacks/callbacks.rst b/docs/guides/run_dbt/callbacks/callbacks.rst similarity index 100% rename from docs/guides/callbacks/callbacks.rst rename to docs/guides/run_dbt/callbacks/callbacks.rst diff --git a/docs/guides/run-dbt/container/aws-container-run-job.rst b/docs/guides/run_dbt/container/aws-container-run-job.rst similarity index 100% rename from docs/guides/run-dbt/container/aws-container-run-job.rst rename to docs/guides/run_dbt/container/aws-container-run-job.rst diff --git a/docs/guides/run-dbt/container/azure-container-instance.rst b/docs/guides/run_dbt/container/azure-container-instance.rst similarity index 100% rename from docs/guides/run-dbt/container/azure-container-instance.rst rename to docs/guides/run_dbt/container/azure-container-instance.rst diff --git a/docs/guides/run-dbt/container/docker.rst b/docs/guides/run_dbt/container/docker.rst similarity index 100% rename from docs/guides/run-dbt/container/docker.rst rename to docs/guides/run_dbt/container/docker.rst diff --git a/docs/guides/run-dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst similarity index 100% rename from docs/guides/run-dbt/container/gcp-cloud-run-job.rst rename to docs/guides/run_dbt/container/gcp-cloud-run-job.rst diff --git a/docs/guides/run-dbt/container/index.rst b/docs/guides/run_dbt/container/index.rst similarity index 100% rename from docs/guides/run-dbt/container/index.rst rename to docs/guides/run_dbt/container/index.rst diff --git a/docs/guides/run-dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst similarity index 100% rename from docs/guides/run-dbt/container/kubernetes.rst rename to docs/guides/run_dbt/container/kubernetes.rst diff --git a/docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst similarity index 100% rename from docs/guides/run-dbt/container/watcher-kubernetes-execution-mode.rst rename to docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst diff --git a/docs/guides/run_dbt/customization/index.rst b/docs/guides/run_dbt/customization/index.rst new file mode 100644 index 0000000000..44021154dc --- /dev/null +++ b/docs/guides/run_dbt/customization/index.rst @@ -0,0 +1,9 @@ +Additional Customization +======================== + +.. toctree:: + :maxdepth: 1 + :caption: Additional Customization + + operator-args + scheduling diff --git a/docs/guides/operator-args.rst b/docs/guides/run_dbt/customization/operator-args.rst similarity index 100% rename from docs/guides/operator-args.rst rename to docs/guides/run_dbt/customization/operator-args.rst diff --git a/docs/guides/scheduling.rst b/docs/guides/run_dbt/customization/scheduling.rst similarity index 100% rename from docs/guides/scheduling.rst rename to docs/guides/run_dbt/customization/scheduling.rst diff --git a/docs/guides/task-display-name.rst b/docs/guides/run_dbt/customization/task-display-name.rst similarity index 100% rename from docs/guides/task-display-name.rst rename to docs/guides/run_dbt/customization/task-display-name.rst diff --git a/docs/guides/run-dbt/execution-modes.rst b/docs/guides/run_dbt/index.rst similarity index 99% rename from docs/guides/run-dbt/execution-modes.rst rename to docs/guides/run_dbt/index.rst index a9bd3f1e2b..1b73d56a88 100644 --- a/docs/guides/run-dbt/execution-modes.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,5 +1,8 @@ .. _execution-modes: + + + Execution Modes =============== diff --git a/docs/getting_started/operators.rst b/docs/guides/run_dbt/operators/operators.rst similarity index 100% rename from docs/getting_started/operators.rst rename to docs/guides/run_dbt/operators/operators.rst diff --git a/docs/guides/configure-tests/testing-behavior.rst b/docs/guides/translate_dbt_to_airflow/configure-tests/testing-behavior.rst similarity index 100% rename from docs/guides/configure-tests/testing-behavior.rst rename to docs/guides/translate_dbt_to_airflow/configure-tests/testing-behavior.rst diff --git a/docs/getting_started/custom-airflow-properties.rst b/docs/guides/translate_dbt_to_airflow/custom-airflow-properties.rst similarity index 100% rename from docs/getting_started/custom-airflow-properties.rst rename to docs/guides/translate_dbt_to_airflow/custom-airflow-properties.rst diff --git a/docs/guides/dag-customization.rst b/docs/guides/translate_dbt_to_airflow/dag-customization.rst similarity index 100% rename from docs/guides/dag-customization.rst rename to docs/guides/translate_dbt_to_airflow/dag-customization.rst diff --git a/docs/guides/parsing-methods.rst b/docs/guides/translate_dbt_to_airflow/parsing-methods.rst similarity index 100% rename from docs/guides/parsing-methods.rst rename to docs/guides/translate_dbt_to_airflow/parsing-methods.rst diff --git a/docs/guides/render-config.rst b/docs/guides/translate_dbt_to_airflow/render-config.rst similarity index 100% rename from docs/guides/render-config.rst rename to docs/guides/translate_dbt_to_airflow/render-config.rst diff --git a/docs/guides/source-nodes-rendering.rst b/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst similarity index 100% rename from docs/guides/source-nodes-rendering.rst rename to docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst diff --git a/docs/index.rst b/docs/index.rst index 8926b5c198..fc215be0b7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,7 +2,7 @@ .. toctree:: :hidden: - :maxdepth: 2 + :maxdepth: 0 :caption: Contents: Home From d8663b69412a11c63d669b20f2d7bd5ac929b256 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 19:28:24 -0500 Subject: [PATCH 34/48] improve build errors --- docs/getting_started/index.rst | 19 +++------ docs/guides/cosmos_devex/index.rst | 14 +++++++ .../task-display-name.rst | 0 docs/guides/index.rst | 42 +++++-------------- docs/guides/optimize_performance/index.rst | 13 ++++++ docs/guides/run_dbt/index.rst | 25 ++++++++--- .../guides/translate_dbt_to_airflow/index.rst | 26 ++++++++++++ ...des-rendering.rst => managing-sources.rst} | 6 +-- 8 files changed, 91 insertions(+), 54 deletions(-) create mode 100644 docs/guides/cosmos_devex/index.rst rename docs/guides/{run_dbt/customization => cosmos_devex}/task-display-name.rst (100%) create mode 100644 docs/guides/optimize_performance/index.rst create mode 100644 docs/guides/translate_dbt_to_airflow/index.rst rename docs/guides/translate_dbt_to_airflow/{source-nodes-rendering.rst => managing-sources.rst} (97%) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 20c689e0b9..526526f595 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -18,15 +18,6 @@ Google Cloud Composer (GCC) Amazon Managed Workflows for Apache Airflow (MWAA) - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Operators - - Operators - Custom Airflow Properties - Getting Started =============== @@ -50,11 +41,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with KubernetesPodOperators `__ +- `Executing dbt DAGs with Watcher Kubernetes Mode `__ +- `Executing dbt DAGs with AzureContainerInstancesOperators `__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ Concepts Overview diff --git a/docs/guides/cosmos_devex/index.rst b/docs/guides/cosmos_devex/index.rst new file mode 100644 index 0000000000..614e4c3c17 --- /dev/null +++ b/docs/guides/cosmos_devex/index.rst @@ -0,0 +1,14 @@ +.. _cosmos_devex: + + +Cosmos DevEx +============ + +.. toctree:: + :maxdepth: 1 + :caption: Cosmos DevEx + + lineage + compiled-sql + logging + task-display-name \ No newline at end of file diff --git a/docs/guides/run_dbt/customization/task-display-name.rst b/docs/guides/cosmos_devex/task-display-name.rst similarity index 100% rename from docs/guides/run_dbt/customization/task-display-name.rst rename to docs/guides/cosmos_devex/task-display-name.rst diff --git a/docs/guides/index.rst b/docs/guides/index.rst index df227dea90..6234012779 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -10,70 +10,48 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: Translating dbt into Airflow - Source Nodes Rendering - Post-rendering DAG customization + translate_dbt_to_airflow/index .. toctree:: :maxdepth: 3 :hidden: :caption: How Cosmos runs dbt - execution-modes-local-conflicts - run-dbt/execution-modes - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Configure tests - - configure-tests/testing-behavior - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Callbacks - - callbacks/callbacks + run_dbt/index .. toctree:: :maxdepth: 1 :hidden: :caption: Multi-project Setups - Multi-Project Setups + multi_project/multi-project .. toctree:: :maxdepth: 1 :hidden: - :caption: Operators + :caption: Documentation - Operator Args + dbt-docs/generating-docs + dbt-docs/hosting-docs .. toctree:: :maxdepth: 1 :hidden: - :caption: Documentation + :caption: Cosmos DevEx - dbt-docs/generating-docs - dbt-docs/hosting-docs + cosmos_devex/index .. toctree:: :maxdepth: 1 :hidden: :caption: Optimizing Performance - Memory Optimization - dbt Fusion - Selecting & Excluding - Parsing Methods - Partial Parsing - Caching - Render Config + optimize_performance/index .. toctree:: :maxdepth: 1 :hidden: - :caption: Configurations + :caption: Configuration References Project Config Profile Config diff --git a/docs/guides/optimize_performance/index.rst b/docs/guides/optimize_performance/index.rst new file mode 100644 index 0000000000..0ed84470d0 --- /dev/null +++ b/docs/guides/optimize_performance/index.rst @@ -0,0 +1,13 @@ +.. _optimize-performance: + +Optimize your Cosmos Performance +================================ + +.. toctree:: + :maxdepth: 1 + :caption: Optimize Performance + + partial-parsing + memory_optimization + selecting-excluding + caching diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index 1b73d56a88..2827fe39dd 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,10 +1,7 @@ .. _execution-modes: - - - -Execution Modes -=============== +How Cosmos runs dbt +=================== .. toctree:: :maxdepth: 3 @@ -18,6 +15,24 @@ Execution Modes container/index +.. toctree:: + :maxdepth: 3 + :caption: Callbacks + + callbacks/callbacks + +.. toctree:: + :maxdepth: 3 + :caption: Operators + + operators/operators + +.. toctree:: + :maxdepth: 3 + :caption: Customize Airflow + + customization/index + Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst new file mode 100644 index 0000000000..d0f8cdbefc --- /dev/null +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -0,0 +1,26 @@ +.. _translate-dbt-to-airflow + +Translate dbt code into Airflow +=============================== + +.. toctree:: + :maxdepth: 1 + :caption: Mapping dbt into dags + + parsing-methods + custom-airflow-properties + + +.. toctree:: + :maxdepth: 1 + :caption: Configure tests + + configure-tests/testing-behavior + +.. toctree:: + :maxdepth: 1 + :caption: Translate nodes + + source-nodes-rendering + render-config + dag-customization \ No newline at end of file diff --git a/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst b/docs/guides/translate_dbt_to_airflow/managing-sources.rst similarity index 97% rename from docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst rename to docs/guides/translate_dbt_to_airflow/managing-sources.rst index 9bfcf0e97b..0d7a9d9644 100644 --- a/docs/guides/translate_dbt_to_airflow/source-nodes-rendering.rst +++ b/docs/guides/translate_dbt_to_airflow/managing-sources.rst @@ -1,7 +1,7 @@ -.. _source-nodes-rendering: +.. _managing-sources: -Source Nodes Rendering -====================== +Managing Sources +================ .. note:: This feature is only available for dbt-core >= 1.5 and cosmos >= 1.6.0. From 554a0dc59256f0b536ace69320d56e7165260c3c Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 19:36:24 -0500 Subject: [PATCH 35/48] fix index --- docs/guides/index.rst | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 6234012779..50f3059402 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -5,6 +5,13 @@ Guides Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Set up dbt with Airflow + + dbt_setup/dbt-fusion + .. toctree:: :maxdepth: 1 :hidden: @@ -31,8 +38,8 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: Documentation - dbt-docs/generating-docs - dbt-docs/hosting-docs + dbt_docs/generating-docs + dbt_docs/hosting-docs .. toctree:: :maxdepth: 1 @@ -56,16 +63,4 @@ Cosmos offers a number of configuration options to customize its behavior. For m Project Config Profile Config Execution Config - - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Customizing Airflow - - Configuring in Airflow - Configuring Lineage - Scheduling - Compiled SQL - Logging - Task display name \ No newline at end of file + Cosmos Config From d86e8e98db6acfc82f2f41dfd20a4a979d8776ff Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 20:04:34 -0500 Subject: [PATCH 36/48] fix rel-link build errors --- docs/guides/multi_project/multi-project.rst | 4 ++-- docs/guides/optimize_performance/caching.rst | 4 ++-- .../airflow-worker/watcher-execution-mode.rst | 4 ++-- docs/guides/run_dbt/callbacks/callbacks.rst | 4 ++-- docs/guides/run_dbt/container/kubernetes.rst | 2 +- .../container/watcher-kubernetes-execution-mode.rst | 2 +- docs/guides/run_dbt/customization/scheduling.rst | 2 +- docs/guides/run_dbt/index.rst | 8 ++++---- docs/guides/run_dbt/operators/operators.rst | 4 ++-- .../configure-tests/testing-behavior.rst | 12 ++++++------ .../custom-airflow-properties.rst | 2 +- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- .../translate_dbt_to_airflow/managing-sources.rst | 2 +- .../translate_dbt_to_airflow/parsing-methods.rst | 8 ++++---- .../translate_dbt_to_airflow/render-config.rst | 2 +- 15 files changed, 31 insertions(+), 31 deletions(-) diff --git a/docs/guides/multi_project/multi-project.rst b/docs/guides/multi_project/multi-project.rst index 70283bc410..5dfd79eea0 100644 --- a/docs/guides/multi_project/multi-project.rst +++ b/docs/guides/multi_project/multi-project.rst @@ -169,7 +169,7 @@ You can use either separate DAGs or a combined DAG with task groups. **Option 1: Combined DAG with Task Groups using dbt ls Load Mode (Recommended)** -.. literalinclude:: ../../dev/dags/cross_project_dbt_ls_dag.py +.. literalinclude:: ../../../dev/dags/cross_project_dbt_ls_dag.py :language: python :start-after: [START cross_project_dbt_ls_dag] :end-before: [END cross_project_dbt_ls_dag] @@ -178,7 +178,7 @@ You can use either separate DAGs or a combined DAG with task groups. This option uses pre-generated ``manifest.json`` files for faster DAG parsing (no ``dbt ls`` execution required). -.. literalinclude:: ../../dev/dags/cross_project_manifest_dag.py +.. literalinclude:: ../../../dev/dags/cross_project_manifest_dag.py :language: python :start-after: [START cross_project_manifest_dag] :end-before: [END cross_project_manifest_dag] diff --git a/docs/guides/optimize_performance/caching.rst b/docs/guides/optimize_performance/caching.rst index 7289d00742..5bf8a6406c 100644 --- a/docs/guides/optimize_performance/caching.rst +++ b/docs/guides/optimize_performance/caching.rst @@ -84,7 +84,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] @@ -161,7 +161,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] diff --git a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst index f33ef15900..05bb21c7f7 100644 --- a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst @@ -144,7 +144,7 @@ Example 1 — Using ``DbtDag`` with ``ExecutionMode.WATCHER`` You can enable WATCHER mode directly in your ``DbtDag`` configuration. This approach is best when your Airflow DAG is fully dedicated to a dbt project. -.. literalinclude:: ../../dev/dags/example_watcher.py +.. literalinclude:: ../../../../dev/dags/example_watcher.py :language: python :start-after: [START example_watcher] :end-before: [END example_watcher] @@ -451,7 +451,7 @@ Asynchronous sensor execution To disable asynchronous execution, set the ``deferrable`` flag to ``False`` in the ``operator_args``. -.. literalinclude:: ../../dev/dags/example_watcher.py +.. literalinclude:: ../../../../dev/dags/example_watcher.py :language: python :start-after: [START example_watcher_synchronous] :end-before: [END example_watcher_synchronous] diff --git a/docs/guides/run_dbt/callbacks/callbacks.rst b/docs/guides/run_dbt/callbacks/callbacks.rst index c754245525..4b602ece3f 100644 --- a/docs/guides/run_dbt/callbacks/callbacks.rst +++ b/docs/guides/run_dbt/callbacks/callbacks.rst @@ -34,7 +34,7 @@ Example: Using Callbacks with a Single Operator To demonstrate how to specify a callback function for uploading files from the target directory, here’s an example using a single operator in an Airflow DAG: -.. literalinclude:: ../../dev/dags/example_operators.py +.. literalinclude:: ../../../../dev/dags/example_operators.py :language: python :start-after: [START single_operator_callback] :end-before: [END single_operator_callback] @@ -46,7 +46,7 @@ You can leverage the :ref:`remote_target_path` configuration to upload files from the target directory to a remote storage. Below is an example of how to define a callback helper function in your ``DbtDag`` that utilizes this configuration: -.. literalinclude:: ../../dev/dags/cosmos_callback_dag.py +.. literalinclude:: ../../../../dev/dags/cosmos_callback_dag.py :language: python :start-after: [START cosmos_callback_example] :end-before: [END cosmos_callback_example] diff --git a/docs/guides/run_dbt/container/kubernetes.rst b/docs/guides/run_dbt/container/kubernetes.rst index 4ea8ccd4b9..d200589429 100644 --- a/docs/guides/run_dbt/container/kubernetes.rst +++ b/docs/guides/run_dbt/container/kubernetes.rst @@ -28,7 +28,7 @@ Additional KubernetesPodOperator parameters can be added to the ``operator_args` For instance, -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py +.. literalinclude:: ../../../../dev/dags/jaffle_shop_kubernetes.py :language: python :start-after: [START kubernetes_tg_example] :end-before: [END kubernetes_tg_example] diff --git a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst index 16dbbffd0a..d3f8a80a49 100644 --- a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst +++ b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst @@ -183,7 +183,7 @@ Example DAG Below is a complete example of a DAG using ``ExecutionMode.WATCHER_KUBERNETES``: -.. literalinclude:: ../../dev/dags/jaffle_shop_watcher_kubernetes.py +.. literalinclude:: ../../../../dev/dags/jaffle_shop_watcher_kubernetes.py :language: python ------------------------------------------------------------------------------- diff --git a/docs/guides/run_dbt/customization/scheduling.rst b/docs/guides/run_dbt/customization/scheduling.rst index 2d4e729c5b..0040135d37 100644 --- a/docs/guides/run_dbt/customization/scheduling.rst +++ b/docs/guides/run_dbt/customization/scheduling.rst @@ -77,7 +77,7 @@ This example DAG: .. The following renders in Sphinx but not Github: -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py +.. literalinclude:: ../../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] :end-before: [END local_example] diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index 2827fe39dd..a8c96d1b93 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -127,7 +127,7 @@ When using the ``local`` execution mode, Cosmos converts Airflow Connections int Example of how to use, for instance, when ``dbt`` was installed together with Cosmos: -.. literalinclude:: ../../dev/dags/basic_cosmos_dag.py +.. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] :end-before: [END local_example] @@ -153,7 +153,7 @@ Some drawbacks of this approach: Example of how to use: -.. literalinclude:: ../../dev/dags/example_virtualenv.py +.. literalinclude:: ../../../dev/dags/example_virtualenv.py :language: python :start-after: [START virtualenv_example] :end-before: [END virtualenv_example] @@ -201,7 +201,7 @@ Check the step-by-step guide on using the ``kubernetes`` execution mode at :ref: Example DAG: -.. literalinclude:: ../../dev/dags/jaffle_shop_kubernetes.py +.. literalinclude:: ../../../dev/dags/jaffle_shop_kubernetes.py :language: python :start-after: [START kubernetes_seed_example] :end-before: [END kubernetes_seed_example] @@ -345,7 +345,7 @@ as more dbt nodes will be run in parallel since they won't be blocking Airflow's Example DAG: -.. literalinclude:: ../../dev/dags/simple_dag_async.py +.. literalinclude:: ../../../dev/dags/simple_dag_async.py :language: python :start-after: [START airflow_async_execution_mode_example] :end-before: [END airflow_async_execution_mode_example] diff --git a/docs/guides/run_dbt/operators/operators.rst b/docs/guides/run_dbt/operators/operators.rst index 9f6658b6b1..448e037e77 100644 --- a/docs/guides/run_dbt/operators/operators.rst +++ b/docs/guides/run_dbt/operators/operators.rst @@ -18,7 +18,7 @@ The ``DbtCloneLocalOperator`` implement `dbt clone Date: Sun, 1 Mar 2026 20:32:36 -0500 Subject: [PATCH 37/48] fix typo --- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst index faaf31b950..35adec52e4 100644 --- a/docs/guides/translate_dbt_to_airflow/index.rst +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -1,4 +1,4 @@ -.. _translate-dbt-to-airflow +.. _translate-dbt-to-airflow: Translate dbt code into Airflow =============================== From 898a5c99094344be407afd5203df9f9d66c01cfe Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 20:59:08 -0500 Subject: [PATCH 38/48] reformat ref structure --- docs/guides/index.rst | 13 +++----- docs/guides/run_dbt/index.rst | 33 +------------------ docs/index.rst | 1 + .../optimize_performance/caching.rst | 4 +-- .../optimize_performance/index.rst | 0 .../memory_optimization.rst | 0 .../optimize_performance/partial-parsing.rst | 0 .../selecting-excluding.rst | 0 8 files changed, 9 insertions(+), 42 deletions(-) rename docs/{guides => }/optimize_performance/caching.rst (98%) rename docs/{guides => }/optimize_performance/index.rst (100%) rename docs/{guides => }/optimize_performance/memory_optimization.rst (100%) rename docs/{guides => }/optimize_performance/partial-parsing.rst (100%) rename docs/{guides => }/optimize_performance/selecting-excluding.rst (100%) diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 50f3059402..140a9c4554 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -24,7 +24,11 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: How Cosmos runs dbt - run_dbt/index + run_dbt/airflow-worker/index + run_dbt/container/index + run_dbt/callbacks/callbacks + run_dbt/operators/operators + run_dbt/customization/index .. toctree:: :maxdepth: 1 @@ -48,13 +52,6 @@ Cosmos offers a number of configuration options to customize its behavior. For m cosmos_devex/index -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Optimizing Performance - - optimize_performance/index - .. toctree:: :maxdepth: 1 :hidden: diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/index.rst index a8c96d1b93..71d581c25b 100644 --- a/docs/guides/run_dbt/index.rst +++ b/docs/guides/run_dbt/index.rst @@ -1,39 +1,8 @@ .. _execution-modes: -How Cosmos runs dbt +Execution Modes =================== -.. toctree:: - :maxdepth: 3 - :caption: Run dbt in the Airflow worker - - airflow-worker/index - -.. toctree:: - :maxdepth: 3 - :caption: Run dbt in a container - - container/index - -.. toctree:: - :maxdepth: 3 - :caption: Callbacks - - callbacks/callbacks - -.. toctree:: - :maxdepth: 3 - :caption: Operators - - operators/operators - -.. toctree:: - :maxdepth: 3 - :caption: Customize Airflow - - customization/index - - Cosmos can run ``dbt`` commands using several different approaches, called ``execution modes``: 1. **local**: Run ``dbt`` commands using a local ``dbt`` installation (default) diff --git a/docs/index.rst b/docs/index.rst index fc215be0b7..8af15370e0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -8,6 +8,7 @@ Home Getting Started Guides + Optimize Performance Profiles Contributing Airflow 3 compatibility diff --git a/docs/guides/optimize_performance/caching.rst b/docs/optimize_performance/caching.rst similarity index 98% rename from docs/guides/optimize_performance/caching.rst rename to docs/optimize_performance/caching.rst index 5bf8a6406c..7289d00742 100644 --- a/docs/guides/optimize_performance/caching.rst +++ b/docs/optimize_performance/caching.rst @@ -84,7 +84,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] @@ -161,7 +161,7 @@ The method deletes the Cosmos cache stored in Airflow Variables based on the las As an example, the following clean-up DAG will delete any cache associated with Cosmos that has not been used for the last five days: -.. literalinclude:: ../../../dev/dags/example_cosmos_cleanup_dag.py +.. literalinclude:: ../../dev/dags/example_cosmos_cleanup_dag.py :language: python :start-after: [START cache_example] :end-before: [END cache_example] diff --git a/docs/guides/optimize_performance/index.rst b/docs/optimize_performance/index.rst similarity index 100% rename from docs/guides/optimize_performance/index.rst rename to docs/optimize_performance/index.rst diff --git a/docs/guides/optimize_performance/memory_optimization.rst b/docs/optimize_performance/memory_optimization.rst similarity index 100% rename from docs/guides/optimize_performance/memory_optimization.rst rename to docs/optimize_performance/memory_optimization.rst diff --git a/docs/guides/optimize_performance/partial-parsing.rst b/docs/optimize_performance/partial-parsing.rst similarity index 100% rename from docs/guides/optimize_performance/partial-parsing.rst rename to docs/optimize_performance/partial-parsing.rst diff --git a/docs/guides/optimize_performance/selecting-excluding.rst b/docs/optimize_performance/selecting-excluding.rst similarity index 100% rename from docs/guides/optimize_performance/selecting-excluding.rst rename to docs/optimize_performance/selecting-excluding.rst From 6b75ba3efb94b3b4732c28c16b7663e32c9d7fef Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Sun, 1 Mar 2026 21:13:07 -0500 Subject: [PATCH 39/48] restructure --- docs/guides/index.rst | 10 +--------- .../run_dbt/{index.rst => execution-modes.rst} | 0 docs/index.rst | 1 + docs/{guides => reference/configs}/cosmos-conf.rst | 0 .../configs}/execution-config.rst | 0 docs/reference/configs/index.rst | 13 +++++++++++++ .../configs}/profile-config.rst | 0 .../configs}/project-config.rst | 0 docs/reference/index.rst | 9 +++++++++ 9 files changed, 24 insertions(+), 9 deletions(-) rename docs/guides/run_dbt/{index.rst => execution-modes.rst} (100%) rename docs/{guides => reference/configs}/cosmos-conf.rst (100%) rename docs/{guides => reference/configs}/execution-config.rst (100%) create mode 100644 docs/reference/configs/index.rst rename docs/{guides => reference/configs}/profile-config.rst (100%) rename docs/{guides => reference/configs}/project-config.rst (100%) create mode 100644 docs/reference/index.rst diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 140a9c4554..03970c5959 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -24,6 +24,7 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: How Cosmos runs dbt + run_dbt/execution-modes run_dbt/airflow-worker/index run_dbt/container/index run_dbt/callbacks/callbacks @@ -52,12 +53,3 @@ Cosmos offers a number of configuration options to customize its behavior. For m cosmos_devex/index -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Configuration References - - Project Config - Profile Config - Execution Config - Cosmos Config diff --git a/docs/guides/run_dbt/index.rst b/docs/guides/run_dbt/execution-modes.rst similarity index 100% rename from docs/guides/run_dbt/index.rst rename to docs/guides/run_dbt/execution-modes.rst diff --git a/docs/index.rst b/docs/index.rst index 8af15370e0..fc5728dc78 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -9,6 +9,7 @@ Getting Started Guides Optimize Performance + Reference Profiles Contributing Airflow 3 compatibility diff --git a/docs/guides/cosmos-conf.rst b/docs/reference/configs/cosmos-conf.rst similarity index 100% rename from docs/guides/cosmos-conf.rst rename to docs/reference/configs/cosmos-conf.rst diff --git a/docs/guides/execution-config.rst b/docs/reference/configs/execution-config.rst similarity index 100% rename from docs/guides/execution-config.rst rename to docs/reference/configs/execution-config.rst diff --git a/docs/reference/configs/index.rst b/docs/reference/configs/index.rst new file mode 100644 index 0000000000..d83b28614a --- /dev/null +++ b/docs/reference/configs/index.rst @@ -0,0 +1,13 @@ + +# Configurations - Uncomment this section to turn the page into a dropdown navigation +# ============== + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + + cosmos-conf + execution-config + profile-config + project-config \ No newline at end of file diff --git a/docs/guides/profile-config.rst b/docs/reference/configs/profile-config.rst similarity index 100% rename from docs/guides/profile-config.rst rename to docs/reference/configs/profile-config.rst diff --git a/docs/guides/project-config.rst b/docs/reference/configs/project-config.rst similarity index 100% rename from docs/guides/project-config.rst rename to docs/reference/configs/project-config.rst diff --git a/docs/reference/index.rst b/docs/reference/index.rst new file mode 100644 index 0000000000..39b5e65f62 --- /dev/null +++ b/docs/reference/index.rst @@ -0,0 +1,9 @@ +Reference +========= + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Configurations + + configs/index \ No newline at end of file From 548df84a5cd585bc4eab9270b7b6ebd439c60cf9 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 2 Mar 2026 03:11:28 +0000 Subject: [PATCH 40/48] =?UTF-8?q?=F0=9F=8E=A8=20[pre-commit.ci]=20Auto=20f?= =?UTF-8?q?ormat=20from=20pre-commit.com=20hooks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/guides/cosmos_devex/index.rst | 2 +- docs/guides/index.rst | 1 - docs/guides/run_dbt/airflow-worker/index.rst | 2 +- docs/guides/run_dbt/container/index.rst | 2 +- docs/guides/translate_dbt_to_airflow/index.rst | 2 +- docs/reference/configs/index.rst | 2 +- docs/reference/index.rst | 2 +- 7 files changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/guides/cosmos_devex/index.rst b/docs/guides/cosmos_devex/index.rst index 614e4c3c17..2ad3dff71b 100644 --- a/docs/guides/cosmos_devex/index.rst +++ b/docs/guides/cosmos_devex/index.rst @@ -11,4 +11,4 @@ Cosmos DevEx lineage compiled-sql logging - task-display-name \ No newline at end of file + task-display-name diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 03970c5959..9b7e233814 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -52,4 +52,3 @@ Cosmos offers a number of configuration options to customize its behavior. For m :caption: Cosmos DevEx cosmos_devex/index - diff --git a/docs/guides/run_dbt/airflow-worker/index.rst b/docs/guides/run_dbt/airflow-worker/index.rst index 00cb281bc8..eaa89c2d9f 100644 --- a/docs/guides/run_dbt/airflow-worker/index.rst +++ b/docs/guides/run_dbt/airflow-worker/index.rst @@ -6,4 +6,4 @@ Run dbt in an Airflow worker :caption: Run dbt in an Airflow worker async-execution-mode - watcher-execution-mode \ No newline at end of file + watcher-execution-mode diff --git a/docs/guides/run_dbt/container/index.rst b/docs/guides/run_dbt/container/index.rst index 634e9e8eb4..9cccdbb29a 100644 --- a/docs/guides/run_dbt/container/index.rst +++ b/docs/guides/run_dbt/container/index.rst @@ -10,4 +10,4 @@ Run dbt in a container docker gcp-cloud-run-job kubernetes - watcher-kubernetes-execution-mode \ No newline at end of file + watcher-kubernetes-execution-mode diff --git a/docs/guides/translate_dbt_to_airflow/index.rst b/docs/guides/translate_dbt_to_airflow/index.rst index 35adec52e4..5ff278003e 100644 --- a/docs/guides/translate_dbt_to_airflow/index.rst +++ b/docs/guides/translate_dbt_to_airflow/index.rst @@ -23,4 +23,4 @@ Translate dbt code into Airflow managing-sources render-config - dag-customization \ No newline at end of file + dag-customization diff --git a/docs/reference/configs/index.rst b/docs/reference/configs/index.rst index d83b28614a..741b7a76bd 100644 --- a/docs/reference/configs/index.rst +++ b/docs/reference/configs/index.rst @@ -10,4 +10,4 @@ cosmos-conf execution-config profile-config - project-config \ No newline at end of file + project-config diff --git a/docs/reference/index.rst b/docs/reference/index.rst index 39b5e65f62..4c430eb4d8 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -6,4 +6,4 @@ Reference :hidden: :caption: Configurations - configs/index \ No newline at end of file + configs/index From c638c18a16d886069ab3b923f99453ebc89c27fd Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 17:30:13 -0500 Subject: [PATCH 41/48] fix build errors --- docs/reference/configs/cosmos-conf.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/configs/cosmos-conf.rst b/docs/reference/configs/cosmos-conf.rst index cc68c3b71f..a8928c3840 100644 --- a/docs/reference/configs/cosmos-conf.rst +++ b/docs/reference/configs/cosmos-conf.rst @@ -253,14 +253,14 @@ This page lists all available Airflow configurations that affect ``astronomer-co As an example, when this option is enabled, the following is an example of specifying the imports with full module paths: - .. literalinclude:: ../../dev/dags/basic_cosmos_dag_full_module_path_imports.py + .. literalinclude:: ../../../dev/dags/basic_cosmos_dag_full_module_path_imports.py :language: python :start-after: [START cosmos_explicit_imports] :end-before: [END cosmos_explicit_imports] as opposed to the following approach you might have when this option is disabled (default): - .. literalinclude:: ../../dev/dags/basic_cosmos_dag.py + .. literalinclude:: ../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START cosmos_init_imports] :end-before: [END cosmos_init_imports] From 426808df357ce795c8360ceaa266475baee8a5a7 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 17:30:43 -0500 Subject: [PATCH 42/48] add optimize, execution modes redirects --- docs/conf.py | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/conf.py b/docs/conf.py index 1f20af6190..f2e43592b6 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,3 +53,26 @@ } generate_mapping_docs() + +# -- Begin docs redirect section +# -- To test redirects in a local build, paste the redirect source, and append .html to the end. +# -- For example, "airflow3_compatibility/index" redirect must be tested using "airflow3_compatibility/index.html" +# -- https://documatt.com/sphinx-reredirects/usage/ +redirects = { + "configuration/caching": "../optimize_performance/caching.html", + "configuration/memory_optimization": "../optimize_performance/memory_optimization.html", + "configuration/partial-parsing": "../optimize_performance/partial-parsing.html", + "configuration/selecting-excluding": "../optimize_performance/selecting-excluding.html", + "getting_started/async-execution-mode": "../guides/run_dbt/airflow-worker/async-execution-mode.html", + "getting_started/aws-container-run-job": "../guides/run_dbt/airflow-worker/async-execution-mode.html", + "getting_started/azure-container-instance": "../guides/run_dbt/container/azure-container-instance.html", + "getting_started/custom-airflow-properties": "../run_dbt/airflow-worker/custom-airflow-properties.html", + "getting_started/docker": "../guides/run_dbt/container/docker.html", + "getting_started/execution-modes-local-conflicts": "../guides/run_dbt/airflow-worker/execution-modes-local-conflicts.html", + "getting_started/execution-modes": "../guides/run_dbt/execution-modes.html", + "getting_started/gcp-cloud-run-job": "../guides/run_dbt/container/gcp-cloud-run-job.html", + "getting_started/kubernetes": "../guides/run_dbt/container/kubernetes.html", + "getting_started/operators": "../guides/run_dbt/operators/operators.html", + "getting_started/watcher-execution-mode": "../guides/run_dbt/airflow-worker/watcher-execution-mode.html", + "getting_started/watcher-kubernetes-execution-mode": "../guides/run_dbt/container/watcher-kubernetes-execution-mode.html", +} \ No newline at end of file From 3a5957b29e3d614751b75e42ef4ad25bdec62496 Mon Sep 17 00:00:00 2001 From: Laura Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Tue, 3 Mar 2026 19:57:20 -0500 Subject: [PATCH 43/48] Apply suggestions from code review Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com> --- docs/getting_started/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 526526f595..4bb7b9c4a1 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -41,7 +41,7 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with Docker Operators `__ +- `Executing dbt DAGs with DockerOperators `__ - `Executing dbt DAGs with KubernetesPodOperators `__ - `Executing dbt DAGs with Watcher Kubernetes Mode `__ - `Executing dbt DAGs with AzureContainerInstancesOperators `__ From 3837e646a3ca0a6499c176d373463a757a536a34 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 4 Mar 2026 15:07:51 +0000 Subject: [PATCH 44/48] =?UTF-8?q?=F0=9F=8E=A8=20[pre-commit.ci]=20Auto=20f?= =?UTF-8?q?ormat=20from=20pre-commit.com=20hooks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index f2e43592b6..b13a9a5eb6 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -75,4 +75,4 @@ "getting_started/operators": "../guides/run_dbt/operators/operators.html", "getting_started/watcher-execution-mode": "../guides/run_dbt/airflow-worker/watcher-execution-mode.html", "getting_started/watcher-kubernetes-execution-mode": "../guides/run_dbt/container/watcher-kubernetes-execution-mode.html", -} \ No newline at end of file +} From 2e48b9730144eedfe587db6ee79156de66ff87b8 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Wed, 4 Mar 2026 10:18:00 -0500 Subject: [PATCH 45/48] Link and directory fixes --- docs/getting_started/index.rst | 18 ++++++++++++------ docs/index.rst | 5 ++--- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 4bb7b9c4a1..59e07698a5 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -5,9 +5,15 @@ :hidden: :caption: Cosmos Fundamentals - Astro CLI quickstart Similar dbt and Airflow concepts +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Quickstart + + Astro CLI quickstart + .. toctree:: :maxdepth: 1 :hidden: @@ -41,11 +47,11 @@ For more customization, check out the different execution modes that Cosmos supp For specific guides, see the following: -- `Executing dbt DAGs with DockerOperators `__ -- `Executing dbt DAGs with KubernetesPodOperators `__ -- `Executing dbt DAGs with Watcher Kubernetes Mode `__ -- `Executing dbt DAGs with AzureContainerInstancesOperators `__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators `__ +- `Executing dbt DAGs with DockerOperators <../../guides/run_dbt/container/docker.html>`__ +- `Executing dbt DAGs with KubernetesPodOperators <../../guides/run_dbt/container/kubernetes.html>`__ +- `Executing dbt DAGs with Watcher Kubernetes Mode <../../guides/run_dbt/container/watcher-kubernetes-execution-mode.html>`__ +- `Executing dbt DAGs with AzureContainerInstancesOperators <../../guides/run_dbt/container/azure-container-instance.html>`__ +- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators <../../guides/run_dbt/container/gcp-cloud-run-job.html>`__ Concepts Overview diff --git a/docs/index.rst b/docs/index.rst index fc5728dc78..97c858961a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -9,8 +9,8 @@ Getting Started Guides Optimize Performance - Reference Profiles + Reference Contributing Airflow 3 compatibility Compatibility Policy @@ -112,8 +112,7 @@ for managing and scaling your data workflows. Getting Started with Airflow Async Execution Mode ------------------------------------------------- -See our :doc:`Getting Started with Airflow Async Execution Mode ` for details. - +See our :doc:`Getting Started with Airflow Async Execution Mode ` for details. Airflow 3 compatibility ----------------------- From ce1a87b3d6c5017619267e44985af2322663bf91 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Wed, 4 Mar 2026 13:46:31 -0500 Subject: [PATCH 46/48] Add troubleshooting section --- docs/conf.py | 2 +- .../airflow-worker/local-execution-mode.rst | 133 +----------------- docs/reference/configs/index.rst | 4 +- docs/reference/index.rst | 7 + .../execution-modes-local-conflicts.rst | 10 +- docs/reference/troubleshooting/index.rst | 9 ++ 6 files changed, 24 insertions(+), 141 deletions(-) rename docs/{guides/run_dbt/airflow-worker => reference/troubleshooting}/execution-modes-local-conflicts.rst (95%) create mode 100644 docs/reference/troubleshooting/index.rst diff --git a/docs/conf.py b/docs/conf.py index b13a9a5eb6..d1bb09ffed 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -68,7 +68,7 @@ "getting_started/azure-container-instance": "../guides/run_dbt/container/azure-container-instance.html", "getting_started/custom-airflow-properties": "../run_dbt/airflow-worker/custom-airflow-properties.html", "getting_started/docker": "../guides/run_dbt/container/docker.html", - "getting_started/execution-modes-local-conflicts": "../guides/run_dbt/airflow-worker/execution-modes-local-conflicts.html", + "getting_started/execution-modes-local-conflicts": "../reference/troubleshooting/execution-modes-local-conflicts.html", "getting_started/execution-modes": "../guides/run_dbt/execution-modes.html", "getting_started/gcp-cloud-run-job": "../guides/run_dbt/container/gcp-cloud-run-job.html", "getting_started/kubernetes": "../guides/run_dbt/container/kubernetes.html", diff --git a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst index 55ab6c2b47..a5b1e71ad9 100644 --- a/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/local-execution-mode.rst @@ -24,135 +24,4 @@ Example of how to use, for instance, when ``dbt`` was installed together with Co .. literalinclude:: ../../../../dev/dags/basic_cosmos_dag.py :language: python :start-after: [START local_example] - :end-before: [END local_example] - -.. _execution-modes-local-conflicts: - -Airflow and dbt dependencies conflicts -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When using the local execution mode, you can encounter dependency conflicts between -`Apache Airflow® `_ and dbt. The conflicts might increase depending on the Airflow providers and dbt adapters you use. - -If you find errors, we recommend you isolate the installation of dbt from the Airflow installation. For your Local execution mode, install dbt in a separate -Python virtualenv and set the `ExecutionConfig.dbt_executable_path <../reference/configs/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../reference/configs/render-config.html>`_ parameters. - -Cosmos includes many other `execution mode methods `__ that support isolating dbt from Airflow. - -In the following table, ``x`` represents combinations that lead to conflicts (vanilla ``apache-airflow`` and ``dbt-core`` packages): - -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| Airflow / DBT | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | -+===============+=====+=====+=====+=====+=====+=====+=====+=====+=====+=====+======+ -| 2.2 | | | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.3 | x | x | | x | x | x | x | x | x | x | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.4 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.5 | x | x | x | | | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.6 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.7 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.8 | x | x | x | x | x | | x | | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.9 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.10 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 2.11 | x | x | x | x | x | | | | | | | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ -| 3.0 | x | x | x | x | x | x | x | x | | | x | -+---------------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+------+ - -Examples of errors ------------------- - -.. code-block:: bash - - The conflict is caused by: - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10 - apache-airflow 2.8.0 depends on pydantic>=2.3.0 - dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10 - - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.2.4 depends on jinja2<3.1 and >=2.10.1 - dbt-core 1.5.0 depends on Jinja2==3.1.2 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow==2.6.0 and dbt-core because these package versions have conflicting dependencies. - The conflict is caused by: - apache-airflow 2.6.0 depends on importlib-metadata<5.0.0 and >=1.7; python_version < "3.9" - dbt-semantic-interfaces 0.1.0.dev7 depends on importlib-metadata==6.6.0 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow, apache-airflow==2.7.0 and dbt-core==1.4.0 because these package versions have conflicting dependencies. - - The conflict is caused by: - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.12.0 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.2 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.1 depends on PyYAML<6 and >=5.1 - dbt-core 1.4.0 depends on pyyaml>=6.0 - connexion 2.11.0 depends on PyYAML<6 and >=5.1 - apache-airflow 2.7.0 depends on jsonschema>=4.18.0 - flask-appbuilder 4.3.3 depends on jsonschema<5 and >=3 - connexion 2.10.0 depends on jsonschema<4 and >=2.5.1 - -.. code-block:: bash - - ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. - - The conflict is caused by: - dbt-core 1.10.0 depends on pydantic<2 - apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 - - - -How to reproduce ----------------- - -The table was created by running `nox `__ with the following ``noxfile.py``: - -.. code-block:: python - - import nox - - nox.options.sessions = ["compatibility"] - nox.options.reuse_existing_virtualenvs = True - - - @nox.session(python=["3.10"]) - @nox.parametrize( - "dbt_version", - ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9", "1.10"], - ) - @nox.parametrize( - "airflow_version", - ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8", "2.9", "2.10", "2.11", "3.0"], - ) - def compatibility(session: nox.Session, airflow_version, dbt_version) -> None: - """Run both unit and integration tests.""" - session.run( - "pip3", - "install", - "--pre", - f"apache-airflow=={airflow_version}", - f"dbt-core=={dbt_version}", - ) \ No newline at end of file + :end-before: [END local_example] \ No newline at end of file diff --git a/docs/reference/configs/index.rst b/docs/reference/configs/index.rst index 741b7a76bd..48a94c8c6b 100644 --- a/docs/reference/configs/index.rst +++ b/docs/reference/configs/index.rst @@ -1,6 +1,6 @@ -# Configurations - Uncomment this section to turn the page into a dropdown navigation -# ============== +Configurations +============== .. toctree:: :maxdepth: 1 diff --git a/docs/reference/index.rst b/docs/reference/index.rst index 4c430eb4d8..3fe39513f7 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -7,3 +7,10 @@ Reference :caption: Configurations configs/index + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Troubleshooting + + troubleshooting/index diff --git a/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst b/docs/reference/troubleshooting/execution-modes-local-conflicts.rst similarity index 95% rename from docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst rename to docs/reference/troubleshooting/execution-modes-local-conflicts.rst index 0f9120127c..52acf85394 100644 --- a/docs/guides/run_dbt/airflow-worker/execution-modes-local-conflicts.rst +++ b/docs/reference/troubleshooting/execution-modes-local-conflicts.rst @@ -1,5 +1,3 @@ -:orphan: - .. _execution-modes-local-conflicts: Airflow and dbt dependencies conflicts @@ -92,11 +90,11 @@ Examples of errors .. code-block:: bash -ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. + ERROR: Cannot install apache-airflow and dbt-core==1.10.0 because these package versions have conflicting dependencies. -The conflict is caused by: - dbt-core 1.10.0 depends on pydantic<2 - apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 + The conflict is caused by: + dbt-core 1.10.0 depends on pydantic<2 + apache-airflow-core 3.0.0 depends on pydantic>=2.11.0 diff --git a/docs/reference/troubleshooting/index.rst b/docs/reference/troubleshooting/index.rst new file mode 100644 index 0000000000..3090a90bc2 --- /dev/null +++ b/docs/reference/troubleshooting/index.rst @@ -0,0 +1,9 @@ +Troubleshooting +=============== + +.. toctree:: + :maxdepth: 1 + :hidden: + :caption: Troubleshoot Cosmos + + execution-modes-local-conflicts \ No newline at end of file From ca2d06f817b82a61e61e00803514d05931a776f0 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Wed, 4 Mar 2026 14:34:56 -0500 Subject: [PATCH 47/48] Add choose an EM guide --- docs/_static/execution_mode_map.png | Bin 0 -> 145626 bytes docs/conf.py | 1 - docs/getting_started/execution-modes.rst | 106 +++++++++++++++++++++++ docs/getting_started/index.rst | 10 +-- docs/guides/index.rst | 1 - docs/guides/run_dbt/execution-modes.rst | 88 ------------------- 6 files changed, 107 insertions(+), 99 deletions(-) create mode 100644 docs/_static/execution_mode_map.png create mode 100644 docs/getting_started/execution-modes.rst delete mode 100644 docs/guides/run_dbt/execution-modes.rst diff --git a/docs/_static/execution_mode_map.png b/docs/_static/execution_mode_map.png new file mode 100644 index 0000000000000000000000000000000000000000..97bb1f546db137ec9bfccb4576db86dbbe90d704 GIT binary patch literal 145626 zcmeFYX;f2N+btLs<;_vS0|yZ4jx9>Bi1eVKf}(&(Cp{t{eG@`}5Q2zujtJ;cdPqb? zAoS8h8X5tWCJF|mOF$$M0*MepAR)=!@qMf6-Wv7e)~)-aete9<*fhz`+H0+6KJ%H+ zTz}i!TJ8Vs&~G3RX#W-Kiw+>r?o1G9kLoWhM@x=fiK!kJ&s4nU{rjcZopAqN76RRQdT38##95w3j!Tn zgUkH-;q1RMcZhzt`@7;U(e=|21JP7~UR;XYC%T?bpFJqLCd+o67F|J_DSp4MZesnv zu8se71JHlX0qDQx0EmYF+6Vu?oP+4Q07wre5XhS)zxkYlFFewA1Pv;w{qF)k!^WKe zK0e&4HNd(d)`NNo5z|{S>>ssBZaYBYk|Nmb+!4I$Yvr6SQK-KS@1F4_AlfF~_ZoGr z7CZ>tC&Y&l(*aO5c>fDYCG2ytjIiot&i8)2jwbN#Waa+=ZMg`U^IkMYS_>(+=_omO zbQ$#Gs4KecOk5y(Y1ZTDifVN6@@Va)i}F?wjT@8B`;1`nmJ<;+v}t42TP2B0m0sr5 zuN%~Y`_wWI2+PJC14_<6{O_A)X`8%iy6fR)dA4T`Zo7S6_0N@uXs{WDd{^n|ENk_q z*;y=;i#EMgJ;}|OqvR^Bz6l;W9r%TL-7u+VSj8Egz$lpu(OeX(;1q3Iz3Ovmo*^4@ z7BV2JcLnLgt-2fJIz!)OOdvx+CYp0K4o-6)R=^Xine*A^t#F@YY`|T7LTx@gf^1jM z7z&i6Amv>kws4vP=qs!d0|Zvyiwt9QI3FKMKHXyus|>cXAf=Xq-P5VWD3^H;u!}Tz z<6+-|NiLod22Gw+=(N3PF<)eSIGivK84{aRr=oR!Dz?>hXWBY|IUil`jJZ6Pe@4Y2J# zU8_)qzrD7Wr_1n0iigS^Ht#)imOv&gReDp}0~(60cFtoTxf{(zVKQjrxT21En-|k| zDXSi^vQGy=0Y#4hN|1czuJcmPrKu&f_e{;w%FI~kKwUh?n#6czId|6!XPPLt?l`Zd z?B)~xQY$F6FKSDd(`50LU17^e?OJf+*9hYswx+Aj)~IJGa6VNXVpL=CIMu&~`K5uZ zp8~`r;Q6zQe&dR=!@z6Z2U zPN^F@RCL_#S}m^I+7Dfb@=E8tB}7C>&LyRPJzl45N0@JpWyIe?BMMiHzcHlEb%N0W zf0Fl&7dj|~9X0`dMgAf<&+NG?=ccUA-X}SJqq4;MpZYXTp!Ll;8(`Z5Goef!PpZmf zjVtPJvo4mE(-rq=36R7^5kpG8a<%foh_Qi8?@-;@t|+x9Hco4gwp726+Ac+~!?}Ox zTh51BRx2v=7wndl3HpjHp@Xp!eiPg^#&G>nLST_D?!8CC=`8>82#zDDbNT46Fnv9w zZECEjSoSh*b#8{-GZg&I@kH5V^H%(NbH&N*4^Qp4s+cpkuIFkJxL0UGrA0{5m<*0I z48t#A=Vp!l6dLVk9BbR6#w7zys`h|3P`#@_Y@G=5^NVUptx114dHSr}V|CtHbLLDL zZ_BsGJYvhgkcMEj;(>>+$C-~?4Ai#dU}=Sc#(Dps!gRVBRV_qfv2&Kj{2p9 zRq3+C@&qg^VfJ-)b<$^G+$G-t_UQA^(#LPMpdvxvbzsQb(^D-5VJ8#ZokHR9oXEVw z-c>EcVi+|2Z%4JPaZW32wUvn9T5i|u(*!~z?1zUrsaM{rfwE?*n)E{1@>XlDVKdtt z`F7GhztB-Lx1MG?K9smnFklQB+cy%iwTk?|<|`$&7T2{(&5dm|V>ShGGH*;tz0F}A zbMO|RVs>Q2+}cDdh0W1jC27wdc$O`ep2~u#=tjdg|e+Hw#jbRC}&+lE4$xM$-U-j zr2hDX;jX>o*&%(7jEr49>Mn^TJAD|RFhFwh$@wu(zWq`PWOq!ATo`hVBxpSWs{1#z z{uk8;C7TT!(@cSSNU=&3^Pl=x(87-3l|Zu_nnsK~0EQ*bi9-6|>^I4pa^v0)Vidj*YRf)X=CH{?cG>shjI0u_soo)`8lPB4f3H_lx z^#d5bKl^hTX+BT*sHYO4Nm*63-LmJe&Fq==5!HLiL2uWqMkU(?{XqJ@*FF+A0Faj% z0ZX+qqXSi=ys~XFdgdWE;E1)NT?HPs-)}XFEo88n@zQyX=ZFv zFfGEnW+T_pxc@^IqGYm3rT=#rb^+Ud6?*Qmzjtc_k1AptjtVd{&mkEAE2{)%G&pvv zRHezRgCA%YYi8VR^!*8g2#Tz;j{+u>cYe|(acMG3@eh7h3u6N3ZRr=_oI(<-3PB@Z z8@W~iBxY$X5dV3rhTyuRv&%x1QkMOu(1@A>n-OIn!6)@K)NfPP55fos4Er^VLcMUz zX4_9DPeC>D*M_HRcVBJ`f67n4F#bM1cOM&bJw4;^R5CAEL)wXoH6#Sp>Wsy4Z+WzM z{KyP>$uWCcf2|+-W=%81jY~c zx~s!$+OwsiRq$pC(?;Fi-S2|-+{VvQqtpiisb{Vz1L_wXrpKaHaG)7ri>svM`nVv0?aIwyxaottE@vdYIj{sA>Nc1 z+J0BeXVzION&`@$sbh6Sqt?N*IR@Y62(=rR7Z(dZj(ZYjZdSV|Zj`SU@xF}jd6<14 zW!WdJ7)D$>@miBe;M}rf8{J3QbpLcBCi;z-Hi&B$jYsUmjeCx7dg-HYZ6%fQt(m6R z7T`{J^|s5h^2;l7#leh30`+#KG)O`bh^ypV%jLG99znf6|Ba1L(a$)o$&EEe!eOi< zE% z8X|T!V$s~CaTgKsTguBYM!ISmtp8m4XB=P6gR45))=j$C=ReWfyS0D?uSI*n9`}#? zeb9rD$xqJAMW|gU^!s3_g_xOIdJ;v8*w`mN;jf`ih3#UV$yt0JqWXs>E1#~R;)d1a z{M;}@WJ_N?QS_KRAFUFvq8@`PAM^tcK#GStI*2F7->V~i4Co(XiA*ZwL#sZ6>l zLG0jpIFHS(a?y5GEs?sZnttzbRKr}jDGSI9s8{Pjw$PuB=!tJ7@!B$97bL+8#;DHt z1xW^jC;rd-hQZb~q@BBWQq(5gi!rwpsP6)*Bm6?3f@xuA4{1LdU(FKC1Z!39Y8$Q; zd$1T)B&fX(9tk&y4dO%8R9S~MrmZ`3XP2ji+e<&Kz@I;+zyqN>@c|$#cYhfvJ#V2^ zN-@``TCCqCfxpjXa*MHx9e7H7sWJqm6rg+x4B#KZ!O##0j2iPM!>i8cTepg)c zcq(Uea8K!MvZZd~(DZtev&FoldM)$TwB{U4U(CFNt@!3_bBTG3zj6UTOBV$vbFH9x z2|+YNJ%gN5VXpsWUJY=!F)RP$dYwtQ@GgzoDU+>SBT#6PmNW-cSIF8VbGB1N`5Q}CmZBe`w(pP-YJ9nc`Cc(Qc*XJRu1>)(tUh&B{gFa= zd$pTDU{l$_;0Iiq8g6HIj`}f1r_!1oT~0iwd$wwvm|~vGQH~1bCyftx`JI}-mS>n) z?JbD8JGHv2|IBE5UiSHVm&vq?5eLv?`=SO|!q1%60Qdtc&hCxb~yXhqGjO$*cnZ+ zg6JVve%~lY#)>UzWz)2ra@pR6yF>M{rr(uFcfuQkoc!mX>K!AqWLnL+mFO?b`=}2W zNt&mq{2Tmjhf^H@xmyzCqN>%Kd0LP3*FCPDryxUTh_cs@tPklfCsEodZ9siqB|x;_ zpTCqQIkY0yM^X48H_yH)tGKJq94t!VQW804eWQo(MR7+roZZ8#b@l-UTrpsg3})Yy$CpaG<_6PjjL>$0 zksf%L^DN>!caardRIY#g&rG3fqaR){dXK!iLxAJ5%Ns z9q=yTe({W24QGRF@-%>qw-hA-dEeUo(E?LAy2LFEmp)|Y0DdUGJD|d1rR8Q6_#=BM**bA&N;X~~a`$WgW8nEQaeq7Q;pK(9`d3F-c|lV}qyG=NZzoKKPFL(( zfsxzgSU^rUrbK(qc$LKCLYc5@77M+*WKrsx`?TZ_{6%d|t%_d>Y1!vQ&IzWxNyX7+ z3AJ8vTAVt4(${z`G=jx5s%@~+v~aOl_{v>-uj2FZ+x@nrpwpk9ubyxmR@rT)ch75N zVY*@`Gn@5T#-7O?8rYSkbd2-`0HL><DHn-`d6(t$|)GN|!RPqbIEmTqb4L zLoA>8XkGAhyY(geL-C(=jql$>T?DfXZi5un-{fh?S>n__JKY=)eZMPC;T7bt%u?@x z$5TiyZt2T>`#s2p=D4<0xaytju5cc)Ox0fT*bcaC>A;Z(;6Ule0k+ac0m@eoAU{iP z)Yw64o^1_yn2K1Fy?{AY`aP=(!8Kq>ZUDGI6T zU3K`qsO>}jLVl?VEAGS^t{tfSM#nzhS^Wf4&5YY*s)N4fir9H)BYjVVH$64mCT&;S zZA;bH9k=O-;un!I(7;0T|L4l?tsCBHY`DYtxY=69w1;FK3CAOBvSI0h(93R&(( z!M3`TN>p2fTgs3awyl3;d@wlz^$1WCy%KV7QEOk`OJ*TJ}^hV98EV{VDBzF;;behx&GwL_D4r1P}Hw6Wx z{L(K1>@mKhOqY%wC)ANKU-fqF&;zB4dR9IUU+=eYJNf+57oRh0R%!ej6%w6YSAMB# z@^2fw&i<@~)&}N1-tO`R$koQo4xheX`fI>n=XPDoW=eo`uZvy_eEq-cbouPjFk61V z^!H6OM}jSaL|5K#vsMNeu+)G zJ|8N*y)~Co`{S!!BQ5mCj~%qLF5oP_^)VOkkGBOsW(_9v2v!-e32e!19BTGp@~&sU z9(7MBAUQW}Y*}GM1zWgq;u*;U@_|lvo`HN*Au%DnnwCU*J53pLjRrTdQ1sfG@TMefSY zgn^TLxosE=WrGnGBzYy_?cqq&{VHcl!Agovvsyq`F$0;HvGM6H-kV zdu`b4{iA>g5Teu6*2hu3+tPg|T2e=-hrkDJo-X1QmOs{u7K-ScxAwd>ls-&hET(?$ z&njpbw6SwzAWiFxS{_)U5xfZF!DPy(1|1yv~I?&BxF3za$^fN2|U!5s7qdV-P3k zVhF{#8qVS^v5ub%-2($Vu|CGb?~^=qyI_R*hnz(*{7vvK=>RJJW~uwJ#|VJUkRI(; z{WAXnakl?iogOFdLx|lWr-?%Qx8Be&YOP{R`050yO}It6;rD^hQO-p(@O%SIA3wsu|enlab@_0MKDnJ+ebSicw#?LRPqntzLPOb7Px2qVx~`A|bksH{F6=bS>2`tHeX zf98hHZr;nsM6r0JDgBlWdiLtIrj$l~kljLfMe_BgU}jvn{BO6Uspm=)njLR*_C_^m zmIS2?^k!^;F9m)0{G4xd*Bb*-^;;N)jT2u|SYJI{R`6e!87zw@l0-5lGP4q=o` zkFTLH^ND1ofy`hdcHzLusRXnVXiydq3ILFLUUz!Oj~BfYSFID4Dkpv2b`l2AZM6c! zdtp0&IQ}IOeB*Q?9>}!lDW2G4aH(`_CUp9t=S=Lb`#T90S_$WV6qN4o&hh5ICI-&% z3lVwq|CH!#=g@?&mdY9*{gre$GAQJ9Fz*viv*31XVG{m6mHniIqfyYJf3-KV;?K;~ z&s(-6r*D>&`cl^Al=+dq+f_6(r&7~PwQKzk$bsUZh6f^M)cM$T4Ws{sSq9xX4k+0O z1ScPm-Ih2>C7~(JsVPB*Pl(b5HT?%4qA)Lb0iytMdIgJg7&-)XPHM-_RQhP#sa7eqqi;9pYvv`oKUCyfLLj|hR% zVL1EI#45(JfA31H#P0sR2|H&~D{`^1+nCASh?Mz>7lZ8MwFq}T_t#4>05+AIfR=~0 zZ$$SzK#AYn-b^euKlqw`(bIn#{K$qK6=eBTZPu|m*X(^ym|Cp+YYEVA|AuXcg|?_p zB;FiFicnlc6g~QEXA1kCH@6zcj~<}ms&yUW6W9$Lm-&1~^PbUCM+Y<;XWU<%jH;OB ziC|p3XK|}zqd)7*JqFNpOQV^k&tJESR1Et2sg42i&U@$@qr zZEfP(R~Ev}=Td7L^!`BofmAmAQ$fwdIJSCq)*x7@6~TSw2& z5aKx2+RI!?x%3XilYs_`POo_YsMEt=q(1;#R&8Fa^S(v5*yGX%=WiwvU&cj|Hh7gy z=JBt?)t^Mh0J`jcY=9XvEW>`g_w$T^Fw|MMPS6aK;C^uBW<;GkwYN6=(i-=J>Ar$O z+-K%QPM9&Wqb%aChiq-3R{X=Qq+?ZDl&k>Zx3zfgpWfS(&n_d(8f%MVbW@q#7Z{+zycRZ zRE)0@wRBSV-t9>>4YTd{wjfxd()X~%P4jU_flmzwJpw}g?_n3KjBFpk1$PhdWpR}) zn#+Qs)oWub<~%Qi`4A8HLmZ~^Nz)ee;>xdB$`eZk^y{W}H~o9QJ0~6L^NCV-$Y&f^ z-KXQGN4WfUCB7wjyo7N6yZ;inQQC{5_u!lixZNc-`%-Najllw7@TaxgoE*ubp!L!+#o2zl8W@{bwuc9r|w$ z*$+SPfdrzJ3A_>49ZT1;{7>Aq+tKN9M-(IfcZ!{C7&so`RWl{NYy{%{z6&(CCqWd! zkl{(Ri30)XjvSKtk0at2VgH}?u61g}E&`0W`c~X!sK^R|4HJ1X?26V(xY=k8UU}a^rhv@aTF6KaO}L`l+88$?%Q-wvA80rLk}oBbung z%aEToVTg3?w3sA3Ky{(?)W0*2`?b!s`a`wJe?RtDQ#N0)Wb=NP6e37W)R>u2g;X42 z7%^)))4sXvoHaW=I!RNh@V<`x94`^CY)D(MbQ*@is2F)=TEQiV$>|?$$I5LL9#cKm zhas}x)zjCCd0$-8dRCx-mW-OBHXkX87>0DE%v&LlYuGVP(8`VB+#H2P`C?v}vhf&X zASxu&&<|}E0{W`*O9!a>=Yi2|2-jssJ ziIcebRF<3HVz_xyoQ4Q%5Qk0ShQ>xJ{a^wN&Rx!)0Eo$)wira+cUK6(!GK$7bGd`XQA(sFvI$>%ZV!bev=08 zip^_zlT(E)%cJ`;2AZzc<=MBvGyq+$>co@qtN$vVJ>wfl=+qtmz?M3B&i(MUmXVy2 zikg>7^QdrUYU5|P-OX&?HF}y@J>MK0_YL>zZh`U z4UF+k!-5UamTB1TbBx6H$&sx#{j*vZ$jwm}?fc3 z{8Gn3kuMI2(nQ16gk7rt(n?QXXi6x5savvEX5{M$UbtGBTs-eMz$lvQ{h$@UGNsnO zclR4KB*Hz@gwQzPI#cEV!$?{8*H9h{J@abL9jJTExqk@P3aT2ZLbTDf3h^38U_AS`r@1G7ckc159)0mx;dJ- zwN!nDIb7*zRUYu&1y@zk5WKu`9HqAARqB@5{@m={OyCTG!v5CBxzp}`<$Fh7vPrE* zN*v%3Sn@H^1`u2PNcz0UZE`ut=5wtuECnysC(&~JM=CT+@U|n&o*mrXy#NSDNU}kr z&J5l`)?1Vm_fM-jjq+s$g&|js|GHqBHlSOiwfI0OpK_u4q?UZ}(R$xQc<~AyTbq9z zyE#fkdsm=4l3yJwp1k!19W!P^;k>UpjNS(;~JTbjejNGgLLCW#M9+i*&5Va z+cgJ|12@=|2Gyn(h!pN{-?}&njm(yi@|GCzoxG%i?%3#3~>VMbbU2mGot?E4F_D{2+a|gitq?J|wNCayAo*lOhPh7Y?dGMN* zs{!h<-rkphc=8CWeKX!e&DSP9qqU~KlX7o)18CV8v0MGw)3p2r!M~+$F*cY8NBbuc zO}?{iyvLEhcL5H^nyevOg)H$~+MabTO=q!@#~%KdvsKGk6W20mv5R{G+J#1Q9|v6AJ6LV8_|OLu1&pzc%~ zQXd=rsd3mh^UC?H`r#Of&Va25gZI2N%7lbc<{d8Y$veO^4v8t#YKGcfBjU$ z!Y4o!&SQJEecLMode%aFlcttDv0Rshx-oS5f19KDfBGy&nltb)iuPV#SMBMXSDA*d zc7pP3|342y41~jTU&k$F{LdrINKU8Nk_{n>ca1eg6Y4X>t9YeP~NC4_AN$M=`(_|MgP#$Xsx|~ zJ<`9h`N1RNOn+DCFF*kK*t*5Ww4_Cn3!4QmG2udAC;m!KSAUIw&!|l8yJRujM5zTt zT}8yTly7>)bkFE+>$@JINq9_6QY%K;5X94gM2?88Z=hsD)e&rE%(-LQ=8Bn=(!@$1P^RxsxY(>hh6cP0Y?sEKa zcnalBlU=O(;}-tyJMI><1hXcb_s#{IER`+9cx`1cf7GdIZh}D^;^*jFEIJ<2zfVqAp0Do~^s&#^^n>&xyLKqLMNey*YfexPmY`@J=F zkTyLY7WO;%p^6FY_0{;^dZ(tvEe3HS2Q8=Q?4*EJ3cCL^N7fMUc#>#d1xHmA%6*)r zMd=TuQ+8JU;i!9SYF~Ybo&Fd8$lE@FY79@dtoUL>@FtT$k%(il9n zX?v^uA(}~0(CUxjaO;_fdb7NhU+oXjpu911W`DR1J2Hazw>32K=1wvf*Ye;J?dyJG zbBq7ej&%zyo;N$P#q?6c$mSq?(rGAH(YtF0jRP3*fUrFJc5~)v2FT!W1f(7-h7HMS z%$hLxLqhd@xU*8NSv}6xE9JXhSq$-3_xz8rsCtMUkFyX_HJkiOrli}(7ldR4 zTdPsdkYdPzW~rD_P>g1B$b!-0z3FK=i9h}ac#SJ(^KJk{X2<#(@wdEIQ+-p%3*1B6 z@@~hpeSeO5U)%l)=wYUhRii`O0MkyGHe2#d$zomB@UT#6{67(-Ik!TkRi27pfAC?2 zpg*26aq)kpHI_$`RRXk#^(vSnE2~IkY1PpM);%)Wh*`vOCYr>1LuZ;?oJWH`NA0Dv z!Z$xwlLXISh#ZbiS&6g|w(A&D@ay>X6{k&=lZ>sA~#GEqHFu;QluWK zb@Y@B-Y|m9DWjXk`_ss_wPQ1}flUj4mtn%D(3PTXA`k5Jyag@6GZ`pwSU?fEYNJ5c zfe!fDIgKs~#PRN)K#TqwFJ3j|bA7*~$c%^BF0gYEt%nA%sTDh=)|9Pj#Z{giUYIxT zi`hl?hk(r^SU1>{92Cj6GWRfDD@Im`FquEO9?{|=hluPg=ht*>9Azgh(AK4f1rs@t zv!!WK&Dw?w)okXLVJ&Mo-i{AX_?^aaQt_IgfXXtVX22t2Z)eFh&*Sx_$InEu#aOq*tIPoP3*0)74zH{AP&oo$5oU0_gcmjP?z za%t*6axjZ;Je9as?zde1u_-OV>=sG0c!fn%xxx9Nxd!k4l=_wiOZlXov{=)=23^c& zmYa=%Epk!u2{R{-G@Ez4G6Qtt7mY=5Uc*sb5~RlA!j!2nq@5-#RHv`B#MTs3bnzNG zx%7qh7Kw~gv|*_3L#hFt72D^A{kpX+5VvwUtR2t$d*(d13j+yN{1(5qz_OK%r~|^ZLgpnJHIS22{Emn-TO&l(7eRnG?%gL zkFc2Tk<`lg|?F{z$*f9ed%BXD$l+=V8bgS&M_)r!^Q zR@AR17n%*|Pp`MltyG8&vt5KF)ig^ASJ{|Ha|1Wo7?_PUE%9n3?U0(S^+=Fc z-!oP%ZcDnlO%u&FI|}TaJ)=Md>O;}T|b81Z`fgNwCI`rbEH#`nEEdLd!we863gn(Rlk z(m|}>Kdi(PrO^MKAGt_bfV59 zdd=MmxiQ=Gs@b2C+FB~NO8(0JW+P2Tg|TSF$`ePO2yGm(FGFyojhPd~w& zo42E6F-H|ukTV@MfKOATqzEIVg23 z)C(1xOsfxZAM~eZ7ANdY%YSlp&dw9=uQZ)W;?B*FC$=+IOy1Xcc2riJ?BD;}LtQ*0 zN$oL!DH|fKZo>kTF$#e}(L|}+Ckz-aPm4BW_$l+p%dkz8YnwMofQtR$cHC2ZAI`Mk zX!_Qroj)rYUmmopIIG;4M>%Cln#>v+F%$#L-S)14fe2VPxXp|9r5<%FM2=Z!lohc% zzJBSgfJgJd-Mb6E%bE~8UoCYoO5fc7+XLOP^;rC7H;bp044Y|ESxhC;F%@bwyW05? zySDe+iOWZGNpsNzG_B`*&&j6NBtNIsS)739GG%s22@r zD?Ce^R1gwggEV6nEs%OuJZ_|+L>9ZTq>A&lPAo2w;wt{Ktq)&LRT{uZdOLc42KP%( zqej|@0A{=^M>qrfwS(!B^mcNc%`P4{Cv|aWgxu;Ikhl8L0!Onzvk||osba$qTQ2;q zk49At&pN;|mHYw7?VVoMPZAcZ^-qzVIkuC-&Ow#r74$f0M^T+Y7hROWgeZQ; z-XEqUn2^jOqRpAfm^a1mi>~!=NuMBQtIf;Bntb8n5?w$A+YLKUgcGkzI)NG{d{v5z zg~l~NhJSNqkzg$Ef4c2&6teyPI3i39a;9(?c-Y9gw+70{k+^0ILQFG%q@YNX7mq!}+`fMi*&sD9Wn}__s9) zzWtW$kWs>nm*;wyOf^CaA9-m;GLDqyl4+F0rwMBWRV@paZWm2h-w{2uA-x3m%YRh- zYnbLQ#Mbf$RWGlg5o=4Te(z5&ByRB^m8{kXw=-V#QKku}Rx}G(Px;Z>{EdM?S}|}E z4Lby1(cb_Z!$p-_`$kk-DQ1Qdtc3g2;7|N@IQ|l9?heYG(B>BX@o!P5)Z_KHgUR{wF8{ zLwpt{2kRax4aC0^8EDgpxVxW`6&Q-r2*)dW@a4)(GGx3a7o(jN??>AlrAM+vdCKLW z`Nk%4V}@Y;R}x(H&M^C9qK4ttJ1(9m?BRUziy#LMNA>!v)8U$a#3Xs%OFy|Wp540q zotiNOBQs$NGi)dwt53sB<;MF43I~=9wZ{iGxkFB@={8QZxx6=u`Oh-V%N{F(jm>9SLe}5|{y~HmC<6qr`9EEh#lJl&N27BV5MJt$^>{>?$DQ`LjJe zzH>|J=C@p)d%;$`_cMj{Ssu=5|BOrS9(<}}IOjJ!I*qbFTk>tP(EJ8RPV22^4nb&_ zwrkP)JshB1W27eiv%|W-gHO}_j5tq{S(!kF`pr~)sK3mJlf%1WJqb7+j~osq)W1H| zVz`lrf`5eSO3pth=c6uI%w6{umKja2D8cmq6pTjoyPzU0W`W@$_lbYp+HMaqcko=D zOUHBwp`$;yT%bwlL5MpXup7f_nk6%Ut?N_s8M8C_em7=73qu+AVCCOHXa8KqnkN!} zHcK5SdthPlxsrZ~mMQP#G^Kq_tDnMcGp3){?YAOBGQ%(Z%p4Q6kYlZsoRPe)`dyU8 zHAYGOcQU*`ns0xJErW1_t}F25gzoem;ZydL@b%`c_f59E`;uR>vAFORv9|kt zu&7tNFU8~Yd&s*%JX4CgR!5ygzA9_F@6cY+CI!la&q~ z4V?om{!_PFqlO2tprtR{7T7aR|e+({IoX<)na1S{xXiYA#}Qak!mhOD0PXP3^Cbwchk@Ll>SuMS)i|; z$=Y{yZ+bl{VnX#JWB|7g4^PirXOzx(JESzulnOxIR3C0Z5=ChdMydryv@SyK*WxM_LM6nDz*;RXJW_Mu z%}a^g$lmZ;a;((o@o_i43%bf1>ir#v9qmj`1+de!>&82hXMi!zt3wn@#aAz0UvJCM z&sj?o@q${he(=2r;oSF>dH66xxnoa3##8S}SHICjWB{9UvEB0mpm_4U33}6RhcEaKXLmNVaKg+>fhOF_MhNjq zh+cngov}>QE6x<}#oNXr>%EytdlyTyFZG>%$}7Nl=+~~FOQL?@Ej9m+>x!ZR2AyK$ z@Y>u~*0h~kp6XhnDsQ^^F>Ctj6p)rAOa?L;5BsK!F02Ao-jw6AHn?tH59Y|5DIREN z{qr%TYqe|OTO3I7kE`BumF5W(<;0O19}&tIkKc8F5_!%viGJ?4Vjtam0o{G;#^E0W zD~dGtR_Ij&kiO)F2d5rA9F+2L#JClEKYuWrkkJ@w@bVB*$@@@2 zR@P%wR=UxS@SO`6_*rF3GXQi#vV;!ROASA~BA^jbmuOz_O0>)VAqqd^ftd7<{)=el z9n@5dF|k^sZ`?R%(23a~2Ab`B_S<~%PKUcUqC^|BB+O0P0|164HF5`Y#xa7J0tH9I zRuU=_s_K8b@0?Ce9iRxy*i7(~(XrX~klL#I10z{28ZTPr)LX`AcIS)dMQKv$>>G=w zTcmbs0nw3ej`YJJx!eHXwlfbs{5)S}MGt2~oT4@oL}`=>9<*yuAnFc@Imif2b$U~tvQ zYd;Gk>#VI?xgYH$#2-`~8KQyZ3QL$kUcFJwNgL?UR5$9nV`fvHhi?ZLK&}}^G}AWe zqVn%CduPYvd%~QPd-}uyr9TvN zd4D_Qdb*}f+^71rKW}prW+BJhIpI3+sq(f*3T6oRnIqqFh>5qSS4&IZrx!!?XqC$p zher<02`$Ugu3%Ko>zREqqkO*&@io28WXU$WJE`U-kC8tehl#ow( zJo@ohOkLt_5`iQ6RTn=zMZSREB}Ha;-6BJ#H8m5qZ8Cj$c(;Va9$phRn#O z%QN==A1Y1clPl^E9rNBCiF;|9)&GpOfQ3fPh7mL1|tKv28n$IX!xjs&+!R>ejJhZ3OzsJ!8w z^57FK+gNwsgh^oMjK|!cUSD0=?_U<}(dS^%7fPwaDeM9IXz!0ifd3`Eq<8#}q0bz^9QNMX|13e)C1ei<$H}qROMlyY ztJsJARPHjmff5D2q&vK2Nvf0vXQrq28J^=HY>CJ(xjtt~=-F9XUux264HWbzdKr0d1lU|Bsb zi1RSE^ayLk^xkqoN$_A}=-2J-AAZiUxyj|u=&u5e68B?Obl9{IUe3SJV0ym$y_8_1 z(qgF1L9fuBsZDWavs(~udtbLVIpVbzgRJa_A+~7Dh*%?1ct>wVDp7TFNRQXFr_I0o z8~nqKm6e69g`@-?Ga%d{Y`XSkf(8`29@5U5jL^xfFIaxB*3&W^S3 zp$flUmVWWySbOiNrn>h}6bmY#qSBOVLlmS*?+R8B5Fvn6F-ot74he`LD$Pc30@6Z> zNG~CPLX?uwLTDi%CDa5A5FjL(gMRPa-(7djteHFa{P8Y*mn*03^XzAT+D;rgWXLfR ztk<%fe4tsO!;BE+iB!N6h z+4^Jam+c75ph>r!u{tCzTv;Xyips%0sf$heD%0@E*Qd=|^L-j{g+yyQg6a_z^5tdm~Su0R7xsBfbZ{l_m* z6KUh79iJwe7&9aJQMX(w-Z>?|C3fQr0POb)UZ1#h!y$B}P>lrO)eOCxkgFV?6d)o5 zFuifS?+T#5uW0tt%a#uIYggRUAXFOh52hN%EV8kMNW}CxG9n^>Ru;3?+>`9et?l7( z9W95;Rhn=i!P^nPmUgOw@BTCo{u!SiS(5(^g1@Lc{w56rNjmF`S52(o$Sc%P#WWRi z1FS$~!>MgFL#l1*)*^XnQv`D!Qk-Jvzy8ywSdfhOd~$WR*<4to`1|HBOUM3*9J(k% zvcYUYRy+5J;GV^gE&Pnhj+n%CJLs%LiN{!g+0mrt>YzkdeB>(*Nz*qOtFqD;(@q`C z(sNqv2b1jP5ceG%S3z=l2wZ%tE zs-DRI6SxNkdospFHUtm*ik~%bPJM?6I9J_&{#>?-R1{B&_+4gK?J-ZScZ>awUSs_# z{-x8W*a|^<5nbP|+Y!RHBRvs>8+b`+g^oo7O%sl69?^nDb?^#4J>UV6h%|{H+~8~O ziWhByZVQzxQnG~!cHr-o7jz1B8|>+E4TWCV&2dOp&}gC}w(k?0DpnC-(?pXwrz43- zQg?vc(gW6X)1?7Y@FkDo35ND8jzUlJMrp>-xx<@#4xjof6l{Tg+~mt{*t~$dK-^AV z%r$y zG^hdX#$%$}iYbBC?Lyi3BZ8a~xj^hY;LwUX@MbS(hwZWXFmoBgNBDd-=C}CK(BCK! zuEVKO`tqXw%yfgpqtRoE-dk(YHElc)S88wzxbA2Brn4CYa3_Idw(F=`6mgt}i@{6=4Wp zLov#eDXv6sPdu*Aw@PEYOVgLCxF$Pw1_%7x$_WzizOV@)a?n zf6n2_{+=Ezu<*2zda>*W)x{iad)c8nR9yh?{uRecAaCevB>X&$2o9W~Ei^mAGfd+M zO=ug1bME_R?dmDQG;WF@LwNDT`cy0S95NJ`XdBV%GV4iTY*QgCu#?VG00 zwq+SkS>PRZ6vyIHMU%(V-5!2HaUiXuoK_txfhW@lM)^Lv(<7s<)7SlfMnq9|ZP@|E zQ14jbdJz~?#P(QuEM0U=T+S%439($i&_dpq<=WYwcD{8akH=2OmDG?iAr5^5E~h7Q8}imFOl`pcKHc^<`})EfpDV7B>-Ch24eT?J-Bhm1_JTJBZ;Nq z2Q2tk4Z^@H=jN3Ea=%}V6SX^P{q5-Aq}jhx@-3cQ-EwF8vwHvBg7nXq1o3&;j63VX0DJNo5SAQvV$ z{+dP03zwcNppBRq_+kfRizEQ;KsP*s1Qgv9b)w?wT z7@v;~+&EcBrZ4i8FfQ$hCyL?h(V&6zFINBL(GT|-B#>VL7QjGL_DaCy2bANp$&0q( zss1=1`fPv-EaXbmre6(|EM`@fELLgrwb!z~!iRJS&{?N{OnOVkr<#+%n%*A%mjb}< z5vPmTVEVKSfLaEJj2bG=(%i%LK;aSqpC!FQl$L~PvodId^iRZeaHkKz#S&NRzhNy~ zJs_+m`+${1Xo#5c-1wO>KRoP<)y7of26GQGz}*7ah`4;=k7{GaGd;2xj; z6S{$M+7$6+j9ZjjlS_!>w9eD%;MAY;E!|_|9C27A0zL#osXN;+Gs$iMB$TpoybO3X zbDwQx$>n>)A-b+aXpHBkqjd%?noU%?Bk83BOSkX<+XOj_x_OX~PKV=&jmA4fX-isH~Sn5y7rM>UGG; z1cY;(csX(}MJL z=S}AI^>kc*m_Ke6#&MD7?WI8J)-@6PQA=oJWmt_z(C^}@m0OZ_Jj;&sjX3P7+g>|6 zI}INKx9PGTB9-&QVE8Nx``%;e6(Yohq?s*ryr8Ro4=|g0S06;KRtbAz7M?DrM4H$` z`EPEBeq7W1(~Dm`TSpsxzjDT6gq6!>F*tU+r?nRF;U<#|uaRFg^>%YnolY`qg* z%weKKFYiNNyxy}-)jqS)7G%)iup9`sq?LPmO`o5>YzIOA!~*#Q05Exn`2F)*;Whf7 zFiIHKCFcF~^Ee@6N_@SG2T2SSMt3HKQ4jvS|KhK>HDizPgcD64*?f7T-1;QqXJlLc zirxgzMHRjF<$zW(glkksdlH&esqL7{cws7TT|V~7LygsdC8#Z?2x${0RBkx8ty)$& zJ1mVC|6Bp|E;x@`t2*S4S=;Kb$PT~&pHPfeP(<4=A8amziY;TNuXu^ollj7$zfpu| zH1~MpU~N-*8^vr8ioTer!#N|I%-m+32CZbSs*TAkcJs+KimM`N%jA&Yv#gaQvYWMp zDsp`b-T)by2cOECQ1HF4+aKBImc1;R%{dON&qXy%I2@?@&NdzIsEP73Rz2*i%GzT{ zM-FEdO^tqR;Q4FJ!x)h>VdaVVJdMV&Q>ylI9TCB9Mhme^k>rhQLpD0!RwwPVty(|* z&8VC!Js1mU_WIyg?SkhP{EAW^s7W8&i`c@6*iWe1ya}Zt(|59@_W6h?Vs&zl>+!f6 znQ^4hqcY`-_SdNFmQso#IeO)f4`U2Q9x>*)9P)xK85pObjOfXJ; z*&p$wNzK5UxU`$2b7Dqq78h{zK+S>5JMhp8F>^|EF?*V&%2&f@UTtF=aqM&YvmToi zTX)$US9mhdL5T+@B3wTqLgiS4*^1dn_xrJO(Q0$KL5Sr!)O zbymHliZl^CKa!$`z7<(9f+XZH+WCmpr=K1})DzbVuGQK%#!NB%4P~mbJbAcVSr*wI z(9KMp;E=iVh`j_3PE~JlHp6!f4Ll$0HVA#$8~T?8!&nSJf>yv0FRKRIovgEQ>dp0y zNhm77-R#KD>g>Xb5?FP^-zr&~9@}>)eS}*_9KQW~;z|5Ro}Fuoe8&-1dkj>P^gs>N z4%jHP@#@MB&%Zc%dg)9$vi}KOf7oJA zAW{3h)TLE`p`?+I1#DW>LaU%}jPw^Hd68dSD!u|OGMw0>LvE&VK?IJ0E__H+GqlHVELw$?5SGmV7%aQUK zH}|&(uQeXN(}_PNb1CD#)Z02%YqW%Kb7Y6OnWtb0T$!)c02_SAZ|(3H^fgbCCoorl zgqokbDS;lP$dK&R2Mg<6fLOXdHCh9Q?C)CHXBkMYNj{6P^=~`KM@qKjl3MmVbF8;r zx3j0m^sto01RLeWN?|GxJ(u4w|1yt${XnI`{@?5TMhTM*?LQtMCa#`HzC}(F)7plR zM-}QB1Jhe;tO?sMvdH(QWlg__->(@Ko?1K(_l~~?OZ?t_ac?=*k`aJMroTEc=B3g$ z71_a&z++TkRbBg}*6bpJg+`T>qZVph>6KVqVmYDr&pQ+R6_OObQw@Jinw&(7(Fuf! z%rrNWOF}I~9Lsm8p4HSv)Xq~Skcw|dl4uV{Hfn1&?mGzpOg6(2FG8}_x3h`sNf!OxHUM%j~S znA`e0_;oooaFx$$a$FC?Z*PM$!@pUKT3u|Bo&qM*2DdsId5@I#vjr@cPP0^Zdq_{= zl6L^Svz{kqGfM+(IIkrXtAy!pEvu~#Q7p8*wQ1~~Jy{!xdekjP8$<-xjW6BG3~$h^ z{7JQ?s$^&3Ry4!=^bu26KLy}exNW^h61|KrBE|e*@?_omnqg7X)CiMwER?NzV5F5@4{U@ckD-v14@qq1>ukKpgXG zMZm1UR*!#3im!8^IEd^qmX+RXNsIo4xgi;+P{{F7LXbJPTaS4f??4`b4CwJyno0(+ zrffmg%msMEt_HzlK`JBZgiFuYsXKWIzntu?l>0H~PjMFJ-`qb}%fd48`Xw=Dr&JJp zE@AKCbHWyIsA*q@|7iw14Xuv98PB*?tMG6sm_c^wRYHW6BJ1o?Yqjz8m!9cy!XLq@ znaAn#%ew)vziWP;Uh87btcr-v$o!95#G5N!5aYsM@s&bMZ41L!?0*0nIi@cK<;VSYqVyHDPCVul3b zhMn-@t#QimjfaNF6#)X{&) zUTlL(ah=r2oQ_7W{ZDryNSQW=20mltdH|x(lE1N=3y&y8Y~g>Gm@WIBN;;7EGJCh) z0_p*0Yu|Bs99kW+O;#{Q{P%xybC`pz`{2J<@^ANo$CwmAhtm-EUBP(50tp0E?o>|! z1imD}&Sm=@wqs01`W&F$L7D9dS{jVKA*{cvD8?RrxWPPQ0F$9RS$7N>cl3`3^wujh z1Kn*dQw<2;#w@Ft%w}~dJ>Lb^Me^P(7^HsY6OV*%86fYZtTt^}DJgDFKxR`W){doB zY9_y#Frd)RQ(j`Y*nCCyB=_AX_lmv9?f+ZIa&%9~B4M^`<<}}^-#5Y}1fvcX?WK$| zcZbHvrF+6mb@DKH5vl@ZWW)Dn*uBs-K*Gbcu&Mc`O%p)*T2wE!6L8yMTUm?0jULp` z4@Yg($81v*ifH!qR8}5@d0~l)ZWq%;dCI>B z<=`Z?Mm_JDTMH()Hg zbY8=W16m2SCQWo@%m&TaB>^hbYJkwJDNl4t+m@z%a3f75XY3Afvd#Ym1-dynczBjC zu_2pJMwA1KLBG498+T-cj>ns(Hn`!*=78IFucbK=0Ft;UGxhBKWW|}wB=r%ML`1zO zRkNKvtjTqYq|w||A$o_uc^sYQN(dQsl_TBK2#-e2->a`@1DW`QWyN>P??TyY9&@^$cp z!3i6i9Vb>EpsKjQ_ak4VJwFVFD}oxln`la|Xcnkd-XF^L*>Eo?)w{*umMajjW9)U= zdQh5&U%E5_Xjd=5^PP?#*Zc2s7A0?%2e|58icXlFNO0EkKNN1wfB<^f5Ty`!b1!Ie zCiQk+t0HP%wOSov3-M(N5`K*dhf&tHp;Pl&tz55kDGR4fO|8Ped{9Nc^#agmKXzY~ zB>7M5!;`k4!h{`7+C#}tjneXm!el#l_p{~g!8zdv>>`He%dU&mh4GiuXA=FmryerW z!L^#ol{cR3u%W>R_IIO2S+D?ES(%J~_x_*2F-bl)%b7t&dt3_*H&3gS%017iRa!p zXVebiuh*b(6UPGfwe@oa&Wtov4+%08?{%yxIU@uRu4p%f+s%Z$ z_iiF8DWKS{hwV9;!qprIBytf3D1;&>lKFW#4*%7u&bXTkNW3E?B8x$BBO+?hui4NlMj5h;QFsCx2(W|$qYj!$3o zC{f4XRU$VSJs3Y)NDbLf?8Y`~WUG)a?{kO0K$qM$dgh%s57Ky!MLi97R48WZf&k0?baLZvPgh9^dYS@ z2|1^anf7z9v{ZU_Ba+!Z`V8N3BB7{Uz`J)m2~6Af0avQrSkKVp@ZRVIj{o5(s9%3_T6b^HGx`{4e!;w69!`yXzNq*kSVeNTG9$>x~;T|4;po?`c~`v0dCi~rea z{+Dfoe{u8w5^VR^@Yizn5A)bx$uwS8H7}yxXs_7+pPx?doCLN6PK_^z=W!$0Ql|d` zy=)ymFbO#(F5Q1}6CT`K>$A1gO=x?~ zkW>Kg8v@oo7%?4Lg$Qc@NlBoTZ#Jr+;Be+l%6npVAdt#(b}#ONtz)As8-&vk@C}1K zp$)#ou^7YePpFeDk@@={^Z&KTvrPv(JE_b-4JGac&L#7)K{Ec*1;@+q+J7@^RY159d!26#0IA;IrWl{u-p&s2J%HsF9awp3Gis7Sd$n18VV|p7 zjg7<&ijqWpGi*S~YeiUtH><#-gQ$sP?#={dx=Fj!p2f$C)99=*?QLea%fNd`!5xTc z+k6D!%ak1SVlao4xYNsZH2JT+ZrsIhCT4Y54*hwUp<8`R8@9fK9^ zaOmV;gWcrkUZIv|2y-5av6TA*iRhXDcYM6A(n>9{H(3z?;x|iyU0B|K-9RFiA}hTp z_6*G-H5S_Xc67&kkL~5AJhTovTUa634g(Nv0$!rqjB>Gr-3FHw`kESMF4W=@gS_UWV6(52su%lT4{h5+Ohh2Um1T;eg4~ zxsJ392PhpfcF)9D(rgWM?T70Q7Kwkk{~%-N1czUq%2tN!SR2FA!O`$nu=&ZIW!A9~uk=oE#rt`LY zt(qvNrVQ#3e~Lg;;pB=Sac|i&)y4W?&!;HITU*qSL-yQ{y*T1iD^CfsHa@(egt1sX z6Paw5Ai{Pt%R>J0OMcbQXJ6|5h`izDrs#TNc}eNUi;FZZiFf0^h1Ix8=*o%cclh)E z(FfzcLuFsrbQr1M%UR77bHCZzzg#mzGREko>PyR&wA96n zv@O`qko01PRw;5*CIYpC5Azsri~xgp8Xs8N5uOfJNd*Y87|(V8R_n~<{bs1>iimpx z!eJGhIbR~ivdgNEPetG_GM;eUWu3AD{TS2z5prP(%b8$X@`mT|7cr@3M1-G?>ZY|* zk=ARjQogw*tra1}kwQcEa4SHUyZP|rt2D;fbkV{b_eW9h~vgd{gBQh1pzz z6rhl}CA%xqyJlR`hbUqO_vo?n$&5-}+?U=*=ra>KJEb)Hv8UpuF-KM;MrEZ3Wt5Mz zpg#Cq{NW$4=h{v7>kVPTwCcvJDxWR9m3y}fb*UC}xhm|+?e#9ZkbJNYeqg9#@A5L= z>rPQ$J=J>vs@lo%u~^9rAy1C?T7m{RE%3{)we2BKpCzK>M-6zbe|X&y(gUNpX|`NP zQNL4M!EDvHy^sEgJ=WHR#{Ptr!L6&^y$i$~{A=;Qr#=wVr2{4C)os}H1T2N4^ucq? zpK%W;=M|?=?X86^`4P6EKJ8IFcLiT;`Ht15Tbv1agTNOH+5Nn{eTQ)=Ob2Z{pdi$9 zN&5`rQOtBb+fW!mjM`n{C?NW1H07D6oXS8#cl^R2Iv3ondMTiBt4YArnW^YcDBQBF zDkC0lB(^yV3nqjewT|t*!AO4u`mSj%!MAorV~B|0*cC> z*kcie!u(NX_qcvt2?wbzMPryD*la%Gko&OtE1V_ChTXQM<$3k;Sk+6=?zEUE;H(f` zU_hnCJQPqF-1aN4+8$s$uq>D7%VqHGpc8T!qQYLl8@XQlyEga1q&oZn{1JWgr!BA1 zZ)7-D`yyw|Ew1rQi1Xsq^Y{&N{5Xx19pl1VPUri<=`}r#YYU`y+!VtLKLbq%n`pqW zV34suN$8>Qn%v)~LyobhtPLHDm%L}cVRg@Dhwct4w>o3P+%DPy?aK{|rDp$PK0*-v z^p$-F0SPT%wOVg|;n6rdynNDO(#|iT5h&knIvR8qxz3m$_tar*zZmE?3M%zTLa9jzzxO->t~`P&Visy*fgVy}kT52!!U8 zW4MA>rVqb^g&c@NxGqiB-hL4Cs!JiU+Tm;(M?Bx%oy4!4mkMQiHgULmFYR+Xl@XRv zB_15Ie8-opWHryLs5VpM`V)k>d#9-+`;r<+Ye+_fE&CAXq91a{%0oAP1BSeR&etl0 zJ0LWSIX>RFJIakPPrTKrmkm)TDv z(X>VacQQ$)qrm4XA%$@|>!A`MtKdh6a5fio#ezekn(STWoYQC*H9EVll(e;LOV=mw zwp%A`<@nl!61V)6)~^9C@w2tbYaxrJiF!`0uw8TBnVnrGQq~hgynTcA{l@1|QrbFL zY6cW2q*D{D%-TvY!95qRx}1e+l+V$M6v8j7y#ZU4(-HBBi z^g4PbO#?ZnJF4s8(ce7#S}epx*Dq4?JK_8bvK=zL_rnDk`4Uqes21U#M(4;uvdJ|o znUvdN_w==1USG>LitoU}D-Ay=X_Q|29W%7=dH|3T&->Z# z$GzsT(vj*3<5w!^HvNqi+a?TW@FmknP3>GtcYGt1d1oc_p?V?aWLM3#RaH?N9$*8z zy@MAh-CrKOrbE5tn`YCyI~+&#omf(wU#xzu@m^)|4zcT0JviB z$`5eG3BCNhw(FFO#q^*_o~)SK%CMo|K5M=HJ0~3}nc$j5w?5qvPoJj1XMe`0AtiyJ zjZ*f>s+J6Go9jY|=R<3RTtC;3IcteL!pW8M6S=hKD0B82WbM6H4g!<@7tZ zbMhrUlMQ1hLISMKj+L^R@-7ZNs*=kpdKQIQlJu>%ocLCm%jqOSQbatrXszbb2B|Gk z3O&SBrG+kY&nH^5i{MX}G$83SlW7U}5zd=eNs1?$m;zkhEb@)pBrb{hUOR(<}M*+>z-?Lu7awPu6>FQUj)|$I#`7w4h{Y~sB z`483!W!|E>sIsl)f9o6^9%LMJRi|kXFQFp}!YE%ln693+3DHUvi9&JaOw;Ez*Syt# z{g9xdgO{&Mz3C`r_?2%x@!e7P=&t^}=$2_flk{M=RU{1u0J#Ui@z~2k6oZW0Q)SlP<%5hvPpzI_zA3$-xdzSee><+0s>WeY6nJ zvNZ)IkGQ4(L9=wa+;5`h zv3TT@&5!5D1fEmZOpa)Mw%An4WQ|VN4I~`dKMTuG;7S}|G_Af~#>WsZwEP?Y1ff=W z*V_*1To0ZtbnJTY3O9Q2HxcS1roo3(pkeOJCC<8!<+pz0HG`aITPt+kdW^>vQXx_g zM0GCjP#ecM;@$4;L$a$1hfC^zv@K}*v4VoXmEyu-a?K;kD`<*$#+F1-VYc4qYRu_z#jT|+N z6*eXqfo3H7Emewwad8D~aq{`*jh z4n=^?2w!p>S>Df?OC`5XJ2cB_@88%Da}OcH2!6Z|Y}=>`nh~arL@?Wdit=ikxjo=aUzqqEB=|9` zTz6VOxMrHDC-@R0KPzX@ zXP&kj69$#;+^7&cW$rpRyzYP6Bfr-aU%ve&x+9k=DYt>RV3!y&-ZH+z7&-%gwISt) zKHg0TSW`Ec#pcEH=p@KC4^LK9P&JoC4DdT!XMRE8;CW~i>lg>MufdizA0+tTaeAHH z5!#$dPQ93*fKW0fx#~!sv5w;1VCKJp;u^I#PTm@VMwL5YiFSadGf00d{MYwys8`u$ zJtc1qvfI>kya#;vhU?*dLyUN=SHgq4O@>j{lNfG2+s3yw4{+b+gBfqFGrdf%2I8vg zx22-yI=LTJvzqW!8fqjxy`3`K958xrL6*Fp3eeph~* z)TgwCjhHjV3x)CfiV$^%9@ev2(v$l+pLED?V?W!wh-6_y?7{a0liOx>u>(i;M5Ifw+xe3 zzUN!zv+BhNc$rS^2Sd$!7i|J>KJ$>vkiM}R)Y|V{4!nF@60d9uqo<+Yn#b85wcmW- z?5_kT)NzVo!Lf+_-cd>FgiHB6~-|03E= zdzT6CQn#r77Nts^KO`#o27~!wI`VYkp3`SE!HC&c8`^QVRxZ5V_&H>`OS(Y~Wh&H< zb;lvYI>95;i!iw4cu!dtDc|@~c8Ltc5L-1B^sfoHe3%bCugF`vjrxrU-x8?cd}JXa zgy3X4lm?Ww-9t4DA>ySkO%$}THW z!G|^uyvmJH*;EOctQu$VF7M!4)_fRe&&B$`sQ)qcVyhgQ3T#cgcT~wr#`n2gWN^Yp z5wqM0)me2?4)WUU0Y=4TlDu;CF&V3LdUFIw;7|jzDbZX+mguhJV^do8qlBaE25;#O zeoC@F``*0Hc>jlEiXcB^*rgMG^ma;nrFeIA{B*D}9qmKvYvQJeWuif8XzX-{AjNAX zy59w?A0ql;(=b=kBRhhyRVgTzJR047wZALyI%P(G{`RQpzk%a_+dEd~is)|Q!1ua$ zS1LMZ&SV=SjE)g?3jRz6QjJR9FaT^JllP`_eulL}X-&XBa-C%iCum!;Wn=8o);{EZ zjD=~iCEe`TO11<{I1aqEIZoYbt-yTT)QftkJ5*!#R-zB`Z64V+R>Wseaf`NV{+}6B zc( zFOp5=s4v|KB~$&xg4U^%5N|rKxhSzLn@a-DPK=h=J#kKy*J2vh{?ZHdfHf1hzJg2o zU41@BJ?9^mJb(2_e4luG3E;E7g7o9+i+Db3o+}v+8eZo2S~~hWW=rh03u@*wvJ9UpxQQKlx2YW1v5^vpk9i0nFz-ET@fQ-aQmXpoH z^wA9gepDxZ17_|`7HiCy9JYV$6^SPfhY_lgud^lWe3lPS!P}+%$uw}G`0u^1W@iB;rb>%v1ud1ZihcF-j8xgZhl}T zHvWWnln|=B?i3ZpN#t)5ftJAs2AWn^lW$qEU0zRLW=BT_Z%F!@InV1}uZGOuUy)SQ zj%t-TVo_Z-2lFY=@zFe+U_shRY}Q6~(yAk)7&iId#lNBP*4RoJ8g?2CO`6tGe5cm+ zas~4f#uz>`;!SJ202zxbdUmtgO1IluL{sH^(~#(aJg0{-v+d}7QhS^Mqm1EXHf{QN zBGoGE--RQE_|QeE8pNN++ITarSjNcKnO}+WhZ#Q-VbQ^P+&YSOq6b@aEbX=|O^j#&|h5jQ{{8F`4DTpTOWuL(M9FC!zApl-;p(Ev7d&Cb@M^W{MyF- zNRt#{6nf`Ic=-r-oS*x~_woXRFtHMtSn9D4)6>ss=a{jJhxyjn#Qt8;=L;u=^Bj5a zGB+qx`pFXATB|{)%@2`~*KL|n{t`NNyQ%^m5k!0bguN6fT=I=1qP$0gpT#k8bLO_Z#t%B(8*z?Mx^1qAe0PFlPu|_DF%*VBtq6)lD%oop=NgnG`uF;^Sm{;X`pzLF*?O0d7cg1Otzyj#a zY5xe-Y)Pc8MqG!-w#Ur|AuTVk0@rW90;inHcrF}{zWDK|bBy5M+M zY*Dn>GRJsKw^Rfhbb^KRY%dp;`dMzv+^~V;;k~8}Qhk*}L+q=rdp`keZ3@L?skVwj zcU^$eeOK>acH3~*?w@9B_X!b|;Z+Acpm{yZ@lPYO99591GGYgvvhXM11gyU^>1Tl3 zNcam{ux&oUA8uXim*5{BJbU-h=-6jy_gq|@_ByGYyxQzYI!BR7c%icJWdg{$?0lHKVp?tB*CV{YvxL}xvVHguNA|pk1>6Pfx~?IV(2YSe&~u_E|PsrI32J^c;_Ch(Yn5l8=>8zWFf*&S~*Ug>9P-b-)q>~VZ3S+#+po-Mn= zTv8o{Jxg6t!ie(Pklu}xbkV_!-*-v60dM(7yg)X6mcA*Lqjj*ZHIxpG&v+@%^9k96Oz?!CuS-C^J zkhP}lKRIdTYLc9pyfyIs7R{<>tyJB7F9jNKg0R0LBSwvHe6*h}V#oN}_@HrWRnSo% zzQ*Zgu=akOAr)J zb-o%Ve|5C{)2ijZd~L-;E7`o34Wd6h%MEfW7SgP*Y8cCWva9YXEte4l_HsTG%5az) z4KT}Dxz*3W@6H3esVfEGnNACd6`Y^3A|FCDZL=|X_>I^tovKwsRk;TA8pDWF`_M*w z$GJ42sC*}}Q&E(Aw5iw4eazry$zyTP`ae5&dmM`&n0t1U=p1i9YPNfts3kdyhN9}7 zR9)=&YL>Pk*ti{ci|L?!4Hxz}VD)0dC40qi65DmByv}K#!i>jMimh`1fg0{t^|gO> z^)lPT@Qts@PAqqi^{oB1UAHe<$V#%^hu_5c*m?((8cj}7eA5pk*!aM6 zQ>Y$|TdIwHyTA4h5!+nIY?e9VYcFA2$c6q`ZVCqB97p{wZ4dR%H*qd1nxpgseJ`B` zt(v>yD)y_c6v}B|`8pVSPt74ezHo)F3SE-2(V)$HbxB*B;V>5QAY0Ofz(~N>0?cZe z+Xf{}(6({zA+|k%m2!+BZgYAq#aSq-_a3#3_M7hqIq2bCc+hhSa*&rD&2yYHE6#5l zo#uUgy}rNwNXb2HHCNm;(WyaOi`tN+>q}8W;LG)yFvyF>5asdR2GVVN#PUJNKHuB_ zNesRmGitHxy$RY}gnsFU^W=LyJfwXAsv_w5+ zp}rS#+B4w6A7%q(&3mBw$qA5v#;6gtiF~6U;{Us4o;m>H92wJXZH&}8_!^qMu;Y2< z+$Vt^w|?3f_F5u?=eQyp-#^-%&CGqJK~hoGEsm1!mD z+}9z;q}t~MB(^c-4LsT(NCspkGB@8UJr1F)`5 zoMSX{e4EbEpxPE%v+grI75E9f$trt-+iq{Jt>p_MMv)nJTf)DsdHh}OU0xg^*;mxp5_m#kBkDX+x||@} zyN^7Q5to~LHVGp-Zmnw7iM^I)8}*U{HBZ0=v$_vYJmL(#&4R>VVN*Af)4Mb;`W` z3`4(Uci-Rr0NB_#ooyOh9Wf1rBu(ZgzuT0jusB->ccqd?e4izG-CVB7K7}A(BWyvR z1H9SC{UmRe+_pZB!ms$1e9)umAE-@E%bm1rg-3w#cHm#Pmp*+UziIQdMePSmWAOas z=tHjJ-zr2xlpC4N632g*`NQB5x7OA}zc3yb^t<)1jnh)?`9v(14@+O*Y$77=tc;GZ zg&g2V|9z@6CQ@(fYbU_d*o+AmJ@BJMj&#hKjNN9*JEXWb+MSPtyBLzJ1$HYS^bkdZ zmt-70Q1R(&{RdZo=2gpyUHkO=JNCnY8+g0iQ&Ydgv!9J!G_#tuf7LC-Etjz$S(odV-$9im*#a}KC1$MTHGI?@ zjsV$sP!7@%`mAx`uat&urjC`<4p84cMz}83I7us8V*-SyZ5nbsPExS?v4vV|--m9W z>XsDi;hsW#GbFHv@bdE6L2Z{CL-t1vS=FIBW`?79+rqY~FH3_V|2vhqG3ac)@dJe+Hw-4av;vMXaE`L7TQ%paW$0@bOW~`2Ltnmx8XE zdX$BHs`{Fh6^A-Kv>B9Ox>0KbFi%wUp4K^Md*OcU`1Iooc!iDPNR|;9wnDo4myk2{7FXLFryAa20)3|_5 zA#;#=^iqPfv4wmKo;orl`8*EW6GPV?aP!RC5MGQv9wo`zV78`)EH#tXwdR^7d|4l1 z-c!xQaY$ZJgHQFo)qh_{i=7duUBRnPt5lKvmbbIVQWOM|oTo|&@b&gsn?7pE|9X$^ z(%NW3pJjG7wOx@u&Ch)kR6JhmH26?A=MxbPxOl_#_d#xvg$Z#`j1X@gZRCOrW0*8@P3duT+(Ks zgCe^FS`6gOL|CC0oynrW#spU&LKZPj`qUM8Mag!Czn`aFCnM|LDC;FwKO6r9YxqH; zVAbotQFZR&O#XlTN3FwDR8Bc|#^hAar-@3boM&=OD#tlA=fi}g5-LfN^Lf~aIm}@p zRyoXR=6sA{3^|O=_}zWJzuzCff4DB!#(mv(zu&Ld^Ywf_-b%jV{{x5uj8%m6)BkFC z*@_r_$%4rW0a$91Xm#(RQM6qvmpr`3<~Ui)(m+OFH|c7%4=ykta`YzS5F#zvkg@U` zdWF*s5OV%?EL&I2yJ}1BZvGo{E@Q|4Yhpt9^?)+Z81|Wt`)rTx)uyS8WbDJC zfBu|*Ln`2#MqClqxJnrdbxpPpjmQdC%D^^m%h)jMKvP7ZJ1mj;thDA^M_-Knr%^2b zuNO>fTg{4U`91irY%Glb*VXYjJ3P66f229O`>Ab@pMbt*$aG-ce*yMPePApNyfv+D z)Mlle`l&%rCK|&0TSSb>aPGo)0t<8Hziv72zbZtxpgG$P*Hif<^}nA1%Ibet=u?pX zU(X2q))xbm=DqqR_r>;NRP|y+y(>F*%6)c=KwcBneg>}zFL$b0|MYEn=(Mr?PR*L) z7511sVlgPORMZWccY_Ytri{Hy>YObog{zt-0f@9Z`Obf+tN&QRET@92CT?}w`z_0)BF7kuY z!2)9J2UzL;z6|Qby6!f!a{Y32lVjhfQh)dyPLGeiclC%dt|2~)uPdWp!)RYLMTTJfEfwpW5#CD22RL?#zOsI| zO&r+fpL4s{4#&iI2_7Ufg~I1lxwjtrBWo0Dsw9GlevIo*jn_~=vz*o-4gcvU3)CWxKY7KAutiE-;hNGH7$Hi#T*m- z-nZtrRCIm2crtKG|L$+M1KHhH)Kx~0A|JD>j_qX=<*(|1$1F@VR{7adn%^wuLp0Jp?(znq;_VwRhsaGWmZM8?x(x*DZ;39by|Eq!mccM3odDOZssZ_ zi{lzfi~+4tJ0=Hnq`^JlhSXPwvzXRMj86@+X6LWIlk?7RRU|!OuViguaNWs~0IBkO zyB+4wCtQqIl~xky%tF7JS^sN0mmYT}%$Mfg*Oev;yL|Y1jWIS%z6R<;t_k=){{Ms? zwyw;`odelne#d-IOq_At>OR^u5f*h!SM`5O3UFZN3jyx9OOroLj@WuV>b)Ol@A!u4xBQC+ZZALHHxv7SuG7A$Fbd#-cG*(tp~&&`kW? zKU^Zj>Dl2bT--N{xqq#b)+`3i^e87YNRQU@eE7Bed8H(MvdYJe!Es@ZCfZu)NO2Wk z??I#HhIBDvPm^}=wem}CZ<4(JG+4%k01Ts?YZG!Z)z)Ouamgd$Q5ok-!SOr&22YmV z96XM2QKLF^23@P3#`qb6ajeZ(ZPj53enC-NiU1~4i7EnZ4|ibb^DRR0JsiH~|9O+= zw`|*A1>#(UH}xC$(6QE0My9vN`uUciR|0HcUkAQ4cit)G?s2u*>`V)4SuUkocs-6z zc9ZO}bd9LhOy=sVGShskx5bFGlD!x>bkn@$(+Q*A(Auy!Vw&`>jz|<{T0?r3TuU9( zkvUDjm2FSjVc0UVeC(b{%;wgX&NUpsM?UyMV0PuO41ZzNuO6|ED)VdmOFt=UtacX6 zlsK`64po~9e@3@;@}GV_AI32z_@B&&Z;-mErE_fmO*9c4i+h#Mbpw}T^>@2hA+#U0 zNTLCu&V;KC&sy9jdJIN}tgYzvzZ@s*DA;P(_NoE#dIZ1RXqC#;<~@Zve>>Nt!J8z< z1?u33F9y*|;0`~8+fLSack7(Y@}MhUSFzy(?L_j-(2C@8TP~wt@5TuFFTUJIKi_cs z#X#%9G(G-i)OOlfV=&_c2mSyoF4;!?L?-RrzuHr`x;pepzsC8}=qR+6-QYqn*=WDm z{jaqVA5E;piPj=}j0*qhb`1T}k6N^(F@6kM{N$#!<}qnpw81&ImG_nNnVK;ovF-;x zC^@Y0Zq;^-7Kn{(>czwP>Of0;V41h+*I2(5ay`Y&y&Z5XjL1yFpx*1L!6+ZMj9iI} zg%bYbTd?bXkxpnH0=Kf2#dslemsNqM7vxh!6B8sg`=!L{(#MlXRVum&jS%|wTfTZ{L`>LW~EDhs%1vY04OQF;02oxpz18|vPY}#%p_Luba~xWpv@PKMb2QZbEaV~IPYI?T8Byn zu*@(QmaTo`3v}*z#+tl}ykXa#tDnu9k}#%N-hq_}Ovg$X`<57}{IRzQ?RpwS=rA!X z=GjCI=x9_0gB*|+Dkc65`j3|n8&4!unZ~|SLtsPr3C5n#&X&9PYl#Ag{R&)DNNNSP zmYTguyFLV3_4QQEn9#Vgh+HCErhnOKhv$Tp_zUdq_}))}imlh5t3{T~?1mzq74wWy zy^hBPLs+c&o&6pB;?~TecYwhQ(9=DQBl5VQplItKMx1{>ogIW7DT-dU$SUr(N^ie1 znvbd)HbO{Ljy(~y$e&liol_n9%0KC+=ZJws zfhg&*bzG=>`{xV!Nw zdwq6`Jm}|vScw-yc&RX|LcsCvZT#WRP8mXHUb2VbGrrfWQV%p>Sl+JQ9aAkQIyGMk z3?a$wXfS1O*gken`fZ~eGHuVca&tTjY5uNgwF`DjC+%o6AhCi~si6muPLkyEacbGsGYM~#s@y_ccn9`&PyX*fYGPs>*bsN zOu);0a6tgohjTUZ6So^#IDCtW--1?*)TwJKo_2w}7&E<|;523fs-p{;mD>LTSNSdR z=(LO5P_7jQC>iWkl!g=qiT0@CEyLKJbK3h4e?n#D)Gf9N;*u#Sy=qR8LA0B#c|Fnf=kTEeP9>&4p=C!BI-9k#X_eIejxo3mn z&?7MZr426!kry5CKOP0TK1g0edkJoS95(V6YI#bm)FmY)k9x^o$HuD6a;3q{-lz$S8g?LrRJ=x}DSvhVefoQejOVcURSQ95mXt zuNA~xNA3<`Z1&!my?Ps1#QmL<%Qe{)9YIL*@nxV{G0V|w%=>aB)C z*f~t=G5#%Llv7b#gFbauPa5|*@x7GJwb;z%mgGlC9~`N< z7CxmA{oxMvWI+1LA!@Bw)%O$;2eX^`Bl6PU<&s4bx#Ov&zb0;-E8>oWJt%=VhKI+I zJMWlnmPP7x#?AHeoaYxGSG$USWT{|pcmxBSQJPvY@v|$otN6D?XI>jKCcoEO*hM0` zO(Q;@%C)xoxvAl+CPcUZLju~}p?XEr`C@GQQ7bXA3X@iZ&HR`CuiK=YJ<*r+u$+)t z$QS-AG=0*?xvjkWv+#&mdPR7`*siperOR%GUlXH=HMDHt%=IvC^ZjNm(z|PNgX%zv zMLo|+NLC3*h&~$e|Bvo`uMsa2#w%sE4R%y2C@KM+Ws7UqI*hpYVy5u$$P6^$eOMUt z~%0dt43-+Un( zN(#9Uvfoz~w)A$MC06Z-545J$7P9w3NFYme&4$dK zFu7w5v!$aBH27z~_QEpcP4_`fFOLnpo-I_@mul(~yz{r2x>hWCcW9;UURlXnZJ9Zi z-wif}e#y4v@6!KbIpJC=;424;-?LQ%@9{k%2-P!Ys{cA9KXU2~sc1Nlf`gVwvF?Yn zgG_&>@(l>!+m!GsVs)eH=H4a0$G5}e+r}KUHPu6_*>_HPoqD-nB;EVsa_nX$}>JbeX5~xTm~UuSe+Wc!fwi_XeLZKfMltrab1HH9$*2jDOnG%q=2b|UPiQ?2_{3g#ewNHzx9wK!{2J}{o zsY(SOy202V3n=l>pmct*Ob9mG(Er*>`?eyZCjNeh^9WFf^;`BfKfCh=Mqt|RkGmyg z4G^XBztSL7<|VNGNaMQ<>__#ZBUZMd*9WaHitna}Syrk||K%!`9CjM-LG>TD1cQvg z-}%19(KAF|N_+QV$7CLAN^TF9Yl7FNWg)xkT4s~Z$C>=0l&4M6KBY4u&@h6?$jat^Jt99Behc6v`3BE{9d1N*>5qw7$ zVyK;9v*B*CvdbQ4|IiS#BfipkYOoR4h8Z~2sz>JGu6%3ajn=8xxHU+s`mRi~&3@B= zCPvb0X0F14MSxN;%0QRl2QH)J+*)bL@5Ng?Jw8V57QBQi>FrS{kRdqV3klmi@YyZ8 zfwXGw4d^l-AvuXmD}Z}c5$MHlS+*r#f8Sam4`M)hIrV-M0X23m_;i4J{MRA>c$?7s z*Mn2yZbFh0(ME|iNO*`&ALc%}R-`&VX0I>~^<E+Au zR?^7{`8dt-=N*chwI9RSA!e?c#{=k<>wF;-h#WLRdh-jq_0co5g23i!agn#wzfO?} zBm>y2>j!%HZVI~`9sMfOr8=X|`FKNn2t72fM|YUCg*N@}L|K0RDa-uD zWY&1I$j z%erirv1ugOk{d*9#k4dK!UCq2{R6qDwAt@r(Kask8HUhfCUv0wVPD|b!R5P1tN@?& zy%nC+y&fonqoj{@~#f_ai1TMg<6okJWUN? z5fcRMk@;c~Txt#ZymW5D9gK_L$g4I}sggc+@b23s$|9qL$8`P-_Ki_Wf_JDiBlle` z(ICRTH|PhBv?>^iNBc)}}=ZEoULpp#V zC{t`d6hZfwc-(9N$`OIeu(#2^+phbSn^}`Egru!|3h9iHYK|ANHg} zc|30u^s0d3yv(raQx&YHkVjS{A3b-NNL|TU?+`s*s+fs%2}vVGKVF-}f9^Z+noB*} zRU=);7Zv9^^ur_19p};2k-$yi>tApWt9fW2kv^yIm{hWl)GZGQ!({H=t=kRq9`=k&~ptW_t$F|QR|s)+RR(+MCM7R_%`jc0sGRjJ&d z;XT_L>TQy3?`DmJKoK#hx4v3#!|EX?a6;5``t(9V4k;S_*p44j9~K#D1V@xKwlPh* z^L~>9b(v=7PpCeuBGqTs*zr^X?=H*xUeyY=223?QN*KrbB%7(HR=T8!w4gsa7@7V(3|8LCkzY+|R;=m2@>yb7vEyzaij-Pm%{s=7;t$RE{ec*St&w|jD)xI{=<%LTYr1~)bke~I!KYob@IJI( z?0gJQQulf(YpD$#QptPY;HLgqm%7e{D9{ly96X+~xHj_&HxOe5p{+(y_qW^wri{ zlw6wglY9eu^~>1|gREpt((B_pEa@NjhmD;EglrbGtH67xXItlLBMQ3K$8x}M5`J+$ z_pA|r_f^Qb8{{A(`Ex(|{n`>r-PPnzH$!21w4Cbd>wKg5 zO~c2?3xdCyz-Pb4qoXqS6K+B{Yyd#B#365`?q9pgC}8hm0?|1iv(rXS%}G`OIa@gMG4_8O70>y`1v8KW)w{0HC zU%;3qB9lIuznp{1*D>1dKGP&YTsPoZnn8s)&FGJEN^#Z2-q+J3@ora`mr=2il^iD2 zU_bM}pV6Ic4&9{2wUE-<*ACTn2GR6_VvJlh?nUZK;EFwnh{?NsX&Hj{Tba)Ps31rW z84RytZ;7BTedB%!*pS$Iz&xtk&h=#F&4ke>fDf>Eay`_>dqQ@K27ZsNV@`XQD>-*9 z6?^nC1reh~l0x5q}SPyyo~A|7OM$W8!`& z`f+j_h;S_1I^3Y|Wiky7vCR7xT=}xeu8dtF zB7;3*Z$C-da~Sy=61RN6DI-!F1v-v6Z|@(A+I;}pRYxIh+??`UV)E} zJv<^xhe?eRplV?Uv*JUls7`XZG~vX+yM=0%`VI$!XGvRxP|1C5ZEOk;%)L4Q7Ay=e zwd<`AOD(?Xo+AN%3%!`)CHSUp#gVOW6ZQ)Oc3{ySbg#Mbzc`@loN3}cT0|VNV`x2W zhnH5h;Y@#Lf&b8=7#lbn1;o;PP7dSci7LZd6EiAJPRaP6ocXWzYg%rLkJsDgT;{7m z?R=bRDVGg(xpN2hhu(b-(i>BL!muZ z@($v0Ezvo_oe1Q3`wlS{ZdNQzS;n$0g3<(5A);i1Qxx$V5^}ln{`GnmikO5-Cw{Q2 zU4}bFCFkS1Oo>qn)+~Ny0f^HtX{_0q?z2E_3w*g7o5CoS zB|`Y-;QFNr6FpR!+=fOd=Zz!rb?Td2vIWn)?_Pc`Y@f^Jr<*^FL|!Q~+olR=iWIAy zIc$)W4=(w1|5)U7cxS1CUu^s9DpTb$o#I}Lq;@c~E4}DC2<0|;NT2MS zA#cA(L;|(K)~yY-Tk*JzQ8)~B^(#wFF?hERsu5&b06oV{gB164rBUQgCWv(PA6GI zvNhOP9-*-$~2R8ndHXpk*%IlKlmpYN+HzjP5uu7-)* z{0X@_>W`$;z%yfvai1>|zm(kft0wSDv!Kac6G zt)meUZS=MRo;cNo(3Y4Ju42x+R%^6=s(Jqf#%*Oc zk6E>l>4h)qq^d|0;{_MYA?ldZSj$CW- z>OTBw8xl5GZ{C9KV?L5+k3~+-Bm?84?jaybAqrN3oh*J1=P$q0RK~r%3wZ#05BalM zd@GDeW~@a$JKzd;EMcUa66i8NTi9(Sk@qrcZLdkEn#0yWjyWsva9!|q)K zrQ@j^p{@MSTETMdLs8aYr@qaF&3^+a^JN>C^tO26GoCBv-Dt+B@|$?O_?ItHQ$g(S z1Dy-?BhJwdw4iRvOr$&Eni}fmgnsyV*260cpvokS%7OFWECI?jrsB55Xd|P&qg}3%Ke;&J=IDv7% zE;+yC0*lZ}-qQhR@hO!ut@$(-X4bBKb1cwvf_hlg1o8lrC_1rar!jw#tCY2=6n97riv) z&9`SD9x|oRE%Qp}ERctW#7FYwLXB=QGt|gn23PiU;j4>g38%Vq^eD3L-C2?Sl zkE2*a&FK0PkA3XkE|8z{sc|+WeW7>a%6+y;Q{O6{5wYr)E+u6_*NyU^&htA0%PqTiE``fP@NZn_i{y6rC3%j_G6fG2UT$uY(*klM zAC3y&T^6(j!)6x)F@E}AAy3$DLwdI4A|OJ3N~KiNQyNI1FfGZIi^Nnu;k!laxnR%e znYC3|da>F37vH6P*%7mqWP5lt7GutSVK?$NciC>_l^Xkb*2e)p1Wuvve2dp|7OPm| z4S-GI;PBc9V6VlAWNGgG$`9&zne`Zj|qv|2%7ven9uq2d?&un1m8**hO(?J3f^Gjcu9y zewy@+>v(i2wr(-IEG{zVp%1;3arscOPG>~qBLbI=e<(JkPs>JJT z(RzfXZ0^X(&Fuk-(!L{h{-Z>H!I%Ui80ZU>8{!M?T4fyl@b2nYl~14V``-;aNR+yr zWq%3wYL>q4`I^&8mS`fIQlaNq=$@L{?R7ng`Aa~vz@ly#{}E~FE}EnhGjeUH7W;`{148T!g%*V_l`n=~*)zPRByAB)YFs&3^d2r_Um;1i`N)^VULYv3U;e zu$zRtS$Wp+uWhxn3s&MHkhn{2aW|KD#*w?VySFX!#Jt>lG;WP$;C>NL2S~UtU_vC0 zL^*BhS$et1?j>8c0(Q&pGr;-SE(Yjh>8U%i{ef6ulf%nIqrf7G)`~9>l9A7Qqu;l~ zZ^q5>XI?g~5%#Qpg!4NHFV|QoK7DF}g3ASAqUVG8&bljVZcf9#pIJ!|p$5ykOTgq= z5sz(J?BXO#^IDQey3KNIJr&hqPm7ZQ0rR8bR3+753Sm_`dIHvmm()l|=cnX9I)?yI ze7b@_lCLrfKu#bs+_uKmpPHYDMaOaDP5?+C&7h5Hk zQpy4!*4oN1&@f#DXXw|kXk2n7tE7B~9W^6Y=i)Xl>SE6~p1)auvB+9PMvny>$pIV9 zs{5wZzoiEPD3<_~on29?CV-6-h%NSa2d+><22=_(E0P|vWgD+Sj{ElNA=BGsx;e&Gc{ogiv|9(c>uZ0E@-5HQ%5eaG7a(6D$eaQ-@}zw&mVL{sQcQ zD$cA+*M%_Uq`H}x-n(u#yj$n(Yc-Dj+{9bomzv+eB7=T&dByAf%4I7TAdU|x--7+N z_h0KTlPHVFJ`7!~O>FUjo5Ac}8$@Mp##CUx1py{c9d36D;t261n>Sm|k1#7U@iPRa z1Sf~0HuO)_X^WV-UvJ!Qenpav6*$?QjS|~Az>{fPAmQTg?Nw2rbCvbbd4a`l?VeoO z((gbC?5dJ-D&g*M>WLFNk%W}wgX?<$z>73L>Fkmdxi4;q>-2c?B^;Ed{jx3d5u?`1ucH|{}}kCxWgR99_9**^d4j@jIO{AzCDjP#GeY2bh{pRqmR zkqrVq{|}m~G=)1zG#y-#aBu>wuJH;-5Ij?x2ay`iTm0&zNK6pPLFV&0U7Q0p8~^pD4-R&VDekm!kq0dz_n1F+R5Zvy=V%>{58YjWH2+ zo|djv-}L*c*FE5sR`%mOyjuds_ogP;^LzgttD{>7ksMtryt!&P8!VoTN0^mabt!(h zk>&EmRrk!WMk8>3;W1o#>Qc}LeOFN1jFGmEhdJlcar;rxuuQfh<3IQyP$6@mZL57^~k?0V;95cFbYx>AykIR_p$3^Jasv&XE{i-$*t0?q4CQ2=4wsK#p+ik&$rV~nu@&M)^WR;Xs+F*IWpIQ_SnI=`Zcu zDIs^$oV$-#mt1ZHTzx;o%J^No>Mxj1O9=zV`E`0O?V+q|N@Y)Ub;QmsEhIdnUPQ>X zZidGv7*Mh%Es1dsG2uL;wu|F& zarsNHUTAB&G>08G;4-jys4HLLbof>{3XwQ8XZP~z_ImY4{%TH+G2SdiuxVboUUjmH z!MPlXMhKsU%XX<_4~yQ=e(T1Ry_p{dNX~^1tBUW{K9_fe2FNWJzZp{#Qx)e}v;sn& zIx<$M%D3U|(n^+tOYlQ=y`thflztu2^)$nD(mv6WAd@jlVLW0<{rrz*OYroop=5L5(=X)TqPHY1k3?QomGpZ zOy&_84d`oo`16q7=Tbl7R&sRBnliWymS+GXSS5Z^*&ameQpSf@#&uQCCGO6&72on( zy0l`znbkbOzdVjE@q(^X;(cpZr&dSk>42 z_t+kYzng`Btwt8q<*Px5o_7Iop0vnf9Sf5UH$4}1HZbjK?xdd*rCIxqG1WXDE=QSk zV!QHo?t}mE4ZEiE|Mq`5X$6&%x)D(*(N&eaU~mH%t$CZS^5ngol(-M5$iXB+L5i1Z zRr!Hzn(24j z;ceGO7g;NBQ_fQAfvdOed<{5|Uo=BaK_nNyCT(jxVtkGjuWdTNe0UA-34_o+r)Uz; zsjP4fWe)KBeo6kjOw=T?p-tn1Im;k*+M zAb|kMQ^%o9XLBvUWDPkHCt?Y45%Y)WMYDdu&w){h~bWE!3Q`-x$J*e#tbQ@@?ya4Ni!uD>S>EL1? zBX(|K?OL`XRIM?LDt%u+|Eh+x&|C=6=qw#)K54ELd-4(sjv-t-kDlI;er&N_sEgGE z`FW2f(m@~VOVFdhjV?VbXOC_ldWMDh%Y_VyGX^WSs7W2^bnjM@jaFJdu? zQk#2uK9t3Y7`L~w3!+`Cs9)5$;teKb$G-4|o+(VcIK`5>p5YU|_eQqaa?$UTMnYZ( zBE^hAs*H1=ea|PXDIC>2{lxj^dhVO`%9&5=0s6bbcOOV8=!EZ4Tp4ordl~vgEUtMC zxrXh?)%Ab#m5^(lEuZoc{`>l|$J4kOhmwZrxN}l5Ht+7iN{qvVVCfJmUt+52Wo_N% zHxbWhTFd5vb&pmG7U0ugYkWRxh*MN9iU)?jTewyV0`vc^)AzYQg(YwbA(+qJkL!7=k2Z@MW4t@TFHDI{1$q)OkIJ3SJb_Z9w zbq5Gx`{kjQZ4rms4pG0Vs{}ZJ3;=J;erOV%I!_rRC-(CM@%=6=pz+w>-6sdY^xUBP zws%;y0{rp8UyZtUA$7rLWJLKolOnI~D?6BF!0pTf8(*aV+*?yR%uL!3e3$o|n9FNA z;>z0AmajBECFV?~WV3IvF9&&87t$m83?*zn)EUKd=~F$u8dd}=JV%N_Go;|I_z1nqJdK(FLZ#Yyi7l&F|N!;4MEch{Idhl#EvWEMeW)OKEYh8y&9CRoyKcB+vyRzY7U_#^WOITq(i zk&-Y2!C*OnB^xr4xOW27y=nVzn&OG&iC5-{iwi&hb?HToYT?IXEA4vY1=cqA2Uo`` zvlcyy4V;RuRK8{4ll9nI^PqZmN9#`L1@IEBYe8OMzRld@x-J7{{p1@x%^jBI&*yeC z%kU8{Cs1j2Vhf+ZG_G z3%$kbVD~~HxBKf|uAq;>K7YJHg#{M!`ujZ1!mP#IU{Kfgh2yT--(S?b7H8}PJ4**> zamIN{Drv?2<$0#srtXI5TOO^{o>@qZHLF+ zh0U+aB1!R|R0&B+#xMR7e? zWAV^P!U^=)t^12-N__CoCo=9Lu;Rb&oWlxyF!x@9XBN8!HFKI;i9r(}$#BHcHk3&a z?8mIVmoVdA5&4$2ODVD_65osJ;LzZSoB>OFo8Bxf&G)6KRC&`6-*7AyIDVyVjH2*Xlu!0IyzqEr zeCkF?jr!{eWCK;tY3#s$y3Epvm7V9iZ1hyky^t5Vy4DPcPIx@pvYo9{FL#XsPUaUC zOeLBX;_XRq(RF0jw(`0L5kt7|<42y^iPjb=^5F`C?rmbpBoBI&W8UfIaN{Ck-Av-CM4qJXs@o zIAGy4w+9df^k7zF@7H(Hu=qiMkD&dTQqYkh1Eb+NbPyQdPP_y#WAH$F)}#FzE7@;l zyxh`aDkRfA0VyVrJwFg~im0^`FXAyOEjb9p4Fr#~?$)aAU9+Aa=Ww;f6*c3$j>lRH zZbx7QZkS6^V-n=FEXXceyXHT)$8rocldUgp15xpxZBboadSNJKPs;Bpp8GvG_YSZP zY{z|@GHByQQieUP!Z)2O?33or;%&LIAV$(8Ghe*-QYwSG$PF$!| zf1F_UC-uwG#1E*valoVa9C$ZZ-8Q=qMLBqGII7@glf9POr`*L`fa~T6_GM-Hf^yt) zrGCj}vy3FwZ0h;FlA4grNk!B$_!$2Nn@Y)czDcsdq2v9GH`#eXhF}xFgb$6X#cN?! zvT{rPYBQw6?=wGvp@pUTk$UN4H<8)y`9)Q2S2{K1;(KO3F|6W38KJHk)F&ZbV04U9P|5 z@65{!+5MUtRLYB}o`1j7L_O!IRvS9IxT*{)1fOB^mStsy)3)N=_aCpX1f7+9ZlO@b zwsoPku%yob7sS#SAcotl<2%r0LXD|@A_w(MMWi@&Ys+}HGWNX#+)T{_mQ+KfQslHl4cxH zvX!dj6RHOPhopbNiWfuruI_a<(eAb^k7@VTBxo*(OsQvZb2-PsX5tSMJwwv4RxY-S z4Qm-al_3ZCa&+%6AF{<-!|iCy!8cq_3}E?kCFWQs_JdMhzFz6a<5iMOjiN<0lpMwW zJCw4kJRmK5bZ?nb?aXHaxlNAAQGg*dZKZ5522s4*u?H&F&t!yyvL$|?Don5oVF41h z+(&^g8PmDZbXyM0a%BQncz}*wJE@)}W_eEH6zGp?!W;S*NC10X%0;dKcHQVUNCm9Bm3U#fCTY65JnrGE8-5)#lALg;qHC)#SUZvYvPS&vzLB$oE-ul?xhlSne zt)lCzYFB9&9u;Rn6v)cCn_x0Zqrj%)Zukis>BktwWlFd5-gdpkZ`l6JWm%Q{3{spq zxoGXVkLho0b)Cu?)T+E^I{$X96Oja$osBQG?!`2LQd^uzhFJ=B8=|h`n;!GxAI*w-w;w4OW&xM3R5<_J=+1BG#M+9b8q`&w3 z{F`|S-K*<1%c_k0mAgnOTnxA^)V9dn$O7bx;JZ4Oj#TIA`Nvb?1&0 z?6!9A?gDdE9mi%j;oNjachwzql$rcZW{|SG4Z!M{;_LO?EcD$1O!SbKeQSY~#qJJ% zR{#V>$$%whMr3rqR<|v+z0XUdEnTl=K@@C(WD7m38y3sywi|VIS=9GFdJl0osggV0 zu8>Z^s0VGZ|FA7MOUeryqVz^jAAZbW4dMT=KRq6P@%7rp0Er%L6PG22SP`3EZ5fqD zHwUi|djE&GH;rd&3)_d&_NX>mI+$v9dV1PYbTG7LTGbgPsIjW8A!1I9K`BnF=%j;r zs3}1x2|}berLCHWkPt&!Q4ujk5RrFB&-1+h-~a3T;oTqDKkeRo?Y-8z*S+rhy05Dn zlTzGM_j>jLC<}1?;#Jlmi&gJk(Hi2J^+H7PL-JH2llXLAv37vOd)n7Pyl+--+`%li ztd}%#%oHhns&=RtvFm7TZopj3=K85xo${S#yznN}SIA_^!kM{Rd?^zqGDYswseQBg z$zjrC_;{#?&5q1A$k#Wre{CckYx{MC+q|?SWg6sug3Yy@gsWwHaCrIar+#v8(0bl! z+j*VhdTjMNm%I&Sh`qTTb@6V{k!Xy~HIC9gR*0l%*I+vZAUG$q3;?tW0_upPECcLe z%TFP{%;-}4Qk*4FEnn7J9+EXmnX#jakG}<=P>^A1vDd96--h=2?=?8f&{I(nNtw&) z5%pN-ODCD4eO>5J%jPEUxE)+yT=nr=C#2tb^8x5Yaw7@84~zDP8H0Nxy||%Y=5|ud z&ym#L@&<)UgS7Zm;kiWby@xXUi6JQ4l<}ik=FL^)#Pg|JtG`a4A6lLiu1`Nsiv94; zdrUUh20U(4U;1roNN5o8Hz!v@C@>zEQ2Hr17WzX4$vz!}t8)f1&jW;th3MYDOJYBE;<1o*0!>S@_*RmRR`;Bfy zn7h_K@KOJ+QA5lqNT3c)YS*@k6@Y^GyPGHPYyI~FNCT{IC;aH%WE&kNCqayl?e=>+ zab4GAm}TEQwf<_k`_-@hGKBEr#BDVhn^Lr-hO1F@!0$ zyN;!wn_Jv8T?ud~N<}k-&|96lu6eF})~(TV_|xn-F*`uR|Y)hx%Y97P{< zDv^&54}ObB)^&476dk^+PTmRXE=46drv`^OSG>G%)nm`ItQ}@i&xd-W{dRg9?ccQ_ z{ODKC?QuWy$8yiYvKFg54Tv~wyvW1BDPzS#OQorZqhos1V-sx~lJCPwcoGk6?^Y*?bqf&(o-ZO5LxUH;#Rrwl@jH-TeIW<*G)o zA1HVq`wv^$KOnPhs({4uj$ef$AgsVCibrigmIK3acw_CRx_1^PH;=(iqrS#J+F-AA z>RlJ>qIPZ$d}P~oK%JClrmKhl45%=>;=Uhs*-@(1@t_)jW#0QsWD#y&u>s*UW_lDJ ztzQ+1ZxVwW%=5G;Wq*%ylDe#Y2NyJ`PfXp}WZUuad&?DyoM!hUhoOxe!B$6H)FLN2|lUpKeW!5%()U2+o{|{>vsWb3pfKgR^G# z+|CHuSf-zdRAhgI3}%C!qKBMXL(^=fH8bnuGg+pAtN z`r<#nYpW%(VyUi&QwPmkyzySd&xWFLHfgF;E@QK4iz?UqHL8DyG4nAY#60yz*$sGF z1-a=pRJx`jbGX?LJIC1jsM%1bAJs6vA z17+~p%}-_~FHpNA#0{b@NA=`c_6Z%GO{2URU6**NkYV|?`1CcFL8!x06sjqiRm^~= zrFQ>o>SVXy>BRTzOL3(9?vSiZqn!E3NdzR_0wS*Zg&SoW^HVYrZT28Y%jfH+yZ6#* zw2IWbL}X6tm*{Z=RlJn^U@fH>JY>fBIG(t8W_>)7)O1hlXOrv8Y}cz?!&l2WPqsw+ zIOMUtTn0;-$;gHv#n#d~)gy=4MF!pY9P&dKVY{C3^~=HK#Sh5MbwtPDWEp@ajtL%| z2Yq{Mb7u}^51`%i8#zVUQtU3Z8l}vd-OuP2-$eeAINt;S+I;}3p)&DV78(>z4qHb? z2%zzmfuyXXvX`b^-o}3IP{%8=<*qI-%mH3AZ@@c$IIRDJP=R`|TB4)01yf}}<-V`a z4t*;n+1(v`gS3Fn+BKP=GvG?CEdYV}tnDYn;O(C$>(qn&zWqA7KliNfK^$>Yth&SQ zzR?O+CAnjN4CHf}*|^EUSH__G~a&npggSvuCw`CcKU{(F42OH8j6ME zwc|ta^z>cM8R6)Spg`E-c*5L)^o~k`$pOu5gui&r%~N?A0cXP1gFC^4Y+kO);0F&g zO7I(h3*@W8hnH!N2-^+LBvi{+iWW_rS%;5ngcpsDA5A|O|8IayMiYN)^2f|NDbXcJ zguZT6*NgoNDK=9#AJ(++kB=5&vxz`yHrm&_&MxyVn|DCsn_L7fhU5vh zT6L#0MD~hctrxtU`eQ3?Xf}i1k@``oqvml??{tFJ9xV)*avvaBQr@Sa!9$DW+luZr z8Xepj3U(aRuQRsI!Fpx@kOQDEshYDl^}VJZSDLjta}1@sFl2pCY6r7^Za!HgUcyHz zLzS&y>&a{@1OLRW9(IxbW8K^CjMvn!$!g2lMetaLrWa?oG1F)BJHYex&qUpR@qbVB zSl&gG0t?bS*Q*Pg*0kWDXI8a)E=E}vN0S8HV*MG*!Igi6J ze+fgqxb=*NbCVmI8sG#4X+<_<&`=jqHDNzc3&|VTN&T)aJ)NVWK&_3B)ymR<-Lf&+ zxN)5K6`O4*w48pNQ>JnkbNR%K%0k1p#yYxM8S3Vmp^d#mxuUkX^)Jr*nK1pIRWrIp z`#La78rmG)ifNOyuN5AFrCgkjj%{am{QDStj#W@@1SEM?2c#<%?~kg1oBRkeIk3$l zvx;^5R25E?0+6XYPEA87|r1g_IFfl5%`mz-^c%Yxr<^NR9NW>`;&Z)zK5$OOtZId5j!yD z?zqsO5j3BiviSYuTWeNwHsIv}0LUKsf&J2v_mbcydVa`W-NQout>=g;72YGA$82m- z={>G>NpeLR2ipMePPqywumzN3}_(N2&zmyu6(YD%ZDnTwu!05xLO>i|-q0!k}2i~b2Y2XuWvQ`zrYh}>SDfW}(afM;1^nOpwm=msoaB>N;&M?%172NRH_0PIS- zMzQX5`vE8qXl7vG@Yi&&os@;<-8S+d6KQba!B#@&6<*EX*lAz$d zLAI9^eRtd}y?^kg;LjjSIl(2@tt@ZY1C%YmlVF3gg5QEe;Ws`2I5QUAKF!?;ATF9K z8aD1hhogGR*+wHknr0_72_VZ8rmaD!_Q2&mgx_@D5^GB* z+bL{tiY!X8NF`TZZ+hl*jlAB&`wWKe_`um%niu^Ru0uWDp%16NM-eD@quQ@HT)Y?! z42)0IxX+8gtzvE!d3D&z0tFbSU%NEj!(xS{9vjupNkE+%n%L~0s&K17S zDD`3pyMWgN8n9QZV(e~F>aEaEftZQg?7GIXrJq~VNjWtDNA$jk9o_Gd4_u5U zfB>LEQ^@MSd&9VBu`22^H56(Dwe$k~i>iK@5m{Vq2iHO>q3X>5D#XDR!e77^wTaL1CGPuva1~Koh4*pGeTic z`)L6B=F@E9C%V;GKC|U*5&&ev;uaMYS1$-ct&>&T95@-1tzw-N3-<;9KiGnhnI{)# z>3oek&s71YYvoOm79T>>Z-*v7Y8w{+Q_j8j=yL0Gdpg8%AE9V{pndHe@qNGc?+>=k zz~~uwS}8>+Gbnb!Mxqf6r&GgI2cMeSO39#)e`J`H1p$al=(#*(%bDHnR{o6ui4PFl z)o}Eh4Xi-A9WXW@?679mX&?*OaCz4qRVf(b(d2%$HXR;?w7YrHy4*ZV)`=t_-fyJv zwU2uOsa|7l-ZnL2&u*|84N?`ZRVp*!GZk?qUD>nfU8=xIL5v@e+5i}%2xuPEH`qSI?Z^0oumlX3DC-EY)zE+`{h34 zmo>(DIQB5rut}56D=I1nUNdIZT)>zFs%aiMyj?-|oU<_uu#zpMp|9&+E`&5!knIUE zG$1%~0~>atIO|e`L(`w>8Iu`AYUp%hBF9>A4^&ac}! zxH?w-=^*Dz4to`dgM4!ILMNCYqP$5j{R4`=U+AX?@IwJoBLIt zu^MR!L!@lvR@!Otux`5@Adn3} zC-dKF3;+Z9|9WZwbRuxfVP#S_KPGRs!ExPN!j8@h(eil})G{xoN(3!>{C-YUV;cx0 z|DPg948h7dD{F($4&!w3VBH>_t6$s0t3K;q>qeTB_r5jqbfh{y?1&CGdw5Fz@cmsN z!D0#=pIm+sshgQde0?f&#LMcdpE;Uw@R>&{dp4+1o@*B(n|B5jeCoeNZuM+pR`?58 z7-nQPeDz~jq57A~2nT+1`beD&A#*y?BE z`uiOqUD55u1`q&w!4LdpFf_3C#6iw+z`Sl5IWS0~q=;oOE3JtSKrt!5$1u>q{1ARz zIm^U7_Aag(Tk?hUv3)M+mB(Y{Ky!7nFepOxe;RkFIFhBz{n08vbZlUdGQ`dy)4RD< zRh9~%8e%pumVg6Kh-;7PL_`i#X2S>I?{`vl8E(QU`h($Yy&eRN-?QiWj$EE)!1gNt z$>r{x4&RSb4NZj)HMKoaG`ZDCYqenhOAPz8CwWIM@;~{Fo=*AYaiZT%D8e4;oO(Kn zc}dhJpA-l9%CGx33v!1&)c2l&MDSoe&DZ*?FM-o z{Woxe-CZ*mEpRX%u43$$@*V=#k~+y}`t1Yt816~g?zEYrRlT|HGP>c#@(Oc5#;ij+ zV{XBSc>2q8xSI2QxDjZk<$wR@*4z}Igzz1QjhQ-U!JLV$P5#ZyA#6wSj$G)B*Y;P* zr5uTzieLRzB#8$4ZLH9=wxWqDs=xFHgD|d)3u170gG%I8{(H3V?Cjh!pXXN=W>F2- zfa7?r%+;SPgZrQuvcP)+j@*nRTI`3EC8}uJj6?Y!Bf(wf1P(qljpL#6}!WyRnh5|V7FEm_mpKyMozHkH7TFxfCH!v7e4#V+j zNe0Xmztx&IBLD#DZy^SMpR_7ESw+%92qt%Hc6j{f)?#ly|Sw&72k2?^r z)#N3Aa)asU;gQ00bKPrb+eq?p)Nkc2 z5(lF_DcW-$YHqre)jq0=!FQMNh7p%_uA%mim<^@O6qN+*9;yi12uq3T7k~UskJ+DP z1rNHuFm1x+)SPRZhuTFWWA-1BHWTe^rqteGz=T=Ej;&wso!!_0Sa2#M0`)8`ni4YM z&keRN*a1Q%ZaWn~A6342^N>;%(NNCYG?~l~7|iuw!(!D|G2>#Uv`1_=1B@Ycyphu8 zR5jE^L>3_HV5$2JDie{u{+I>Yj52{4!<%Im)*{HmszW3EwP@}1A-eUIvV}GBSAtJp z0363Yhn@{bpDYkP%g9WfHn8gPpIe{~tu_1VtwopZ6(#k_D0F8H zwHuBsK4H%fR;$nk%$<=Ml#MM9#PHYt{UA$&|C!EKooZ@BbqsfI3BAecy}!iEVHX1j zpXHQ?-1!kgYo(oJ&gPz!h?UoT)jco9&g=kf?~;H~V=OiB^3DOjk$1c1s*5bj9JGCWtMrHj zKWXVC)A4|Fq>-yfIaue2rHgld4sKj=So5XNOUmly*JiOfj8<9+{PT;%d8oS>1*XfLfB3Q(1D3-dq%^=e{4RINQIAPT4M|L5oP3@Q-)dg%+UUf2(2u`Y znN)d3FxSvqco#5KZ4tKvL{9n7V5%3@7hd?{#h5c{>LI0vKs-$JtyUxY=-~;Es*)bD z*5Ce3sSOE?Txd69){6O%d3`M{EHRG7&(yV6)#RatAFNl>$ z)+@L0WB+|?9?+dg9H{`p=No^LNQEm#70{|;yE#t--|oA_0+Ww?O>-I4Yjya2SX zi$k>MakX1D9aKMW(TlX(CquUG+f~CxdPg_NC6fdgCtGQ*oZ;2bvJe^pA=}Q4E|$tK zx_U?hj|l%Q$2)U7-Eno;qhZ#@9XeQ}_a%?R%rYpFtQCon7Itw3=IMhWw;{jr@Q9zx z?zs)u$k*YS-_@<;{fi>Z08b;MmIc_c%UV9dD`+vUyt&#-2?al_OS3kcz{Q?8BUO@H zS$bhlykpCOaH63>EYh*Xoiq1he_;r7ugh{i3y)U-#>Jb74-vo$$BRRf4=C7$xd25d zmwqdjf$;A(l((4hfW-WaS@ein$2m zoAAmj!fGGj_tvf#n2S|Mhrv7;VKN&-6sx$1J1iw4v%jHhD7sPaX?QCiGqMynK1$jO z8D8$w|FTWD$~p-!$#YmVt5BRC_% z!v=)US&%X1KX0w)^FW5=8z^oOgY@CV6DQvcm*BWWZrE)MC%r6t6 z@dFRYwk1n-O-<&#RnSjdMeUv~UK0x#In4ieQy6rPhuJI(l-!9`em^^I_ELU!WN6|Lp^B-w}7$ zJA@9vtpcekC#XG1Sknk-cRtEJa-fE{W)@ujm;U!PTKPq9g*bjMe^9G;*NqSj)}U-r}-`OZe+=#`r4_ z8AQJdvIt`WlGjctPkb4N$5$?ePKJn8 z^I02iZtTVA3f=&9yoQn>I`_a_boK!ucKIcvsPH#2x-1rL-%2I6I3{urM5sYodOH7GpJ|?>Y_hGB_vVBk?+w*7$A}L0_4#1t zBfDK>!Yi2@KE;rZl&nteN9(yEZr{h{pmBye4tx{R|@5VmoPLL(vOS zxE0ndG1H$G`v7O)_m&QSCccfP{=~3DRqVZ1laGYe-N?;#lRXA19QMzo@6_<2xsrfh z?T_x(Kj}NZ4IcWVn*SijZFSn@p<)H)VB}aUD$nl#NL60}{xcEx7cdad2alKZ>6{;%zOU*!YxFMBm2ecCTD}j? z9%>9qVv4pbWgBp?oVP|-J@lF<22+8J8P`#ZUE)`DRBWXERzpCN^BPTSb?eQT8ON${@w#8sz2i>2 zl(Cghm-WK!HS}g*8abpYSoyhQ33Uv;IA-0^jo~6@L`zKlDY!vgWhk0`!84{jd`++BoLH?U9bF321lw++NpIgEG{um#&q4c=xz^<1kExOYTMc} z-#6=K?z~|h!KhHu7e$ce8;v-Q!jv=PVf~jyM~a_S&Eb&3`>Mq3PMrb5)4whI%u6Q` zFl6&uo~1pdbaoiV!1bRQh33OPxcRT(n-GKFwh&tZJRZsbVN_-dQW!5PXT3#dUI41% zlaIm9G>4wf(JWr%{;W93G~7s%exqy4muZ-4Ni>)ZPsf|=i=tno3>AmedAexDPX-(~ zz?lnruI0!7v|DRM`}#k{85ysRLJ9g)6X9H^0}A{zds1~^P+Wm}SnDmhk#@wqiq^Dm z6_u+!9KG4^DhtX*{9X-$PlDMgMjG{YRbLv7L>S=7g8c9f7=PZuT|C3S}#0?(T z8u|;b%{)xJT>Hh19F_GVTt262St-I5vdTjGsKE!FsV7kS6}42J7%q!qxF`0Y8*i;k{1O>G8M(Pk ze4>(|`a)dJHc6IC=Q~Dup1SI8cVbom$-e7+th^wA+n!8x^-`R^P1jctW#~%Gw7I@L zZOji8_h~r8C8@-j(^+(4k&;1^1*873DQ)Nsvhxte#QiG%aJ1p13PGKPr&YSa=MUv3p+#gvGQs(cR!X}eYNo5*UfDYse`Lrw-&JiUK>pCe(`Eh}O@f0L)_ zVt2Ca%tlaTk5xUb=fWn#B7<}KwSrVKq&>;Cb|=*6Zph$(VTd0O>1-Bgm*?{D(Fle6 zJnV#1hp+1KcWUgTK5+*#&C}XD8$ulF2XOhL3g6W_8(?E1sW!oeAmgy#QLo(Vmi{G5 zT5}M9fbet{l#tsU4qJFP!?=b@aJFzKWw&s?wXT;dil2Hz-Y`+FP?UkX%J*BDp+$~q ziDIiQOUeT36q*k0Zhas0Hw2>U19#_K{87Xy`%@+;?*c2r7*WAn{FgqSV;zDCXLlNn zC;I(Vu%Og~jH15R)RQ((+6-N2truz9>f?VA8ynkNJZ-l)Ust^`vG+LMXhFQ8m}db`qCth_R+!x)QyW1g0TOtN^oF6Qix&CrfTp;St=$84 z-6FjlA|``haam1ObYAPzZe-nODkR?DIcxBev;xVI#S;Y_PfncC>PLr}k%s0io}yrL zg8a`k-ER3d)obOCdTNifJur+0mpefZ$|SkM%1)o98k%5A0fmWzD0YR2;}`>f%jgdX z`(x})!aTK`;kWq)dv&;tmh_&+IcZ1-qj<)Sq(U)55%?jpMCH8U_@sVg7QqmSFsX>{ z{9MtVQ@y}ouak~0qL?3OsI&EbWe^c_|O2ScVGJDc)FRZkqkvyhgq) z#k!s!B#|Bzo!4+4vc5HMZiN9+P&jCwOG z0Ijq-zGWAcOTKgR5}bLo1vSyU&J+)ntWF{nFW5=o^OXi+jGZ>b#Kj4#D)-N1&DX7B zC9hC%ysb}Onh2xDp)2C}GEr`7A46r)D5%WD&7kYqWQXCS!MP7ij5s z^f~wX!Y)11l<^MqR#|1Y*abaV(ybPuTyVVvO2rtG^fdH=Un5G}<gd-zykWA zuf4sQex^Pz#KSl7_EZ17MlH2fqPc<(MH^>M5iTUcha!k3xFAZK(jW5>Dz!P3E5CMu zvXCeYY?ObK5oD18T{s_roYP()5FSHtl-3WQtRnyQtl=;4ISF-JGz(VVxMb-&Q`!R& zA+Nh(Dl7~h(I!1_|E$C?7xvT_O`Kq9yN=*bfT2%tdD7u^04 zP!ZeriKGV{>viGwqUBR?}DAB zf>;NS2O*#tH_=MBQ1j)3V#3sLb(!mZi(~c@GaT{xPv7>hEh*{gM$v#<3wj-upo(u8+kuAGaOOQvhpfSyh6LO*JGY?H!4~z*6b;0%i^R{L&#)aRboCiTE}?-{yh`;jD60=k#afu%DWV7P zfpHq~x4!RJgP`jj3XHMZ<{`q;X+ietv>?@huXKP+Al`hV78Sv4VHI{CO}#+Fl-Gy< zL0l><4_Aiv8ef;bZpYFXeLoUS-G))iUc3Iwlc6T?Xbn+{N{LbBDc8lKP5{{ju$2aMA%Y&d zK6v%_tv!y$Gu;4loA~bcD7hx1;USEQtr$(QtEo>$W+Ty6kR5LAMtwrAEEcy0l?qpO z6(B^n47Zc9;qyNL7>O3-plkS|5CVCcTCg&yOXdenw z7cq3Ql+B`3Co}FUyNlvT6(GSp}i|K4&?v@2$ zCg&xi8$p$|o~gECut^Q^nL43eR?Y^q5qE+*e7WCnyXI)Ah98aQ#wmttbF>A5F8T(o z=xmf@X+@VqdIpetgf%Nh*k8Y%1RGx~f;nwvEWm+bAi(70COc`k2)*Ta#7=q9Z+_H^ zBS}QzE$vpYJ6)xe#`zO3E0^Fv4cTM2`8w+qG0LScWTm-*#o$x%1@prI6^^GtcFND# z{+TsiK5abo!P>^=#OQ_t&3gP^mIXa4=g6y+@@4ORTjyJ84|hUG4mD!@&->1myl^`J z?KtJ(ngv~}RCZe{Q}`Z!lRju(r7f*$+cXkBc8EA%LxaxOJkL&H+1(IRjPhh~fm$VT z7w*$Ec}l`A(DNGr9rlx6w8aWLL87+>K33K3?_udGWwujzE+`lIvwFc|NHV%UG<^pM zFD4jhWMdK1$=8t2^Qny}=lDvnPPFtN{|ayn7_jXQlw?g1^UyTe!0lZbm|WVtXj59| z4^afCNxW=B$BOi>ZnYr5?m(*lqgEZ9)*JT5u<8?_>}Yz+U#?h5o7&VOREKh~JyumP z3(azsAaEiiNI`K-xOzNUT}~soVgX;S&ccr^LiQUNNg^0)4ySu^962tV3P9dfQLT-x z@ddy^=-wji0L47@n$#6~4e;y!MTZl#m)j_WUAYBld(s<=!;T3!2|xlyYC!5*_L<1eYw_={A|oJL=ud$F>+kY7z#2=1l5LfJ zhd1()({Vnk>u|zQPi0QpMFXMo2Rqutj6^~#@Ed$)uzMZiJSf;l}t}+JY z3JlkLfhOt2dk`55gOI?w&`hdb&-)*M#2f6K$C$#_`rH#53_!}X_ACP+7P?O{_&!7n z0Ip1*J}`K}+p5#?OQDnnwjE)US4 z%hCZVS7_*^JIi?r9o<4f`83xJJE_NR@W^IE!CM=z3xWckyM#&s>z7_BW7mu^9A$t{ z_e-V<$VXcjAd|(w3Ws@|=b5Pz3Au@$u!Z1s5ss~-z*202r`)^@&B?!PA!74=Ka9$( zZaL%nrnGIS4z3m-e)O$ryB&I;Jb&W`q*gB~?*cF}K%mzZ+Zw?@`oHGA>0p)n<%As% z1?4CTpTJ2HjiPdax7V<^RLLI4tUBc8+W!4qc&Rt}_VfptcDr|@&OlzSJo*eB$|(bF zM zNSJI*c+Aq8Bj#*hh1cFTR?Q!t6jR`KuA>hwLWJ4?3ifTWUmQ@#sm*NsTF%<9sC_<^ zS+Rg5=+KJ<`;;N6Qh_w#Ii>dv3nM6}t z`_<`YwkVMLZG#I6lQA_-#WA7(CjmI%$s}S@gTppMgpdwwR=C8qa(PfF?SnK zAXIwIA#d?zd(KBiq2_>0Q}_7K;~8+rE=E5QdDFhMG4iuF`dDm}SQz&&nChUy%U5!eZ1VL=uk__}w_0^kFR zs=6Q@8gF6*8o&_-3{t?K3Dj?Ac zhQU|t;d9iWps3Y<`wz1oP1}*Crfn2;%_Dma_~dINMo|{QtQulUK~;%SP#!Z&KyyE$ z?d7u4@lTwi*?qwD*vlbubszm+5nTJE)x3K2RPr9KFN>bk!QeK@WyT3 zV6iR@0PQNCVlL4(Yj^5;6xT+2A0u5xR_6~_qpWl7x^Jo0pR4o$>Stfi8Io?dp;~V7 zX?uDUVx(h|`|9r$F;^1SfV-ftITd*@4ApKwD z9#Rz_g#RMMzt44|ocIHB1r^Vl4&PB5uE1~y3jbpRwocLs0}R9HKxrAMqtPiMgBtI4 z1p{(&^ObQXn$q4k*8y6?gYla{{ry>1_6>0CnxHB%v)-l-dB^^VejZ#MzR$U#!V93~ zBgvfLE(hBEeIrV9Pi`wOj;^X8$2bDe@#Y9{k088^U{>JO;#<@YyeauJ^h0Ok*G&))X82|18Sbj)o?&Ef%jlRP~{4yUI6y`54CO+3& z_U*n#)|#-mOpq0WC3hMOLWQY zmLC$BxUn~6rIKSN1Co;uxPe89736&<)*#TQ@83UlggjA+4BkO|YhhrxnK|TZoc5K> z@0g4lvHTd|JgH)hJ>y?~z~+do*opR(ryJ`DO!2O6R%+R|sSngfU6;~T6X;$L2Vfk(Q_v!8rpm4&@mF-K^SgS_nLqX% zXn&?HbJ#Df`LFZWB(xnao2vB3Tw;0l{d>ap;kO801%cmyfwLVVFucCb`LXft(uyLE ziv|flG<#vXmd_)$NbZzWd22c>hOCp@oj!j#nuk+z#M_9?Rxvn__nC!8J%t;!A|y6W zE2CQkz+up3S-})*zinJ<+^v!%A;)>+Xk}yibJ+r8^TBtIWK^|(HFPxZeLK!S&@W55 zWaFpzBV4jYd8bomXNko}wX-B|)Kgny)R0J+&4|7eUS=Nxb_2X-SH2$A$&@*Wt0=jU zy|i42kicWS&*ej_5yu4&J)OQ?M38Pu_ERHJ?MhJi;g-p3C7bK#BK^x$sFjcwoE?tK z>MzQ+yko3!Uv2M3$e(&h{c5398?14{%V+eF@qnz+Jk=K(v zCrt&7l1>(^>OH;MuP?gB*GNjrvhyGl#V$l|=YxF+@Lnmky1*+IZjeHiyEa#bJ?k0* z+2O7i(8d(W2d~&-D;ojoO;KCVk-^mRjOcgdrh|;=9!6YG8ST`KiV&JL$BulbCSC1r zv9_{uvIi}dwG0^!)6_`!>%23xHjiI!yU&O0x6^c#Sf0Y{&*D1z)vLefnxB)r7+D%j ze2HAS@o)D*K_j2p3V`;y>G2ISK=cVjG#MS zFAU|o9aLxJaQ@@|*sqqZ6ZLPqR{@q9rGcq9nbsD}>zx~D454s#%Nygyxs*16<7alL zuK-e6YsssYLz`%a(B9Y!P59wyGv*EPV8(F3=c+EO`d%)3N8As!qCJ8)DA*5V-*RcC z&hfFdI!>7o7G*iM(x2y6MzuR1NqXwHfsl&B_yt^VT1qaW9Ge+kZd!uZF1ACF4RQV5 zyq88rs2An4Yx8=-QkB-K@ktequTkl~ugUHuQ}&NM?0IOY^?5xf*r{OWbn7ffIyQ&_ zXXjN6PMhQ*er1np-j5bq1d}uJCLHp@&THu6Oli?q+TNNmEozXX^V?M=m#cS)Dn@@x z1ZZb%3McODsBUR~F}Gj;5FNtU6JIck9QDNd9u0QJ8$wNbzi*W*L>RQsXNSj%@g;O? z^NEWrYN#@Mxjofja7Q4(2<;E*>1v8p;q*)px7;K^R^g;i9|MT5-MQ0qNWC%19EYbWFL5Y$>4O zN{OQPyUkUj6PIQ#^htq0t&g^Y`6VX2GFH>_fydA^P+UXi;=9NhsreQNuY#l(s$`HgLsG^p-#gHZdapNJ6dixb zKzz5Oh%kGj(_1eOW&?qyVK;o~{ZpY~Tu)h8M#$*G@MM6`qya>mfmY~LC)?duZ{6xz z>}IFfJspiZWroqpra5rhrgO;w%y$Sv`T2*Dk$NsK0v>l;93_e+Z;49Cd%RdCcfSMz z1}y9D<((ku2Ehxsx44Uy2O}{}^HN%eu!D05_P%x)t08=JQ~&G(-HRz6dY(#NP|jaf+_nf&TP8x>^gJ~01Z7`T)x|#%BtnPnp^Z> z+A$-^#Vl7R2;P*7-F|=RUuTVWs-+d0R;kAk@Plybx0Zf!qZpQn|E~ zf7n*2VQ533@F#qUQGt*@L6+DZwaM?rP;`C z)k4zK-EPOT&q@1G*q&vQDx%_bX+vmxKrYPNHJ+x6d;F-Zt+>7){;MHs2tIVQQ5Y!n z!wX+-p80U(C-Q6?il(B}rB6Rt2-7sjN1NYNPkiwcIuM{GthWXCKZUT!ad_iAf;zYpG)l_sF z%K4ZmLsK%o*H#{2@F?D?lcl$hdMb3)jgivqPQhmB)N7`0)Kfoo&6^B4oeYMgRhp72 z?>W$V48EX-v8QcWP=#t!3nnHALgjv)Kvo%t1)b(&46@{wS=W|xFMQK2g2*Q>`_f2+ z6ioNdAS5Y(#`}xf-ZtMJ=8KT1Z_E2eIju=@a_Gs%<_J~YAqpe*-wD=t{OfF#i`v9s zlU5v!bDe0e`$jB<^MLPiXH6|YA}Ta_!=Z|n(rtC;- z39qewbUE+J`k1W7`Df}V{rUB|eYJs|8#u0#iSjEUyF2B3#UJ0%^7?M7!7t`%t+JH^ znxime+jfZOk;JHrDSBa*AJ2a9{t?c_I@A@@+TGc(T?*H0Uw(NtDAOWb$#*cQ^tmLh z|FAR7rFy-Fxfykwo(eHyz(w%YrY@Zpn;R)~;kXb3vB^!NS~9#-TCwr*&$i2 z^VcH{$8Xa<_W`_4h3kXWS%bDMZSUDPGRo+8g1i(%AI~Yw{6FlyXH=8<+bxVb$QTqH z2Le*Xag+{@C`FnuU`0SkfCLf{5hPSYhkzgvN16>$dXbhwXrU7n1r((df}uo22@pbP zNq|t^8|VML&spdF@UHXqtS28>VdaM0`Q=xxYhQcsOM%JuZ1p@HX+`_Zge231doKc8 z+sPi?>R+##?&yGVIO;;K^J@?H3FY1IHrP41K>Cp-zab2xs0hwU@In}D$po6rZspr*t{qGEO?6$p2&0qqA>xjYQ}kT zWOxFdshF9fh_KdZB%QmB84qSAZ%;r9^&fE@qWsnPe%~t7^fm5k^?zAakHT-p{IR^v zdAseZIL0y$NA&AVzn4NHYmJP1Svow@ka68m;ZdpcxD1^S&(WyuXlci%sZZ6(QY}sN%MYdtM;0}d`SUr z80~~wSjou^>o>If9^M^U+;+@F%~8Q?$@2|4nh~6w#*;gXV*#ha?*LkLFQph~qC(fa zpp?Ho7By&Hw`QW7z8k!4U2SR{NGzK5#F~SE{%+l%<3PUO$UwBUS6y+N?E)$UkvldM zJLhPE@91rYg;$fs3Tf5%SWT{KWU#tP@m=f5&s!%kZFM&urczhF%$74zXT};uB3%5B zRD@a^DK$Jev*}3nE4Vvfp1RKVZ&^1`eB>S3QnKz;4%C~MZq$$lZwr?>mf8%4Qe=R?WXKQIpKD@opaAQO?` zpt5i1futeYWt)g^MjC#R(|C!gQzAb8KWQa^Yz{BpfSOB10@6nsR@!;y}oc?Wr zw!MJvh#$9(Btcr?Lic)rCJR9S6aW#i7RFP(elx;$fPT?XmcX`y&PuwLYb6xr`;wX$ zwkM0v@2^8vPdFS^y>;-uyXbjq6aLk*;Um4Lm@TOG@j18ek8VpU;j=B67(oJx9{Eu!DE>z<&L=Tl87-rn!KgH7O z1DW$7qpE!^8+5oTX%y34xWaE)7S-k8{Ou8g4Q!{dZBKX6B2b;DdQas*z;_*(S|7hT(VixlR zc-PvrUY{qCkNG({B4nz~)x+@5eI4`gz-IfFjAjK3lUo&@_0*q3WcqW6aLbUiJHORF zM98fw3&HNLfE1-4ye)QWI63~#zS9Pa9*MX>JQ0=VcLdSl#%UB$2t{jVy4=yVjmO=F zQ)%`5kDdu~C`?1#Iedf@P4NN1y=@eU#Jn`PbK^*@A6`L@|L29sF*IyXCc{LJ(Y9Ws zI*lhad0dhvo_56gYv61&+0`ex5n;#`^;UPQX;h;__0uV_&bUNH6P|s-7U>lry z142f%JbMTI9qG6L>o7i88X}G}lA@H$?e3dI&Bl`JN0$-Md*>_oSuk~m^GQEM z5L=M#c?z`GHD*5cCe!ZI!H*zMGG?dz=7DE0_nCM12?Bc}_Me_7TuMhF-#%L`el^P_ zX!iJy)URx5;NX8qrE7g}@ysue(~vKrI~YwC%F}RnQ5bSuvMud~h9h3G5t4Zkp4Zbu z0}Z1b6Mq3i8koQ1{2gkFESi?s%3X=nS@S~>_ORgK@VgwEH$TrS)gK;YGgT`raVmE> zBWJ`-_J#LUCEuXoxV%uG9uOS}IhP?TS2>$j(*m%4NU|(b#vnS`l#pQMvXjkDoWYUN2bQ8M+)go!Qi{X zCHVrm*}atMeU!V(&cRK0FFE>-dhfaQByepi^^>u&(s!HMFxQ;shNSg3t@x4JvMGlz z7y0-!2tzu1H3{eYC?Wv=91zVOmZuXgN)9v54&n$%4vuG#HTxJN2G$L`SQp$DOa`Umm29)-}1T zZy(+`gJ%f%*M+3|zKj6bn2!)0e-d>f~sIBsNK!)Jl~0I%`lsT|!^Bh0Zv(}pKq z8x@U4uf1<$`*AmWHr{mRh-CZgftmY+lA>34QTCecZ6gZDPXlU8z>Zcq?znN8zVB=A zsy!NfI>k3$AA8K-Jxoy8xA!(Gl&Ce^WK~EF+Y${R!DlwdIw4u0=TzA1OJR6V)6v`A zm&ud?Is#KR1(q_X6Hf_U3T<(JwULJR55Y$yr=H**fhyy)FGHH4cS_&bx{bULseX4( z_;D?%O$##AA?QMxl$pdoN@eXE2)*K#C-0!-7$Yj$7%V1S1X>C5+af>@g9hyL%UERo zin7Eo85M(N7z^cQlD6dyAH%in;XK6h1X5z(Hl(X(~%$aE#Z)Au(VxYB<+!` z-AW$M|kP=7W)~d2^HyhMgdaSYr)%TDY-6c z<0S>+M{Li5>?! zvp4epNseCNG_F{haKKT8XJ!zZ4T0EJezxg_CwtWezlRFH#C@(&z^7`KQ%G$mg{|G- z@!pemPrRY*)*m88hX+&eOW(QWkQsXMS!R_8w)Ho!bfLJM$fpPCDD(ou6e>jm|Gc(W7q3SASS36l7dnfwCz~ z`B9M_#?i*Gn}w6UYmtRtVUqo)A%C;*6P^jD*yb%)9YR`KLY89d{Q{YkQuc(62x(61n)^?Bxi25vrEXKBPNaSA(689v_|A+`#+ zWxQpU7~fHtgX`963{v7B0bnP0@NOkg-ugoR_GjlrLU!~*^A@-mfRQQU7l<)cX4YG9 z3>qTejQa?c&bD;hDOesWJsZEFi^s%A97Jgr^|ym!VP0o|HVgMEQ+B;yk`4>!*G=3V ze{8WxNbrU-#wxF;wTBYxKPI&2Ux$|-RE`6#qRRvCnpeG>gKD=Cddn$%PDG&%aBWXr zvEIT+S61@hj5~Sb!0TS2z-ia#w9CPM^J}$nlX7{@OBc*m($9i)vipDgI)HYu#h;I3 zK*te){-Dl%It<`;Ky3x+Edcx^b{p?ps{oxle*jgsWAnYalc`3)iaHT*^oM>2 z?>m>8hMrsdnaSyExKe;Ldz&HSUe5*+#I3;9#?LKLwHy_~8!9f?B7ddeMEu0+poX8z z^2WsXwd$vpjysb-K1zRfEp7QrzsLW^6g;xmcL6{!i^|c^BPSWT&E=;@<>+l8Tjvvs zP0{7`ed8_PEKNH8)E%EhX1?$tReCWXKTxM-s>^Mss>1geJb@XP##B-^dqu>z8+LLh ztSLMQqYl66l-x%19(^E~Su;pYF{$XJ7R`kDo{!X!WBJFVwOA3C9?-m)QJhfC^x8`H zoWAD-C46dd$w$tKHhZ`HqF_iSM(;XVPq_+7th_CMhO=JDmkozj^&nU2=94X@O#M26 z&8l;9iq=?bKjTM1FI=rtRA}bf?!?!b1*&uF$JSO?rW+`}BnL!{@qq}=l{PELCae@7 z)BEBp&>n~VRPD#trpVC(Z7;BJ=_JqK#a^m~==2E^(Ru=I`vrOd*EZ;M`I5{7NA6;(RF0;TvW0z}a;%+~#uXo3F6N)5f? zdf+65nJ5T%x!L%*1^X;`NNp^2po`q^WSy2DO}#U)_FdsOxk9v4;-jwK%Qm4Chl*la zc~jq6r_A}bt~x9JS26bH=W~Ge2}tLY>@(}FFSer)JLf=|K{?y$&Pv|4r?-|@dx!C7 zR+ci48tou=lr0*N<5HHLIgeQrH!Slpqw|3%2oH%|*@cSeHuJYr7&Kxzq!6RB>Z zOqE`hn{YkU>hRd;u@ zdnc7&&jX+(GFJ9MM-{L=jlX%*R7SH8#Lv;vY%K_Y(4qL41HhX1cE-0ID%)@*%ADKu za-R#vQVaUd)o!K)Rp4zLnRY+W(i9jfp~C8Kl_8gzs?u*jXAPd)4BlNmvF(OKLB)}_dfAUP=F~0kDfwB@jef&H03u8 zIIJ~23GJPqpoa%zWD?7fCY?;*X{WtHNr!$esmc1g^^HmgkkzGTG-c;BE0a%}>?+_3 zP`vRrP{KHY@U-?Qx7N?*1OBUnhE6eN;}>1Q=3sRy++)n`uun^Z0D&<=`wZVcUb8LI zLg7A8Ep#Y%;_O@Tj;(N+s#m_Z-ynJKpAZVH!o!4V>-rk^y@Vq9i>t}egY=Vsw&swG z(!@Gf8{^ag5I?PLOI^KD%|qx}a)#0}ooJlE zi*F{mI@wZzZeJmIUSKXte%2S*%jPaT5f4FqZr$?piZrseh}I37(2lQT4&jJXU+Z6a z9uU$#SD^>s_f|7G2JJD5BLcrkj5oFgbZhj<2FO3kn>gCIZp$cf^gSyv0~LLkSi$N( ze;ryu1j`osD^tgDTWtEX6b0w@hXkDVQ>@S+~ts6ofv0lx&apm$mu#LJ} z)LHq~oIb}XGEX9Ws{4FS+u!(e-E_+faKy{ou~oc8wFj7&NHPAhYL2A(ZQ(MRXJ#G@ z-myZu6W?lzim`kA@=ugPMm`AQ;yHX;qaxYd{Qy(RpPE`@a~l}jLRlclWSFGs{v7?RK@&TT$9jumpP0t{V41_rZgK6(6R@0;Kba3B zJ;|BI7=s9yE+#+&`(OmLHYvtAFphga3~*;Y*-fCCC>o$-)8TxHnH+r@C1j z_3-@Svgx2`gp;SJhuw%OQ(hhU-mK#ecK`K)uxrx5wwSs$d zM(YD1n4NK^8xVZ(IO@2=&{S8(VA$PRb5T~XVa2JKC}55^KA04DSD(@J)-eLM@~vG{ zr+X*Y+9#e?p!?(0#y3PF?SR4&gdE)077kk@4w3kOGFXmCH$`{hOkHO{1^6kmOLQ!G zf1{;nnk;!c7Qq+|@849zkY=>UTwZ=gjYXKD$-EqY!5gO)SiD9@XIq#lTK!hFaML57 zQL!pjO*;-pRwfAU1=aXW0I!KLbJqlU94%*fwSa*OuMpUNr%y$e)v9=B{?=M9C;_?Q zDTkc2yvXwX0-lGU?1nBW`%)7lv(HObOUU)sE?=haHJ&86$R9CR+e!y}kMg=gkVbfI zC#9L?QQ9w~6W{9WQeDZgzFH7cD!NGGG46*Ml*-<&oxIq)&T;~f13@7E$Ge&`yQmS? z*q-vE=PuTcWT&)-a+->!PBCjLjK&Zx$q*-X(7fbHR1Y_!jjwlXA?I9sU6oLuoahy5 zr=#R6wx;tjVum-hpE~gg)T!|r3YnXZ2{}szk*jMVsSFrVil>_yDO~Mj6y#3Em9y#8 zP8!TuA|LwuFdBHVhGgb;J{PE)%RY-RV;$`i`LmWtQl&JnI=2(g%*ivh|H{buQ|QgM z`EEGF8p?@)q8GZ&H1bncu!JbSk3fN3~jKoqj*+wD4l$GqkF5t&o#Wi~~}TTIMTh zpyr6!`8p*Ae0e^0uETSPhIemam+e_~S$xk8@5^~toXBd@AMT{)1ds8fksIaUL&TiI zex03je2^3TG_*;?JJY6jscqte#ol>!&|6IABV-yl+3qk=^)|*SU~FcMm)_57Iw&+^ zvRlIU+9QmIm0Gm~OWpv=w|F^CY^Uf}2%SQ&-HoC@(Tx&Y;}vBt(U`xz*>c zb}M6Dx5#8F^*$NbsgZEQ__5z)L4*tkWu3O|>s~db_BCg{Ze5hO&)&&aLY zL{=|c&-!h?muA)U&U^F~2;LyZ$mg0t3?F$2gJcc_&((*srIwW!haA+H$k-;YYp+%U z{O%}JmH!Ez5cgF~9V~Z2x=Dr?{b^wtTwnVMv=f{%>+rne1vh$xRsUs)dXwqJ-5dOF zT^-PDi0snF^W3|aeAUGQSPJvOZyXEetgsoNi;@v@4BXStu+MnJh8>2r;yqbLiQ>-G z+;F{}m>}sP`O@G)2R=qE4+^XcjPk8%Ud&{J7W=FUfLfZ;v7vvInU#4$A9l`T|l?+>E%;p z9z?t*bu83M8B59#8ShTd|LvKS$=!a#=AIYv7K5n6fVt+c5Otu2Lz)1OrHr;gE# z18nke;r&K~fx$ZjTHtWK>`|l!?kTMS=qZ`sQT#%dICIs|88|)i(7RZ|y%d0Du}@l5 z;tDLdP`q)%-%9B9l;KMD4Jv`N$1u3!xb2?wy>{4OZhWdi@m^=hjB3S=zT$gC$+wlt z^8j4?8QM^G?;6Pgbu^DJ@mlrQRcfC|dWUy@Aol||a zvY8-Rd2Yub)QB(#qYcl2LD5TO`J(V&b0zx?XSNEMNu+U@%UY#KFJjmjxJMY-i6ry(R7xY1XSap5t4ln%FbPOS5n%Ye0dwB(XaVf7XC z0Fm-{-MEeeVgLgB@L#hx`Kv}S$kBO0eUEaTOc>xtJpwzt*1lPQD36Y%#Apcj7#kWl z4EdSXkg!C$e~m~vrDQc{@=h%9R+d`3=m~&mUA<5VmJ*dxKSKtuR0}VhfwC3N(rT-WL0l&T8orC{AZv5nW{d}(S@0$S6?>*lILjdM! z+jYZ|)h%*veab*E0X=2H9(yC(!w#n^Y2SH(PKX+7*iO%ilLX{ZE=4M*N1MRjzn`VZ*N7bT!ID= z+-Ej|w$7i^7kVT8U!Hpp2={8`OS_yjv~O;>x&a(Oy0x+pbW!%kF`xX(y>1xk-yTeW zK5!e!Gc%OZp+>W}e*we}x1VTW-ah4DZz#X!yyH@~=Ml*8QE3#>b@kL1T`P(RK5seE zu$q%xlCjp<#wv9P>w%$ddMlUhJE8;DnfFn>R||uh3-~IO)=3ICMa>vGG4cq3CQd;j z89l!v;ZX}=vVw*W!8*%LHd^?@&@eeWX+og#L5I>T2NfdVK{vSNASR@3xvN+M5J;h| zf5Lki>Dyz?{ONp-b!)T4iLwHlEo*nWL+8k5>!9QdirLa~{+^PQ>K||CO|T}M2Lqv7 z2V&~|`=O%ri|jr2kfW)P&Kr`q#xfz)^tq2RB5rhO424vw8bSYz;Wa3+Ao{897OoaJ!mXsS#syPL<$8GVW1E;r!#DoMgcu~pwS|}h+Rk~9uUg{$7CZH0iRM!QSv=-SNQ+Fb(#7W; zWtbpM;W#Q-bx@k;qsDfKPeDx*r!A)jqP)avTNCtcs_GqG?(SRtn1X)qJQ(uVhz zc<|e}`KavtO2?~l^%im2)%@a{q6uH*RPuapZ4!$;O_fOG*AJQSxAkq;AQkY)q?b7C z6b`yYP=)+6`p>QYu{*|5MNu8Rkc9$YeZsOaU?`8}09m}3UN3L*O27A5gz{<+(CWkF z&WY#LmDl2$ZKN+(uPS~Y;n+mUpHt-8>yDm&1f2mmaJyaZfg?&uqD@8Oky?_5G;{>d zUy1vnJyw-h4g2%;-qRmp@oL71v(Og8V^uXBv0gW3(j6qj^4r&WZ0EOo)ylGo!nCE+ ziSYq4b_hsa|Jpn6FO%R}j}J}zW%1nhA@@gw|voeoY<$Ma*h6Kur zP%TF7b&hMT)`okZcz#IAAxX+-D3Y3!->mSV+YbP64gYm^HHbG;z}^HUOMnFkf=CH` zzkS4u7o%!3M#5HsMMYQ=b+5zu+fW<$bGn_QYbjXFNvXK}E#D)$wFaZ$dy1kM5Ucw2 zc#Uf|*{33!l3_02>g9ohJcm`}c5bX}m{tpmG9dDHeVlWlh8Gy~^;#inDvl6sMLwhU zO=He5sDO#+wI4}oi?hc+t)tCvs43q|ZEca#Luk4L4Mt~hYXQ>Bs;^gzo32)69($rjR**Fu7t-mw1a9;}A9r(|OY ziN$95>}eqvE<)CxiM)%(IQ?kZ5K2hvCT_fOi&;!mq1Z=gQcawC?~I%*d=lNBCT=7{ z7!eUl^80xxn~Ho{@t$qM-gGPS*rec2tBJ;)wYhU=kLMX9x3l{(PJKo9R7pk;$W^69 z)EERw`#KV=?Y-x(?##IB8o!hXf5UahCYM0DxBoEml7;YJK4|cWRrR8lIJi|#!(xj# zl`SRPOtpgr#40*pODrc4-(08avP0LdDH~CYy5}S6u|LMwb&|Bxdcfoo$Gw-F)3V6d z+HO_c1$}qvIDpP@iLDxawqEaL+#mR=gA51ll8O%#scIS;w;o9ih&uTX-;Q&~jz$O8 z$7llN1BoSlqM1FdaA}nL9dpQG5$CSvJhpuQhNDknBNZ1H&EXeu666pq86##rPf8bR zBBAVU7Ym#p8OJ3a+yTYoD=QG_9Nq1J3)RhYJ-1G*JE)^!4V8F9p~Yf z8lQ*1#Z+o*lj=GH3craPuA&R6^MM|isML83KEUs!oZ>8HlsW^8TFkVwARZoBX!1XF zBucrUq1H8gn(<;GHiq4~8!z-v?aIGrw1=zSn!Q(&>}-{BeymykUMB*xvZIN;3i6QD zcKX#rLo?{0@8C83qnC8ktYDv6xl=hzXGZJ66sJ8e%q)^Go4_Tjk}vuQpcMhYUXP~m zdt4(y-MJct^%I8N_Y@wU z6cGs*)tw84D%yK}-oF2&^~#RsQM}oHp(L08EVCOn#gA@^{*%y*M92|fdL`Pkug+hc zd}?+QZv&Qh$iImfEWsR$0wB}|khgE8CWNB;tKAxIDL$zzaX%j)}v4`eL*Lmiq+f$b1q`j zE%vR$I;%oNooH^Ye{65|9$H~skjA7jAt;R7oK zmbi&>h5JLY^+!122msEC=|yd-C>j9YlaMcaLaXH~?-bm}TH+F$lT!j}Qr8oe+L65u zE}x}Kdg-1KIkz2=>cKzOj3B$)^py)wORVaexmS7*KFR1+2c6+wz`{aGH==4Pu(B83 zwZ0M}Nv!@eMf}xk?TMb#>&uuD17H{Zw?l-fmEg>^&jWK;n8#sGn9-TF>V;UYv0t$@ zX>V_cqg>C%9bLGu;Q)O$v=HIMpQy81PajD56K9|)10*B=IoyPjZq|+99cQ;LD_@5UNdIaN*lgugGym)Q-=%|@x&Z}$w5XKxtM!Rp z{hpNx=HsFHt^2Yn&r)PL0CUP?nY3?gmf0iQkN)(Hq!Dl)U;wccqQ;cqVX4;|5^xIMEBe$##7Dj#(PfDJ?3Ft%)%bkia)_s@F z`!eeXFh)M27cGBq7Zqk10v2MI=v6BI^RC7vo=0zv*r_3N4VQdExxsZOI8ih2hT0)* zKcI7wKZ23%5d&q&V~@vrC1h9=_%^-wMgT8O$nw-rLY0ni97`e0@W8LFUq>OzruM>f zmu%9|MAI9Rq>-3MV}Afw3bB7u$7-Idrb-_9DW&FL+46H}*wq0E07=vuCv-gA{dR>c z0F)7Y$;k+DK2ViTph#6yYIcUZQ;Xx~*GA}wP8&6>dPr7)T=*#T3~TeuRybP^AXqWF z8#dd9&Y?;56@P8Nj1XqEERJo|sW!h`nw1N83-6E2^Ic7ApZ=6mk}H+g*~H5keUM}_ zX8bzxNKZoTCmfU(65&6*?s;oM)Oa^s6aqYvccK0wdlz~gJ6Ms2HS(3z_S;;7?Qntt zEjW!!A3LGPUY}1I`W8}~2Lt$}(WAbAZWFF_%kTW`@8%tOTTf>jXrMmulZ2OjyH#Smaz@xE=+jChG09AKlJ-%aPRi!(FK6DbV$Esp{$(R zQO?%kUH-jOfZDY%=`+|3JMw;<+ww^i|Lv5OOztm5MaORJDKWDYj+*>-r}xz81o@jK z^0N2j>>WD4-hG@(o4DZ=c0CqXn~na0{gJm?gmbs?irQ#EUdhH!1kT7v$N+S5!Y=(x zb+r6S+;#a9Wo!=U@2RO7LtGy<(xBC4@M)$)JzzXwWyV5n@j&j9eNt#n~gan)dI(~qT}J8v_dWk@#4VPZ zLSoK80rL?^+H0VLyhPV7eC%~-*K*_D3l;+c;t`7CR4+?zsLUkZcNT5~$w&zk4*|Tw zxzxuZH>^&Yj%M zg;wjUdv9?UyGJ`=aGy&6)hq`W!zO%Z;BG=*!g}|jK-TWwp-P=)M&=!QQBCo8?(ex3 zdc=~~vb^!Q;>M$RTH4eJ8quA<7XC7)q5JjJfQGI;l>i0TO+>jA;q>GZrmsxmJJN$S z64E(u1A!?4xHMc}Vw_EPlE_MZ{^iGGXQdEu%L3fy1WgKo9yZ#YFxrN@ki6+96=g}$ z2#rgK=apaKm)L*Sr0z8%d)av5g!*yT5Ep#c2;o?rg6=K#H!t*aei~M^e#9_T@hIC? z*X~%@#qT+&vHGFjjD(o_+3TS@Ms@4CR1rNg#-U6V4*9szW#kurzDb0Am_m14HvaY4 zv0bk$9?*#rn6Yr|6D3X1Vs)m&nyJ4uMSOo*v$JSTw)QR{C^_u zj2&H#U4jfN!P)3L+e44c=~zZtb`V?qx|j~E(|zXE7hvI&RlyqXSahU8 zUQ=(kwFP(olX6z2;l**ukr45rtNLm|w1!7XB-mrbb2{qM#PI~)Tl4C)AHYor$#m|z zJ>_{w$i$##xoPcRlG-j{?F)yDPx*3&*&rat99ixfWIhX2mWKQ) zs|b%R?WolAa(COhC$%d$aR1U4ZeefKBPrFGY(37oEzCZt15UPGDSdR4%QRY?qS$fn++XE^gCeKijCY~*B zWNvJukdNKiz1>?sn;s1jnbl{aw2xMGIb*YnpoeF@`vK!B#mge-9owt(Qa5& zE6NK1nEyndCVtyaK^go29aQJ02Lj-9oEBbAb+^akbK-?==i>tSCa*1j;&wE2T&Nu-FnW`%!NxD+qJtEBn>#+1o=n%LLl#k;mF`&4?Xta2 zu|%`UMT4`a(3)ywg3Tg8{h@F1 zAA>(fvuBlPo};V{9m{Taa>y~6QPU&NT3Ipek0Zw(2_N1K9dwxgMb{USzJSxNo%;^k z<7PA}#?-6HuJv!%n{OUAE|BzzGC;)aMxN9Ew+@(h*~LYq;hA;L*A)E?8^D0?)PI@}di~L9HFx*fN%@8QmoSG8&Z7aL zYWXN-%Xf#PPDcn{Ng>DB#v^>i@@y;pN#)ShentG|=4NqBXkpsIV2oh-U})ckqMd(Q zqQ{{$#!jS&&>OG+D#m@CDHIR(%sulZ;f;Av)AyDs7bL?!>Wz%Z2T*ckgt60lIK0Yb zRHV}`C1`v+w{&yYbztwpK5?1N+}`s7ksBPv^s(yuas1$)^pL{w(nBe&K}$z~n{R61 zJb?q;cJ2)>o8Xjt1)J(5QKRYzwLl@br(xFlEy5cae2AKE$pnM4RriqyJD){9P4dvwIXWkr-)|Sr`HhQXKlxL*exnXR zt74m9{w4CyU#z}eVm`$t?F@7M)+|&3u*8wUX5Ja^)Up3!P|kYO7045ddJJzM3?CFx zSVO9s#4Nsz5j*gAFu|MC$s7{+w{S1gKr%>L{k|Fy`;isBeX5rV@R`Pd$kt)pZy<1LyhD)`j2T6QO1U(C*i)F?)?86iX*Q;(Q%j@f z+?*_Ul|}X6 z)DqpR&s-bQXTxNEsMS2HaOwPv8|azaV-(5zrwsZ#e?ssy(}-nPc4dJOuDAaEK-olp z$hJiy;1xeQ{rZEmlj2-`0=Wm(f_D#5sH~i&PkBzymV~Z}%iU~bj(-$6=!B_W+3?sW zVnpImU4bL z^)e&2|B~o?Y>su?+vY0}(Q}e(vn*oWM6gBn@Cx&C*hZOBex>rSkP4hH>~cP#ZD=hE zYJ~Y3%cY&3b=+ZpIumtRhJSy!7c}^;Yu?Z^*GOMdX0-l+yTadlYNn5?y`hFnD#xtj zj+P0pRn-oX+dKPMQb<13%r!P)rF)L0Q)>-|C>ey)g&f-CNj2t?2UYL11< zcYEasx@SpKe}~z8$WZ|xEcl+=x+^`n*FRIQJQT_5p40fnUa2vkFSX5@&7BXhD4o+G zWI#L?615hVk#6L&T3bOP)K*V+AY3vG|Cf^+^MGvtJW>3T)~gIqQm{I9c0sQqcjvMujrfp{ChJ8pemV;Q=dpC)0Gcec0D=Y`Y$&z#cnzMVO>12_BvJz-GR3_- z1F7?w!S=sNNj`Woe6nrtC#fCZI?`8iU>2IUbckz}qCybEM@xp3U+qCrhd^gp#CgaE z1qC~$#x{o?e?I53-g*547fx;DO153spS)DJ#jQJ&TC`}dggd~OWa9&%Y{k{Ho(ry5 zpI{U>FM2az-&mVFCd*g;T*EB2kNYi*JyV5zk$G4Ay|D_{6P&v07Hj6&?;Ow9BF?-5<|J=31y-cnpS+iXDFO|?=AeIF%_ zrcjO8zwA(w;f!E;8UNv6Z|BLSV58yTW0dHo7cg|wQkh^p>yuUxbs9RF(3w(4697`A z{+&qMyGC@&DU~ZLhf^<;B#*{Kk<2ql0f^P2+;`x^D9(DV2|_PK@WoLraZ!(cl9cxM zupDhP#0>E6_%*eRb~`)9a_&_n>QU}1@73l`bQthK<_z{%m;fQ0bS+LK!0`tz#((sj zBamzeciyV0E~TCBj8f-T@9ot-2nZ1pXT)&%!Rn&PBB`$yej|X!4FgTw*s@SbhtWC& z&hH#3>7Dl^fi;u!!i7uRQr7`@=Pr~iB`$m^05scY$Q)qXApFu^Fz1fBmq)TX+LE?M z=Xzr-4#@P%IWfRm+_vG*C*%t(RJkyvE) zr-sRewTxX~OQjx)4x5vX&?&%#mQ5VTPX(hwymAp|=iU!Mu z&N?QNaygA9(OtyIJet1=fBDNtha+9lGk=lLY(6F1hhGj45qR%Z*K*T(Kc*vS;oOb+ z6VCRfgb)$8AKt^#!R*jeZ{7JrDrHz9HRiQA$N?sA;jVJ?_FF)FHKAyfIUf_3899cH zzM$L9?fEWe@w}Y1>k!d#t2JGf#swIt*r*&iQz$4wW#|M(Xa)g}jak1EH)c{5k-F>t5X~fo9pu%0Yx@fFJzf%i!Q!sqEa?)OilozT(R{t(b!_QuVnA;jT1xg z%y=JkRDind(|b08-pg%Yeq46FeZ_6&sIi3lk$IJeN&Bp1>)P~0PfP`>J+8^4D(i^V z(HteUds@DkDSmsvtUx(}lUyWZ%CDwGa>@5EtVAG}8V9;}v&e?3({lzuD#Lwydcw~= z>zQ24)Z+#=gKIfAk^zU5s(!C!_18mw!&lB2q+fI=bR^U>yRQP=zl+I`3zhO$r|SCj zvd&cMLzTTpb6+G~R6OD}&+3}8=gb|tY}!&GM*%2xr$|O5)7q%iZ+-Yk5i9(7K8ks^DY3k5R|j7EIN0yb1O=YtsO)f@_VKX3ZY{q3vW zzOb_6=jfFltoCB&%u}-KUQIx>U)2rBYR& zTfX)BRg6&(B6^juqtK(@S6WYZXbHwU)q54gquw3eW_@z#4j>==JF(6~_*G!;kK>al zqh(yExM2mwpjdl^ZP3g+y;5EXp}x0xuknXi*RRV7C_~VF2jbYgLil${1~6+by`QY_ zF+%Z504Fx5T}d@CQ_sGhu@-P8t4#!o%NuW+UX6B0tghGYzdYjkc5>{KraD2UFoyS%H zq(AHIi?|-Xa+!blXpL}AzCX`X0h^`u@6(9w}cly(5E_Q+vLyqekHF|wa-_p_0^Y0li#16 zdTrLB%OH$jzv*v(8TUdsFfy(_movdgZ=2RzHJpBFZYRnt%h1=_gs8<=K;pOM9d0P1 zuVGrAzjF^sKJRr2F5YdWXR@!gx4eMNZc~MC$))g#V2vAk6`{G}DE78OcE8C+eQ8k# z+`o2utOLAurtvzq-R-J4*D}TA7JvusN(XGa+AhJpG$J!Qb~;ONk0ejx{<2ddaUhw$ zl!3sRW&0t)C}^1P{0P{4!J0Z&*;=Y77<*E`CAn7mY#!O_Nu9WmAWd}%w~9U{-vW#~+=H?9qF8<0S! z_5r_)PrN6wS{%prIg2k1{>d$eXzy5VNL{uNei2%{6DmCMr|fDtGLB3S3%bu+R#eIQf?Ay$&^v>q52o1fozeGKjGikt(;=oN> zLC)rKa+bkpcvo++@f^rM-vlfJw0vnc0x3Hb~K@Ieb|QfBYk_m zzlLnRBtsVx9q|pII&V}u-n_b@KCkBtXjX^uAg-e%-k??DY);g}d-bj<@0=%z65`31rRxobO51PT@WX_C4DkFM z-=|Be_oX*J3lZO)W*2YL)?%ffH-t;Gn#_VRVOZ#I5ekmXEp3%bt4=9(w3xc&Rp)JQx3 zEL0+4-s$UH`Of~(Ie3ZRv=#nesTHG1;whKvT~Fy*v4xT zp}bh7RY(TURZ=S%!2K2xYC`jaIpc4<;0ZDVE*Vg|NfLH(E2+$g{xHsMb#!3Zm5m2K z)GF#HiYIk`j5rDrk8f1Xg`DFKDvr~5C72Cb!1#b)mL+4m9@GW~Og{YNqezQekXNu{ zd7a*4DU5ZA9bsG&n@=)i-hLtYBI#afA?iNc*1kmWnuT;F8KtIpr+{XoD?Xzi!RmXnn(N?{EwY zKY(F*{Jg8Q@nHl1epmNA$v(xI9gUfG6J8JK$R5|j0@PfEZ{ z&KFP*J9407`*NHeo3@?^xd`tW1VCM2OqPiF({e8(X-ro2y!gb_?b!p#=PI%0 zrG*l^Xa+`tDfm!%X4zQFkX=KbGS-dv+|@$07^kB}*l9t`o)5>blmy zPG5Y#Q_1nZtQwQfLzk%A7u;G3&Rcn{EK5rLdoSoLCo^`PrWnw2~EXW6u!hRBKe;^0LIKG_P&1^VyAm<22RCBz*i?=dF zSk}{lfa3AgN1>Drc%yIx^lnHpw|j)lMi=OLx zhXq+fn;lg+%rQT>@K+Vd{?@yKL3b>?VoqQfFByQTh$_goNnsInQq z^Zy*uk%k4SykW}`kL=2Qb+P7;P=IyRXJac2`E+)&`een-ZG$@yqe@h|1YV6J5TEYX z9D@ZA++MV44l%Iw|Khp^6l7ul#E!ESylM(OC-}ULN%7qj*XI%(MZ3q3AyxVMvnjiN zMT@O@d~S{&huBSP(C4nvYK)?xa?YN|W;R0d97=q@bC~$=IG*R*oY|UL4MNt+;t)C! zh;*}?J8xMGgszLnmmT(E1LlrMwOqY{L3<~~?!!G(wj=E+QA$W+MI z5X?63)$GNAu>r~YOLbq9>g3nkZjKFnbT_@A;}#PZD6jpc?RLlE6)XByDDDO?x;Eqm z^)s7s{6%|as|V>q^(XDcnoWc$RyTosNhO3dy15iyR~wKLHatvbDJjagEE}h;Ar7T(T}LWI8J-hK!F$*;s(J zSxQOAp>!)p7VOt%c3k~ec~p<@e!BIh@5NDSgpHO|JHUD5jttKBV0~(IS_K*RVAIAsPXx#{J}DnCv&)&GoXsb14IiZa?Kt&Q~DLc>Vhw@ zLoU7dY3nm#9C1Q>)Y8b|H#HNj5+bYaMjF&xI{c^p}K`AzWZPftL@z;GTmc3LjTNw5@_67&j49Z^cXNR@A-(w?>}qdoET|aatbCq zwgwRxOlm7tkJto-uFIGIR1>G4dzgAV|5iuospkYPqzGWv<@c<>dxZadv!C64 z{)bf+fpA~U<7Kp$a=tsVPr(W8DH9uU%(DQYtfQ;`F)iep4=SVu&NElZQp&V%|7;Lm zz$>vnNOSPHwqAor1nXNp)+uwaz_J?1*v&@wGyP`FHZ!sdmG$m#+>wvEA#dS(b4sex zp$b3;%!T;g(p{AB_Q&+H+{!jy^Pd_Om#b(vn(*V{`_KtTS23pZ_j(gE9kt#)h71SK z?4>r{zc-vwXP(N2>v!?ez2V1J@?K6j4~c0$j~_EN2YEx+kvKp^-YX>fW$cUKr(@w3~}d;<-b_~7j*+z`g_jsS^g}_gb{P97GQvInT&gI z;M!|@6mK{|Z*HKy(Xpo4?ka9@Ti+sWT zMI8~6Sr4LzPuaMA> z3Nzn@rZ{nEN*{Bn#5!lr8ci*|S}?2>4VAL`Y9L+$Os!Jq+v*rN8( zeqZ#&{+k*lWo+^+&5FnC%7GW3)CoTHLg zOC}EJ6`u;>klXxX&v%MZ4$S4K4;k?Vd8C-X{?@T-F=KsZh_IZ=4&SFqE}b=eW|jkc zkgY4j>*FApQ`>1B#O&kTZHZ^Ddii9ySg!IqyzZxy+g@~>tw~5&oKCMNQ)b@kdU>9wnj=z7^s9eX4E+%xiLUV4C-+Pck;K!AAoaF5?6amC6B zx{3n1K?RBFOe-Fz;m#4$6 zH#LJvPi=I3O9#F^hF~qPyt>+c8|$j{4%Y&z;NJr7(SUBYS60k={QSRUxiTnLI@ES~n#l>zxLV$ZPU zF?XW+gTrid+7fm$ZFBTbfMu!Ypm)53pBA8vX^o)DG$Yn8_8=TbuSDHkqgq9oz zqK~9~C{%;Ni1qmg$hNoS!j+>*ODbM?L%XFgtE4`+hst#K&5jb~B!bNT`jj7q?{{`{ zr#l6#@;>a;z34ezxWV0j^Fw-bRKMLbSa@>&Xl{Uah_hfQe(4>q{EGXG#KSmRSvR+( zlg-D5U|-lR(s<+L*!3;T4eqkoiW_GeNG9;X_-JuzP!_@mW7ci+p6CkyEVi?;Qo{s_ zN#j;hIJ1v)lm>PQuKVGB%n4%-=#Yd$b`a-?JIAV_Z7Zp4V+F5q$25zxPi76jey7c) zqX8W0$1%c+B#6x5mq+|j7-MYJ5UPwsbPdxl>ZWUeekG#z}m{|W@L^_6_N31U>M z58fYAj{1Eur7|aU`htz>MaUGo#U3(1%6k5V7$mkDK|5i@(jmP&v*tcA5{Ifx_870e zCl$V!WCl^hKmveq1WuvBqxOV#Z`Y--s1`dKE?=)Kf?4;QIk$iPXr&BVuR|D>@SM6l zeL<)`WD$}e_$I`)9DW4KG6-B)K+6&L#$ZsLbpca&z3?1+r}L&Gvoywj>tN=Z#G~__ zjG~75+8(R{0uum;t@TDYOUg`k;0G~^U**p!WdJTG;I9?sIu4S&WGR@BM|U!s)BK4% zhG=ZlLe3c5Y#`yeov)FrHLWmex|9pA3by^RK9&Bmsq>6Y;H+uo?fJa1iKWXU!JB%J zzUPS&HBcArm*iCsX7N(JXGu$rHN?v4R3$I{vbol9~iPm+qj9|5z*IP#m zx`qmj&mI+H7JA56csSDyqNTE=_uVP)yYalQj0y)PWYT{5F}3{FrATzW!sG#m+{s-( zvyR}>D7BRhYjmCQn5^ZFI8@>CA4$W^_2QG?bO(ff0HitR&-!KUJm3*Iw+_h6VHj9y zquMgqnh0XMm#iG7gFe5=XB4}xcrxdr=lebPOnL)u*2uD{6QVy3KxNQF3$aJde}%dH zAZkuuldY_LmEo;P)5u-SFJ&THR_kY<`LJ{ZpKgp?$JBWYbKHRuizVaxZ{b?fx%f+TPEI8dl@dY|^z z>Ds^o!y<*k2oK%avR~j zfN$>sx51zn1{tF~vU-NjdLA{1Q5$c!R9-)7u;Xj!&28kHSHOgkduz6ElF?!&7m{)> zmsjI$=Av5#mS(Ka8;tc_wg$_WzdvHknx9ofC1%7d0GE)ewS}*aana7p*r@&IdPS5% zJR8n0B#h`<+KHH5ONXQ@o~$j|D4YT9#U}~cu48A<3s{|k^;#}}yzi?E2{zr=Q5aE< z{Fn$~&6;*d#v=5^cHWTF7U>9o^z`o|S_xiJeJH}c6l8s@C`x1YEwoFvdsW@EYqbKW zJTH;*djfmd!{T~-nPX1W-2oOR3>@3G^5i+M__#yKktN*?^P|}3p4K*}N?1@Tp#vSi zR+YZ;1_;Lv{~#qecA}y0VK3*y#GSEDcunjo*Mrft3~^JE5^H+U9y8Y-#@YOtb7rIT z2KOCotjn|Mu*|@>Zv84i(OaEvSKZ*Ro#*DjSJs=`tV^~?V$LOn7zvcHpT@-;_tA=}El2HPy0F{P7!0`F!%G!#rCct!N&SRR8*un%Ke1~8k;p}nnu zo^yL->Tj-DsX?dA8+Nj7MjU{t=e+N)IKDi{Z`Q`!g;6UJFA&{r_1GH6M|yAE?+PJXh|5QyRomC4DG#*UD=e zu1rMWnMAH5fc4#JCfER4YXjbttFKJ|ORf48!34U#pu5X8o+dS>f^TX%06(J5zBB59 zF>y7f?S^PaAmd8(qYA_g?&Ud$PFkCWZ>#Nd^Iqgpxa4{TylJVb6;nwBvKN(GZ{-;N z1ZFRo3q-r9Y62DlTEGV}2k;H*O()A!p#f%5urN1#` z{TrD3&Bn+lQ5?SCMLgG97>+O5=_q)2{hr`gr-UYV?e~fS&M*4xUNoME73}KZ!$l2U3pBePz+tx!s?>P!iu0={AA}k`J`X}S-Fnh&j|h_%XF^(MYCWl33<+wqcD z4hz2G=eGzUL0tfQY3@RemxaBxjErO9`AlStnC4x203m*248T!V1GLtQ>dsoMHyNCw z5K}y&G=?=A2EccA0$ntaC-^9vq<9n`|GT#q7d`-OBPaAJ1NP{XB6`2Tc_9!5cgR>Q z?_rXf!wr?plzXMfI5Cn;*&NVGYzNeSre=V**In{Kr~0H**sEOT;8RDKJ+T8;m%^`A zO@qFTjG3)a-oHC7DLJG9zEsn+ zd5J|d@ldOUaW!S%Dt@t-yMLtx5IK%V|CFtK>ac+DSYDu@c=FXW{0Rs`G5GwwYsJ%T zH%X@&C((ooSRYVhl$efWqay)J~MfI z(K1B_yG8&8`HCJ5ZPx6HbmPmSod&bObfDBG-{`pvw0=4hm>nsz=2Bc4)teIg$z3F6)(E~yA4GICA`t8ExdZ+(Bz3cg~l*eHc z*^n2+3AC=>jZ{vA`&-_lLq{d^5FI-3r+?f9sYcuZg-di{w=A1#>#pE{_IC66EkO0c zZEVb3)DAz(FI<3Av<@8eevOD0i&SJK$}X^F!Q^w46+~WAG+At|NbG^qhmZ<=V|{6AmKe&9Qk&TYA0;PZEjN2qeE z+xROVbnQ?q3K881T0%NnBz>H*U!h+`J*VnXbl^nsCgQUOtoLp z`-6rBapZkC^^A|HUtwol#3gWRh3)@o&i{_0$Q5Ka)2-@Y6`r!)(>5;5z3`U@=AgeX z|L$-~u|M;D>`kqCZ%46&2*GnOFtFBm=Y+M&E$C9Pt)C<*TRmvxvX;qot@?znccQKG z1=wuUC<534S4PF0Srrz$YXIhGgfKwTtFNm*xlcX8yt5eh3gs72 zr)pk3mFtFcLY_m(cMx*#e#|AduxU_eXLa z_oxyVh_?REkG6{(tp6-HZTSyly0%>x~jF zS2TS1t1M!PvY~n1^SE7mGOlb&S_U$%1#dqE%xb+DmMXB8dZ!qPpYNVzmF8P*dtKJJ zaO+t<1goe6Ia?fgPx$v^k=8U_%T*h#j^_ZG-Yu4a@;mo#Kkr1G+n#GByR~(*V zZ2w3|t#|7{*dKNNJZ~`9fe(UQgBUD7D;YGP5ESB8iXyL#7_XU zPFt_lzNlM~e7l-lXR)bXNR)O7>+M`_boaKUFOOXxV5VPpf;Lte)~7Z%C!S_WT1&z+ZFfYe?nx_j;!m?Jk1Uj z-9m12Hq#m*n^!)?3nlAKmn#OM#{1u9YS) z{z1Ea>Nrs&ig;|=SiM^y7wF-*TE!a@Yv1oc%?~IDe>|sQ{Y{#C%KB)r3f{pyveu;u zJ{OY%#n~lMaso><7d(jU2o#y<_7mSmn0xaw{hSqtdaUIWo7M4*v;?5eSe zc>qQb1u}(z;a18z?sGTwW~@@n1S>(emOsC;;@D~^y4YY`?QidOEomnRvbuc5rR|r! zq`s3@y@{Jrrh1R^*~}F;!)NS@r>mI(;aT<+Xc3y-OXs>Jx&DbG{{xt#Hi94$eAjmi zd;Rys-{IewTnC+%WrH)6(SWeXiodw={(q@_#Ypx9J>hWHAqQA0_?+5=S@cVRU4aykLnM)d9LCh=chfZZbZP~r;*bp2ZzKLNd>tH2hDS2gNiSleR0%0im zlo0R6pJD5hYfv**jwz8aH{B;ua1jJ+x*7mA)sDlwx+oHyA?I2ZnbGlCuy1s*)i-C@ zT$XP7!=rlP<6AOo4#yT{v0_fI|6Lc{j0$1&a#9E{DV1=>0;)sY0W}%tL|H2vQinC(tz$y_l`YNOao)4H%rC7)+q>8eHNS^^dGpX z5w_6$OYsJ1jTntLmF6uVUmlpi9uqG~zmbTr(6(T>%NGI6cyOSPPB}Q8w zde0w13DaHkretmI0?M06i)i3E@_uLUMQXakz60R;wA%Q}Y+JO+ zeirTeGFg(K{NoS_HwJW=<&jSK!ud@E(B*|Wpzd3*hTZsElSL8VXB|-DqK-)}&QVVu zy9kh;5#ZI3-k;h=^(KgXrHl^Oa5`5$T1D z%K9uBH?|h!HKuvhe*iQ%TLacWgEVn}j|fBt@~flUc~e34br1`V89;eO$b4Y7K2& zMn*xh8A&a82md z0iv7X99@;_P~b(Jqv%>D-$WC6$8>uxzN+_kUTJRh_LucC$M6L!1yNR4^*Upd)LmoP zP-74Vaz$FLk~(C zoAEo%(5XbdQP1gfW-N>*5I5MU`8@);Sr)#D7M!f72`^0Kr~D3|ez#?4HS(zBz3@l` zJ^T2RIG=VTvSNBmLf+m+mV+BpdT7Or_i+G4j1h$M&A(1+dhV#1x52NpXB5^h^Z?ww zp~gI88wYqM&7*cix14eQ8+7|mySfBkW(p{UuLjPI8k!`Eh9JTUtsbX*xxZBM^pvj| zHHXI_2nmXZI$tXrtfY%FFM`g8YsZ~4S(lLXHXbuKhqFo}nwLvAm4iD#*x8LxGu>3Q z-0hJXR4n9XNy`lBZzD1vw8)0;oK#nKSSty|azxVvY2;Gc4^&NG;o zS4*wU`zEVU#o&18ujvI5$}(7!;~dyhryr3gH>?{0i?XvD`cm@MencgESe9 z{5^YAdDa>9p`=BFy?IY?u5*j5Wo8E4x|`1oK>;P*tJ@KaDcFN5TTHKd)ohagM%7oF z%@+Y#wyc!LGLQr4(bbvq6fQ$X-g{-{@MZvZdy{nRSe0UVjL~19_5-RNV3;(?0bny% zP8BY5IoH{nKyZ#j!Fg5dV3EWxaWF8Rrvl{nM0+M`(ynJ^v}(K;);f}s46{B*7)EoD1xdqdlcGEFE9SvJwYRe1>*o6|l4_$h%2{f7YD zpbALrIlVzCDmZEhA=s_9H$JdswME|M~SX(UR4<{G9Pnm_@VPS9@+BGN66oL5MV4h`03%YEe1LFC4^# zWc9Y*i4B}x`kaskEa&^$5h?xCk)b*(or;Xb>Z#P6Vht|3Dz?cg7^~wS+lA+z!YW_|9lB} ze5(;s2x|3q9$OD0jR+*3U)JJB^ho_1&?9hFZqoH%iJiaR6A(CiwdCyk{;Ec(G=D*s z_3s%MD2mb}1OJms;7b<*k+QJcz5oZJlfwT6SJ4Kwm4vZE**~XEXP&Cm*HZY)cdRX6 zLeL(4VYmTc#`&vl?wPpPzovQnfYVe3;0c!0fK&DD1E89HIjfraCOk%5er{0x6_Lsp z1V|d*HQU^e?so%Q>z5P5wl~R9gZpmUXsT$wC&fu1>lkw3KTHPNKE4H|2q*31umsCZ z{jUtIS~YPG11}B(kAIIRaaii^$JE_16qAOsh^_4icgs0Fih_Ps9qKuFed{UD+p*5- zQSC6t%~~-jsnJC+O$!@!r{I=N)EHfu!{VF8hPFv3KLN!KQfzr+@!z3t0ltz`-~#d* z>$I_Yh7nN1P5UoSL~w!fiRam4o&Os6@m<+ivex^?ua%EeRr9kEfhK< z>dv#p8}{|Cd{fwewzy02!&3V*-W-&c!K(zXe3bjhId|3x@U`9Z>Awzn=EkM{4u;pIl?}y9weL%q9OHcDhx`liBULb7pvtBgX}yT#QTMXF+Syk2Rb_Pic-F;!+&>-T;VW7S zU;f>vNMm51j`?Q9^QTAS9PMUIQFye*V%yK+!6uY7-iJN$nFVn|K4^mLy&|o?nY%R~ z2t2a=e{7|Wz2nj%))fsHuV{s~tbvWitsdV?u|Qx0Ss|oKgA6NaQ@X&pjY`W$0XHiaJG0SP(5iN1G_GE|?$`S2xXM_Y}&j)^W z)VQquh2*awasGqmcvVEi+p^}0x(TCtv5|!hok_cY=InK#T7h�`mql)$F~0Ut_D;m5yJ~rFHI~=_Na%XeXuN|$>q zixL6w(DET{k{jYhcVq%cEtGO6*#d2Il;;_NrH-I97+5`ETkRIVcuKq zg>4_Os|zjH|1z1mePZA8c`56nqG77D^Pz6rA4Lm~<;7>@oJsWH4LZ=_HXd{;(8mhK zo%WK;19=O}2RxB6x#L=eoPB?*_X(u!GF^3pzMwoq$YmxMBjd)_LUuj-lw9nq1Ne`o zBgoIN5}Fl@)ufV;L^I5pvdh-7zwQty7i#8Vc~Ki?}&_4|pB+f18vy5v`)rrR{a zRyj%87d_^lGw>RC=>ArHcx6GS8Az4)=HC1bm}9j2p}eW|rdI=sfW#tTH&v0XA9N;1 zr%TS?d>yH{vb;&w-CRnxitPWjc#|=(xsSVK7BS@%?i&OPi|G)SOx+5rspP6xmT=m8 zc>L*qKh~Cl-gQU(Q@=j}F=*f~x%`l-$m+o1A+cl90ULKSB6J^S8j({gb0+T7XUwW^ zz7;jA+oIcZPH$_oz&jrT6sHp!#sj05!6W&uzN;T-<%A+fBCImg<9h$Sj_r1CF!e`(CQ;+j)M^`_VLZnZAKORDHL4he0W1TDUUlGXdNTj23 z`{?j$)zR}1A88@Kyk%3$=mxNo zLzSbQb#%mD%J!nfO1tNZQ$>d1$}a>^gLu#@PJV!>qK;b9N~YN|M~NyrQYnumi^Oco)O^ciILgUVri=J^`BUvWIPv8n|1R>$+ROx zn2Up-*PxTMNLWwDLw&In1{>rDF)Hzk8L5~p^lf&*MY#(0^_ejqaI(T%b}L9LJsj-R z4xXhFkwJXa1M~tsqY(ME32@p0x478Be{~DdfZdKvD1N8NY%02)IXbGO{eFV#el~wTM`;$8VU04 z@hMsA9N!pUZRjB2QR%0co!hi$tjrtI>`O!#aVLOhz;fUF-I?BXwkTDg6KEk-I3g?cJ25c&G5Pht&U++Us>wU@83^uY|PiI|kzDST%fA&9ODd=qSw9PnNU2h_x+r^|0Gmz z-A>0DINGLjqKSq=@*g(!<-nPW;=rNBw1RTM>2}%)1u0uL^AW{;tIKDt+AO%k~Ce&oNt}XaD7H9~4WNxM4R;;2dXA7)_lFk=hmTio`-^I0j>%)ur z+Xw@}R6YbefaO%xU9bA~dx0WG@GTuy#hl;E!gvh1Wl<#p79p(-s~d0=`BCi6iC%HVSLaI*|+*mfS=;=U*2FWnyL z|KBU(?0GLsixMc#pu^`H{+y?{Y(k@s!fbbBn)ftG5G?cvc&)(#_ zHg{!J4tc~cU6FvG6rWeX?32829WqCsib*au*VI{AkpNs~5t5ZhUrHK00SgvdibtiL zJHQC>R~cpZ7(ActU$pU)+%p0i4iH;T$DW7)5@f5W_*<*BQ@?w3V4@snBEL=+&vPQd zSh+3zuM`o>kB-64K7n@80&;QEgyG&sYlIl@s2E9Kb=(Ffl;=BHUx+FETJ!)qtN2v{ zi;lw{V{6&n9oyu~sJLc{1DW@n4w!R1`Q#7bJv)I}8sIMeuo;(JOlMo~n39an0q5c^ zi6B8DXcn;k6aU(A-D3$O=7I|+QbhK1{SlHvyj!w^q}qT|SZ~kaY@BV~?>MByB{Jn7 zP=meP7q7v7{fg;OINQ**D^t8q(oCmIk7_Ot8By7a15WAe>QivQoqDm74OOGO&|vB< zma?uwMJsTj_$C*QWyyS>u{*zu@30`*nMgx^2Q)lm+loj*|NaQ%juv7!87Z(XjP%M6lUk>eAlGi=y33&>WVGB&isSElBRndH-EsR zmcM4psjq6Z@vqMg2{gGac|<%z0deE7o>HiB5C=9y>I4d8-AwZfhkloiNRmnnbTLWK z8y)RyO%rDKBeriS+@Ww-Wpq&x6l})_kdQ*530=1OIDb!rKY zHBiqJ=!XX7{PP-Ey>`<7!v)t^Om4eyA|f#;jt*!0()y)Cl)&bl`;>2jFWIGBGCkI3 zI+H#;I%a$W<}Mv{`nYw{k()m|zHfYNnY6(hmqYfvY0)n~Qat><_)vhmyvs$iJwG(Q z-UpRmx`z7X%jh*X>oyR#?yYb=V7<7SNyTxve~I;atbV`ksZSg(oB1q z!>bz_lB?rt=i2s!1$JbP5DvhE)a04RiYP$0Nbs$6fy7!y;^{_wQH2)S%K`X2;FM?$lY-5Gux;wIu1>(M%kI7jR|U~ zETa+LQ9y1q_srYy=u+bT%tb<$V)8HY`BtxVmatpn%HhBZj~hytecms# zvxBapF=6AkMFpgNvAc$w)FDxJ$f1QBkOJu(PL9sYrf)BOlwmH6c4%fwcR+c3uz{<9 zfcZPY{H$}z1S>gsU2M6m&T6cfErRCTIodeMd;z+HS6IGnDtsUE5*@P=`ev+IBCuUi z#qLV&u#>0wb@bgBaxd{O>R`g=?=jdhUqhd8rm1)~g?a9%wX=t={Dol#lh~Pa8~V>& zh2LwtCLmzOScVU-1Wcmwmf(ech$OGnas=CG3;B&Qie5h7A)F%Qq&u?9>Zb5@{DJJg zI}deLw6nEmnrvq&+}juM<$EQ6{?hn`dYxdtjI(<>nYlII0e_RrWo+qy*QbNEoCW)Q zjRLB&$Uz&u%m`wzgNqcl9(^bz_ zdWgM{(8qil?F#s|N*XQTj04h6H`YPhtZ1R+&vS;UKajlD1kzvV4z%K{cAs2c!pS>& z6Vso+tt}|jPIANA-)Nrbu)Kzs_?i&?cyqJ3(CKe#r?sQ3SK+H()SIL&EM0aXJbuD1 zZ(%`8u@C^0-2I<*LLqaSvhzVqzTHemR8Peq*%-IhQR(~xLAs&uEchKr2ko%4e^Eh~ zY=r`|827QK=U!r*zOVn(qGA@5W@u(rj!vua#o_}F&uY%z$5lQ6-v3?ygJ7MN5vvRO z)j0Y}xq|T`lv+-oEjqtP;EvUQ9~`-_8HEI>X6#Hh|9tSb|BFkl_`g{};HUo&{+Y@8 z_wRoY@S0pJwg395Ne;Nom<#wreZXGhrsCzTlN2{ zH@Q>q2QCe@XN8sFa^++<@bFm4!0m1!%r0EEkGp9->vY0aH08~=RRd>VjGv5Bos#7G zd}vca*lF6vPJwBD(jBmUIFPJ>WD0e?O)}{7v`5YXGMm}IGMhl6wj{oCt;Iw$V&lq3 zT1u$nLGS%p=eBK~o4nz?NNvA~%3@x7=wR|AUN?J*77^m0Z$LAtqseYEc@J7 z2Xn;)&Nd>fh=V6^zb|nkrl__I%w%0!plCY!IM-R1l91v^1uQL6|6Kw1`)$5HOeW^B zPLSlT71eL=ANq1sG{PEF)m>>=q*5kE%kx#zEwY635u7t(t zyV(fBDEFG-tf(U-D%RtUjQp%pJup%A_aQIX0mjJPR6kLB4>$DaO1q42A|~X`;9N1Cnn#Y+FwUtZfn*;n-&MS?^SjP{{4b2=Uon?ao!u+^1|qA9$0K$plP$lNqBMq^>M^fgk^AT z_!{^G`EBLV`*huVvrd(8@q5iR5|Ag>Y;l1PGyglQ!XN|XL`Z3Yg5VtxRb1elM(D|!(@a{(z2fpOEL5~c+dhRQokc^n=Wu8b5VWQBR-hwGYVx$oThNee`Hm8;x_mJ|G z?3ff)F*q0#;dRO6ve1Z5^R>*V3dqkY@s-C8pn;_g%s3}Y_`>oCmYgZ^rlk>U8!;4E zyOQwb)L2!o!flp}lceCAzDn}N{*=|bLF;YveuZy`J148^>Y!`-%m{Tt_23<%;1e{ieEc%d1fjc@*UQgXsnFc-Z!1uEE8rz9npJ3-_w^2jz6-eHxaG|QMHVl)YUodklTJu@UiCXD*Yvv3W9o0$(L zIMVC)Erj{pAHIu}ui<(oGT$5^>i>y8ByjcAe{;sJrg_ORWZ%7CK-Xe)_0HRADD7)V zw$@1_H=g+Yb?p)3PKYqc_WozJx!}21!j$69I}J>Q(>w0#LW=VdO5>2K!dKAL#u2sm zXQ% ztG82Ax8ym$6b(-4rAYRxv??6;AnnOfl$V9dxVy6w<6B^mGwfQhKb+{387THR;!<*j zx#O`X;kDMCI$s>U!<^6F2L=`6gL$1NMXl9MYV;dDXPoN7W}CML`^v`5jUH8rKvj|7 zw>97fXxnMrV#={W zhAialVgG=tuUVk9* zfy{d3dGZorHI%IeF^A8=^EM}SQSA4D9hWKPZeNCk(}z}pXXl#jdSIZ+m-aGp{;&fx zj)h&M*I_9Lc;aNgy7BC7&VEEMK#MT>Hz_aD_nM2Dwf)PLsD$B{iXgDYG<)T-i+zEp zTX^%p&hx^J-5z&BvkWFe*DPWqRB;ANmT*Lr^^m>vtk3yED2iChsa-upD(8T~MXMi$kMU}jv@5(kJijgbs!H|yNJ!P$Xa1n}P9ezzi0PS| zHit1TEXX=g*C64Mt(-n+x>FySYc7@+t<0{gz_M?s>qr_|7N`GMn%44q) zQRa(#xY4&C^bkDS27f9_V;$L1UrQIZh{sSUN~oCttC{;9K?cyTJ)l^4N!#dE^)cgN z8?*VNOp^cWjHHUG22i={`(l`gskE5iKs9W5P)E)-E55N-WzE?|O&FO9Ivl>4=@>d` zlY7>b*f3L$)`BFdjQ;F6HstNwyVjD6G8p1;;*LR)vZ{J5FlBq0){@34AX@pTkD((b zLA358o5`CHrU9&@>{mRaBB8m4@6tzv#)nZ;d9S)V4(J5|h7?QKfJez66En>7s>?u0 zjTiAw)w*SurH9jOV<(4b&$q>Jn0qc7Vw`B?&;4u||$kY5BcIHr;jh&V6UM?xmkF*hdFlrzmg4 zPv#eA(~lzQPD-!h>_mpEtW>=)<1LZy+tZpQ6b;1Gst+SCeC)V%0Co1kbR?5_{68Cy z%gd0r;I06!)Wy*LtgU68iuv*q?D{5;2;@xO8%<=y1Jw^4k6c{PD#IfIf=Q2AeMg3j z7*vXR688&|UPnQ%6qzefD;&nfM6N>J_X%kb##}nb$FOMozObXhzY^Le6>UCL$#Kn4 z6;W=*m&{jF8~cu^nq%`c-bFPD>bunvpnHT_Sn2kM+Ue4F}ka%1Q*xCR)4OL!4R zpiqHDb!knp`qutR!69#x&W-xSwr|hSMWgdhtbBy|KsM3E)LAB`o$XhDn9a>e`Wjz6 zjIRv*c*^}B(sZ{X0#($pF0wZ4()O)wEOdPDc}*>pa?W5}v0j$A_uFUHFhNVF|1ldHxmFu3UhZFlg&X7@WY+gkuupXO7_6>iy3$2%Th!Hzg z12-e&H5o`vo*#^$^wspNl z#)Q@->b#2oK=F1v_*(CyzNN@xJziS$un1@hYZ;PD>S1Wg#66H*6uMPqE~ARRST=t+ zXIQ*zj@P;rr~KnlowRR8`}FNs(~-*JqLZ4w7#YHGUZ%V^UZUv;3d4=LHisSp(763K z+7qI9b))%}Dq3j4-kv+ujC!A9;{FgP@7X$PI;O5r?MhCoFj5iLTr{racMsH&hfO0z zE-lcvl=|pkjxG*pJnbagc#%5A zpa$1bf4|DexHo-moMUNQOlWB$aHQskLyPOG(bq}vbY4gB5RwN`044`1>)cOm`~`J2bG zp)9aG#%@iCpIstb41Mc#?ZI`)RWkRPFnN;YvS^KM?cqSilde=>eo4!WV-Ga)LMlX?Zvp42-eV0m@y+pgmiR9}js*?Ou_9Ep0-o0R%- zujbI_xYl*yk8JnZdC;(f{^k^L-%{H-rTieGvfy7wktxaXMC`!xhff<(hrJ5aS^*h~ zU$bp{!@o?4oAMF!fcWBFa62K}OPw=kh`b36B1%t)mSk7r$>e$56GN(DjDX~|@deu@ z39=neqiArEARJw08zc4PTJ&azSG%Kb^y`AaOQL{iY%N12DtiYGoW;WG`+f06no(z;BuEDm zUP6v_T}WmAaGbw(JW7mPW~*juN_5&(_KWm6VpYRjTSEU}ZW}9JX{SAe|LcqBvB?dR zib#`o-x3zKD7vzQD%E6AqB%-xN$Kft!YEq`9Epc}Gl?0HD7j01!9~s-=1N5R6I}lz zZ;MNjuxDTNG@OfB)RHTnqyU-0iR^<;EqBs3BH0vE0)R@sbpvsvDKR)L4vBx*1;lzi zJ7kB;@6C^Ld0W)_&hof~(1t{6JDu=)bd3q^b^y2hNSWIiN_C$;`1We2!fP}Hk0CCN z7A=*G8E)9nZ6U-*Q^DIo+X=!Ce`IY{?R*ZIwo!0VXmIy-hZ2FZC9@5@Q(eC9QS>1H z^!1SI)R0R0r`RmUeZ2T58Xsp`;(|KeVF3QymLf(DDW%5}d>8%KYaI3!KG+}ZX-+2h zc{2`Yc2@m-^P6st`(fz-f>i)8w;t|M*ZJ51v$QYH$8k}z#k0E=Sus(0U6%@Y)RZyL zv8d&ktio+`Eaq1%eH#T}A8$o72q>Q#s05@0lQ-LDuXC zV;ce=tO~B3jH#p&Kc3aVu(c^7vxYuWHtQo~S=@am9-w%D(&Z0Zy$r(aNZZ)!9;bFn zAs3rEv*7G+k(;_ZQ$s55F4R1IprLon2;?pI_vX3oQEg6$OO=;Z{-aKg-mg1N}?4ZP8Ne<*0S5!IZB=s?~S;|oth3qb@SX$)no;{KZI_ZP( zuGjCzQUxJGMwEK?pQGhz<__qeu~qtGNuw@xqZDs<;dfefRnh(Ho3jzn9+Ut4VC%qSH^U17(cJi{yi1xtMaoUV@1hoqsY^7 zRq8>S4tCilh@TMrtAXvf1;nBoKxM?oZ%sfw6xWtt=e_6z?BMVjfA$@#8z&XmQw76> zo?07yQe#KTpoUL1J#%j{et66533}APF%{aqbhQ4u!si!m*kE3znAqHT!Ajz0`@x(2B_6Pw?vu>$of21${} z_r9GmrpKoJ31hzGRXM-~?G`2i3oKR^Z*3sRGZ3c+Ll5t}o}Z8$TDbn=_0h`!LIS&S z5Ll@TqbfCo1AC8VP7P4~pnIgNcVY+f+~EFJ6_U^2%MYgZQQ6PnmBboj?K^ zVqD+&1fk)b#dq&t$!6AIsW4G}#^dNbVVG7VE>c3kx|}x*!+&4Up{oV+m_O`6^Rbzp zFkW0WJ^N*XzpgUBRR&9~b|@W?jdpXO7j~ZSD_h)M=BYII4}C$*jnOAA zC6m{SHqgjeJGurTcv)gHe_i&TZ<<=rN3RNosjA1<)wJ!lQx6D+)B@`pPwPh=pJl+kbM33CpC%A-X!mEuoq z`f4Atf9O0?i(GXA4?iHeP%aEE$OzXDnK|+yz9!xlGnpSgSV&xzvav2y+b#mchlesH zULpeE3RuanwbIy1&w@O(cYXb`=e}vb3CdpkqfADLS2b4aYs6M%sDrb)Z0~|FIi$bL z`?O?VAUrsIi|+|hbnk8^q53zWbBy6(dVvuEh@OL53`Lr<_j))gg6LwwtYKIe3_aAc zX?5ou5eHzv0K7fB(A!Wwx8BlX$x#0`93@i;;CZ|7N-@Zq@4dcgK`PUcxrSx8ntOslvm7rTY_ zGRC=v>;$^{d&jvmGRa>8Dj8Su&Kl#Pq_c+m1Ygst#yR>Vv95FuMwV!3yoi@~IrVEp zYn5UL_%Q*e!3~_=lthuG0vA)J=52KkMNnEt?`E;-O;*janU!1)I%T^NQ=Xx(50Mhgz3`cxmGb;W?r(fnU8Jk|_Osu?2IJkCI z%$8U9&SU)g*8xH=enANXj_jFz`w6vJKchgcsXVI`H<{0ySmiLq&mrk|jQ#;us*wXX zcq3#PmCEQ{sJJ#ijdbR}`2vUm+{O9=nkL{6WIjWeSXhKj9VcpE!K>0cZe*95%zWs- z60q4gDreWg_K=*+%#`;Zi&Ir`E#Xi(?Zu1T=@EzCdQ#+5Qy!6MiyS!kKNK_!nE|>6 z_MZv>5c>5lORdwR>_Vb-PL0zKh&aGft{RxL=g5;j|8?omeQV3&*P&}yY1v5N+cm{` z#CU+SW!K|485BWEu1hK-HWxwm2i?aqO)uSP^6AM$2w#uGVEOP9O0`*-T*Pbl#T}0w z3!FryrDRV}s*m-KML!~0>OVN9zd9X$glV1|kmpOWxMzHuAo32)A#Rf7qZ-}Yv4x(- z`}7q%nX+g&Kg{AQH4Jc3ZN#1`$i-K(7=mV3gtu_hMQ^^9*~5=sGD#BuUdUm|xVkd7 zK@!NC4VG~mm)(e1w{J=`P%lADReM_m98q9wMXuui*etaU-)%5k?k$C*!Sk2%*c|8R zS?=#8VyB~+!d`0vYAlK@&vb6gI&g*E;?f^+NtlW^+7X9B6HyPOfQXe@N&oPc6F3tT zGfy$tDSCr1DKYs>1ZQpN`4qNyjiv3GhnF*}q}8F2Xo@gZ5v<{?5}H(>XL2OF*SrgA zx9+8I2D;+4?a%MoljqU^SaqQxJ!e#<7W`LRJr}3>V?Z9`Rz^ zrhqChJ3*sAz+40Jrg;~|e0jUn-s@I_kJJ4dVxt_*THjSZ#-ybMYGUX5me3;>cj6TIQ=xz>?ALm$tpaOm+ueS_Of2dt zOREzjwra@5_qwW+1L;&#amXCU8Fi5w-&hs5Q3>|5s6n66B3AS&`S+llBhM0PhjsdV zn~|ni>aRBZgRw?~le|%#>+8vnx`0sA6+FnvD}R28PcP*yOZTC#3)G{CP4GS=j(Q_T zl|Nwu;a7Dy9p^+ZTvP_#vmqK%KiA=X8T9^G-l#(P|H4kb>R5je4uC_A=_ME+>f})B zyL(q0Ef#Gp5h{B3xS1G);Hg$C`8=g*R7ZixaQQ+0NJu$~Z~7KXJ4!crE!^R|l5qvc z)$h;nKZIl`(`(4$E z)Rfs;HgxplSv7#2NiJm?Tq#9e!)`vP2?RPnfXgTn2jYU1*6&J*?0XomL zBsmxV@Y~p+Z_eXu1gqV{X?Bnoz@374L5f?}Zj`R>O=c z-}E?ycG6g>GAxHq7JDYUQAYy%QsI5K0TR|tzKD6f-&h}oeS9nQVm?1sLl^9N$Brd) z^sy>561#bzs0*@}f4m<(Bv^FLv=jv{`eqakC0vbsK;nNHh6-5`qRD96kYsW?-#Ekb zC2o}t`b$^dF;P!h-PNkfNUUemaprZf=<`Y;+yBU44aNFC3hG-7DGgViojE1sdr+ux6WM=2iK)5$u7@J< z$ox`c6;N!-B%9x zpM}m8l18tqT8=Z?qL-G|D`)!D=xLwe8RBs&G4SO1Z?iLSYgwwNI>TV!g&rF028xF0@(FHRe`q*9bdWZ1 zusJnkw%j0bN$Dj}1YCkLowa`OXO=(7?~`n*&II!1r6O4(ie+^w4T+U(+tVW)cU^ym zC_D)QF(vgcdMZ8=bbX3V3rrBoe43g47zULQG9JMHhHMuiaeCk^R4nyTs}!<`BgEuH zKE867JZqR+?D2%e^)iGv_c4s+5qDE4?s)k@BV^p(?!1uGinQ3eG_fipjg!Tdqu|!% zie{<2H;IDLrbz9>^4ED$R_LLw@m$FX?R9BX3MW70QK-FrHOz-_Uu`)Klk2t}UR9(4dnt*f4 zIm|E^A}G%!UqL&-A?)&o=$mjmG;-ae{`GQk)|~@CEZK+>;?ju#(jhap4+}nO7@u>r z&V-)Eb1ygs(e6P#94$|>WeiDFw2s&hl?`M30?wQDutfk2vM0fd}R%rX$ytY#A zvM|nFGgiSpnW|rugQPmJ7XK=177_~{QXcFbaD>AuY{7X`*1HO_~lkSDJ=>);!M@u^k)m%F-a64 zE0>2}*t}q|;q3K6AcW13X!)KYMcI2ZMfx?9Bn@T?sUF=mTPN8zn_w&?rlmn%5MSI8qYZb#AI;xKMN&^XWYnO+}LV!^GPE%$cxdTjdN81 z-oGrj#P!GgOOxBM7-9Ksa6hZ2TcNKAsG?T|G96klw9#N)iR0&A&7Hum-L3-ccKtSO zg(=4}!!suQ;*M^e*LuGF(oLrj24`Z)E40%<-tMMs`V02Ex4iW( zkNsFyp?v3{bCZSi(^6t`7v%`j>I=Dqo$-iB0^Yx%x;Vba9NRBzONT^6}0yRHy;kpilnr~P}pY`9CImsk0cgwg!yts-k#Q)C3<;`~zj?q<4xVp)& zxFGBbPMz%Em_64=mSBkqd;;74j7y7ca2;3V#|f1dVNKvn??t=|1sa*E2dD+kG5d zkv(HNPc;vSZ<^k;zBjs`x1qnrBUps&j- zx>pnB3=pzeDMY{a;|`!>Mnwf?GlsL|Y2}0Lx*$0H6@~pj_wl`QwmySNv_}u~1C~zE zMeIJVSfx*PRj(<{!ohOGvU=^d3J20d!FTwu>7EqsFxeH;EKTKd0vFKI^DQn69YEuO zEETVUyT+lTalMxCyMt<0J)dLqZI*Vn{l!)e)Sovf&pn7)uG#D#I7L%cVnBw;~1`0X=$f4v$8v`go1|_n% zcP3fuv7|G0hP*Cjz&fl5N88-e;Om(&_BIC*>DZ@z@P$=!1 zFJv=#YR+#qO1QY%=m?X|pH*PZGG3({UujCQWI@D@==gAsbT$_-j1t#?z3T00m-o8D zta8O43=IZ8bGBb(@m0}OlXJQ94wnWcb7ZZ%h|EsQC=!hPUCDT*j2dcS-yNPv#};Yg zO&%BQ{H>onfn--=oPUvXXc^$U)?RH5cxt&&E#t+3?AkpUY4cIq57pj-?i|^g^<}LS z^%DM2T7P0v0c)_k9C>WNz}fVQhhm0PYZ3 zJs{O2=TUs832W1MgyRTGh!(6ApNI=H!6G;(jYhY6P@rNzOB!F(;(BlD0laO);Cm8G zl_IvR6y=mVH8Ii)DZULb4FLqx>e-)~NdSI+3dV$~0Nm}8nPD54qE@Gnxc<9>H|9U} zFUt>*9_>fb^o1b?4iv>$sk5h!GUf68qkDYq4_!;D3tsK`rQR3IA4IIu*L8HYBs&}t ze`~a?FrmLpFJefdpYt(=N&btizf%RLho-#f0L`%Tm>$CP_>EaQmnnpzkP?yWFYN~r zg+Y(`{a403lNhW_x)=WUkgQTTfL}8*#AV&oPb5}d=FvqX7_qxRQqiAclYiiU7QM2Q zOn(ADKyk%?#}T1eiIRzp8a}`7w@YqA7Z{US;RK^oo$dMLCi^Ewd@q}-A_JCD4sU)<>anBO@$mIwqt%spCm-0;0^t7(2yK7a52r`I~G zCGvA7kK2|z4!A}D71OM9R#}!-)5$oVCBK;d#!4L)v}agA_l=?bxW#mCOiP*CylH~v zXT~LD*B_jHck5K`y4BKPfN~O*%zgb|oW_dzDsbPde(jOhcRrLl;Y-@i=<8QdBjbhZ z_Ku&A3wEHtV!O8Kr;xzhyY6=RqC3|qv{2B4^TA@ZfjG8icgu#cp3g(8I9E3j*W-U1 zdKtK#+6>FZ9){J`NiUSpRLXe|GEYfeNC{i_$kkT9Kk;b*#pY~Tzn+PS{N8;b|68aL zEvu0)eQP8LdJ83_7t6={oSG`MCH1><-`UdnXl7UIe!I9G~})-j3D7! zH25}w6Oau}^!ss@@v3jU_Zk`Y+26uh?hsbu335L4i)EV<#~@;?0K~2*wdCfK2wg|-`5$bkecf4{na9-VtVd{%65NG zhzdaI@N-cHlKDP)had$0^Xfv)e9f?Vo(j8nJ+1AxZR&dJ(w>aU5Ow#XM@az?M!_TY z%~TeyJvy7W@roqrk|7^@sPLIH#q>2v%%y$==m=-Imn9z~w0amo1VrK4ffTC^I25NFOS{#P<`B8OVxP7wD&Xvkg#9A=!#lh} z*z*2*d)i5S(9Z5dhLD)ENS7p#F(qqbtysHDfO?qx0=i zj3FGXK#TEo%~VG`*Q$sUR%7u4wU4fM&yD&g{KE1pY6!Tq|VCjF4NxqN-uE_Tqbf2T+$kcyq7 zapXJg(RNnYz1djiTziACOZ=xwh;mo!^%isPK2?2emW8tFZk0ufPM;RKjA%9i&8KU% z{&=Ha^8ta7KIt#=;?ikQ{ARpQn#~^8hxH1W{nm@OL}tO26+xD-vpWt0JL5H7UbxA$ z)8alqsH4VGD7a*zDQ?p!HHyU<6veQ7%qAyw94H+zQWBqD;Q{T#-|H&1Dj`Vlu|`-; z+JFw`woak~M}kyU-QPD4C^KoW@?c-3w`wY3JWWbPK!E%u@$+88=sQayuSQA7N1atM zU`{FJWXQ_)24R+VOU?*on6@ko5~Riu?Elf6@YE9937~+)f!|#~Xejlib!FN|6^yMy zgUrTHWG66*H$O@%4~$7ELvJ*By=nd%i4J>kjwkLz>7Wj~h8!i}bX6uk;=G`OLla?SWhZ70?Y?J>Q(;nThUo~BQg@K&1Jc#3RLQ$dEJ zHR%n^zp_WNmRTWNM;`tVdolX+@lejKD_CP$VUR=5-XKE&gVcl|%2FO>HU2bLVfbC% zyW8IDWZHXN6hs*knvKBYe9d2>r1-@^!BT(f!s0p6M=8J;# z8~;SaZ?zoz+Zabt04aNCgfJ#QGpFflM!r;jV%WKyxCRYkkRujYt6V(tSI#PwL@I{;Teqv z(idkB+RZ8!A$OhDd5`t>BzHT#UkCZM|EBurwLhq`2fKF%A{HF$N~oO1CP%oKwv!!C zBkqDVcBM~u>lmJp%72WGDJm^Yctda&`AR$D_nTz@;xTS4QRw-QTFHZmmU_7q&qoOO z5m$-IKpbzejo>TJ22KO4>wYX_YuF+{q&AV3bI^S$kzi#DNtK4VT1dtg-EFLU4PAFy znf|P#s$SKZOGi%m1@Ip;@j>__Xw;}xleD1$`P~zyK_G~3q01L~LMNA4nBSZ&nQHyF zr*(N+)&=!6vaNpZK6U&e{vdiJ=E2?PUpa1Q#-`PTS)89q6U17H19gkJ?BEXIFHS|j zpMqa{hUHP@IlqV<7_uR)Mj3{zVRkIG~&v?SywO)egN?}<{ zWJx@31WQOSd-XXTk1%pCAbgeBtfoTJ>W^1?q*~-$t^tTImTN^CWA3wdfI3|V8!1uw z@}oPKpv}MJ9#82wQK|OR3<+8sE+Cfika%3*!m7-gAv`FDofJ|kfI$ezXb`%*nqEfoE62AD5cO| zaal{%7N(BrHSPNqBWFuduYDZfbtSD0BFEYe&DnPnak3dUyS0gy;?mbBbjd}CJTW<@ z7q^tOF2xsP^J2rU_`Y{Ytysmv)fi@%25I6&s^t2^BQG24Dc08hp=k2?L1AQ{rt%I$OK2CqtwJGrnBbNmeU*ZCvoKcON}M* zy*jg=gdJgHw@u-LAkA6#0UKlSR1OX2Sja02MAx&b1N2U`452=EvW=jPS1FEMEq_&^=^LljT8y{pPsEPE$2@^DpGq(F$wInP5)8cWjmwSB3Kmhd| z%L-sUB?i5Wk~D&u-GQ&VAl$n@b|a?V2Vb^<{?JJXY#{LpQ2(mvFoi8Bkmknmjntc& z3V~F=g1E*6x7m2+hU}3b5Yg2d!us0aq9d!h9j$?8IU)v6oq3QKnw<*4MQUx*aXWBL!XV3YAp58Uk`Q_9APL`i?Wa-TNJn{up z1Ovmc&hIgXu>q2C?{lYu{3ScdL1+4!yxX(i<>eUAf|t_<5IXIb0#xEU=e%=#{_^CQ z-u=-FO1=-SUoI`J>GOB8xR;XGq|?EBw2!Zd%RMS@B4HH52h3$i?kem91)2j4Vx zIp2^6Q+f9Hl(4e<9&cYRlDBqu4~;`9Ay@rLl3r5aF=lT(VAiOv`?RRNb7spr7!?6D z!Tubb3K2GvKXnl_~AbS-(R7nr0W11ad%qN{eOvQ zOkodzUs|9P8G0QVlR-Yi<3IWD*-SA2Tk_`XgX`j}0+Rn;;_d~2C;hSg4^@)W%%d@g zl;$_7kUs@c2?6kh1NtA&8Iy7+fAVk#U>9E7x&vSXRMO9G{aMe0#qu4y3Hv zY>{m#8B6r|Z_y>`6gM3UK*OoeRnO@So=&wg9|n-U(#qB)-oli1FWKp+0q& z$!M0ce7ykbUo?%ntNh_u;~je~Kl5n&I`!^2H3FbE?f0wk18@>%Od54S(NeQz>81yl zbSZssy%5y}LeMU_^Z~9tW-J3v z|CzU6;@)SqBS+Dai+NPT_ZFgfp?0$lgE;@Sqb#=3n3Hrg-s&-Lf`4zt#y6 zNEd@{qu$84ozB9a`J^@$`;^TlsF>N2rP_&d-IsD{%E4i{buPj5oYlbB&LQyT9LhrFw<`u>)6>3Z5 zS`rZO{c}|EwMc{|k3I9we&^qpuIjhgpVByIR1AQiFb`L7%Wk)G%=?F|(@v^Q&IQu{ z?(4|&=MwKkR0y_t@*_Zcm|Fe%{>t>I8uXh9MQDzNw&(jWiouDtW62W#`;6vAfH%zK z^N9RV#Ay~|Y&=m{E=&u+HYH`07jrb~Bg>gdI~N&%)&rg9b%!DRZ>(iX|7ra*_{~8z zGnT3<3ejMAEGC&N@2HR;bof3oIfQ2+3aivi=qo~r>)6h5fp=!$N$hEEfT zzuW+TYx23~ZrQ(Cs>=-7L9FNGC)QP|rsX=2ec+&=Kx_qH!BX^J9(Ct= zAVqdL4>S!!1%oAp*G3}zoQkMrG`-#rSf#%oizyk9E^RK*wVba5pe#3^bH{s~1FRvU z!vYuo{SP4G!2eua{zGD499Sm`#mWd;U-xzrcb#qZqCtV+f31HD0npf-2dopPc>;$Z zmLa*8#$sPC2bi9}ME>7Q>*N52ksj8sFY~YlfL9x*%L))ItP}q})?0p{H#=WMJZt^m zF0#J#f3v=KtTSmnO8k+u`iB?(cku;)+rJ&a#N=>}r;XeQK$-)quToA-y=)Rg%)*MmDqGMT7h& zC%q;i-!YrB5+7##cXvXR+4*dqmoaSui@BToZ`DVK;dlXa$D$3^$n^)eT&wmWFiGOy z%fq_kfZDA3ovE=~&x1hFi#nsrfyns-dd6YDROA8x*QY)52Qo>7|210+;9)c6ox)x| zN0ZFEp$Q*OkKp*H9H+&5tm|VcOthf&X}C#FbQpLtv2heg?@!&b25iU;!XdEl^zcT) zPX9!PuFD$0VYYSk`zFG1IS-?rb{WCzV6dyhUiH~T_Rpkxl8c;S{d6`SQ*d>@bfYXt zr-^d#RiP`p3leB96)WdO@anp>YUvvL#?^(`9_rrd^CQp_f2Y?Na;PoEK%8c}(EV4+ zfJt|__F>~srp$9d(>_HcA8Ubn_mh!&oRp#MzE#gsy+2$|h!Ab>f3PBk|0 z7QJNUcW5JB(-fh>zD%L~W~CLD>siLIb!CzOmU!K>Q=lYi0ZeqH0#cJr*}H;22X_8N zbk-l@Io>ZDx6!n})p0O(1qi+KXG9LH8{QyPy6oioveC& znC=^ELf0bfy^c|_(CD9XAmb_CRTIzcvOJRm57NrL2AjlYQQ{0ixc3d!Gg@1TA4`7?7y9jITa5^=J#8`|v!XlNwXcUib#PjTP&E5Yta9+D zu4jGV=pWSC8`<)(h9LbsXNWacvd84lzJrDyA>?K&9@1BGET|cSeaGjLpH7eB~P;$83d+v&d60w8m z_*ULB#$=;-J&za7=L>OFKK>Z5WAk0!FnHg48j06Ywc>eTnB!F^T+`JSP~HVkZ{_P0 zc+5KMbP6Y)&P%<2I?HF2Hn8uZd8_@vjlF-Ffb`;sg<|l>A(vbL{SP&pH2uA)iZJO@ zcJu`0W7VQNVK?`Jf@7`E@=o6gpgZsdWqgUzXS}{=j_ge4gbFz96SF1kuwA8WVv-@| zeXwc#-!17p$D{BZK++ z{$jOfKRC9bWQ)Cf3)ChVXI<)GUC^qN7RvQzNlq!R%ylnh`jwZ#Od-XaGKBjm@bR)@ zJ!d&CMPI(L;JJ~P6}w=F^u?T2G4!0<@bc3nP9wkNKHnnp2OA}R9!>m%jU>YWXVn!T zhoJ5_zdue_x3EUHz_IIeSJu9%APR5%DyYCalyLlXEvK}4{P2k586bL@L6B}Np&#?& zwNOYiC8RnP0U$98*7tCHA5Bl6r2C4E`TA;e@Z_iEAQs>Ih7x>+5qWe6m{i~;b4z~d zhx3&iL+dL?MRMp^wfV|WQ|~lMf!c&!KM6_$8QrPhIP$;UVcslCBFnih=y3S~|69SY z191_eF7y4bF6M^hg1qJrvO%xeSBrECKg8TRrGMfkC}|FHO9LxD*s59PJlnhIvcqx7 zG>p9pQi^A*-(7emAy}lJbSX~tNUoYZiWs{l<7R zIz!%hSu4eHzmQi7ilKNSN0Us6isUgBJw4!->XXk&6?V5mQ(klIJ^JUQBYpzNiRO4=p8P3%?V(TF-~oJ~ zdS#j;g`PAO3M3c&IV#Tp<4+G`X?j=4mcn+)+H%~m!>ZmmLC`~}+zM)ZG7a2sT%Ci5 zop1*1rP6{r1D3*l6Jw58vEV6Vm*{$D6LhL}N|n7xQas1kH-~S6Z?Uu9lXj_k4^=9G z50_jv%X}iD(TDmyyw1Me(cp}2Dq8j=HX)Mw1?#F-U>)_9fBRask># znBY8}gRc29EZTk?eGw0hBk#UY%7Ul+u6}C_g@*#hdcJao&!Nfp=q0_lxvP6hAIqkH zGe-=$5Zw*Y=Ug1l$5!NQx@Ra~{}vc(k6?mQ{QtgYrko>G;sJ-OCM&>+8vVsPLp)1e zQRCf>L&~y_FN8D{RPggEGyPHnW8k%=5KZ!c3w~wyXS&6^b4;Uw`yy7o%O`mN4fLQ^ zq2i5L6RPjgsF&0HcW#$J?*!`S>xmZyeF2te$b6;Cm}fwpDX$33SfWw>sO#T1?B@YW zBL9qFg>kViR?Va{12ynJIm`(0DZY5Ovi z10doX0)7k2Em}Q_71lt9h(6zUj`urE7gE9(&D0Mt)#_3gID*&S+euPCOZt|8P(zIY z@3&xo+bLcPRFizgJXnNTV3V8`9s{9<$xszgxOWjtZb$_vW|K~R0AVXPV6|=I0Cy5N ze%ylI_I=P&3Kb&G8;0{9?-B^^GbZ7#hyM&eSdQL_kxog`*Ap)|OMB0}L3ETVL=6>- z;adHBGB~&^Pb-V;-sK+r)x8g}bO6XX0%MO=+^H zHDft=i?^G*wgP_tT-+p|rd~ui=7hS#5Pj_{wL2kGESxj{?8TgDpjN41X&JIY4Rlx@ zSTIV~EmTKx3_oSbBNz#;3}tLYNFg76zz0dMP`Rm^SMuDQbYXjao#$0B@4hkrJQp2n zVwL#flTF##gc2Q#yH3SiD*-Wjp<5Io?FLg>t}|7_=U-hSXQA0n{NBBbM`_S-y*eWu zy94Xs2g`xWT_ri@74{Y?E-~2*QFP23^TZ21^7c#0Mc9a)uLHBv*V7&~Px)QlN5DsZ zfeeEitWPNOx3Y(DR`XydX;%h&qV6*2eM! zgHzON(i*95tzp~8Kr(AyvEsRELc_*>lzO?7`*_)& zz9E(TUGgBQn@7l?=tR3_bgUg*!1=;iBOk&u2NPn?Em~0{`Of6EbcGKC_hit|hfAQN z8%{R5+b&?T(@Nk=I4_{#p{?Uy7ou(&V~lk6(-uCBr;l!$J52?=Y@<9Yf zPL29nn)xAUWvdr+znKP2J0L2uMn*cH;crl?^@rUP{-Pd8TD*yx;b82egZ5;XDM- zpxCdI16bz<)s#)gbgUD5Z=LxJR5~ijnx=<3A=v_e$u6zitz{UZY`m0|#Kv^*%){C*+v1C2P=#5jwUnvG50HOrg z=6qeq?o2b`+KX0lBByB7rgikBjMEidtW9{xVfX@8ri{Ey=WpQL^fQ;)KhtMPf7v;3 zvEE2+>xkOk?7f`d%3QKfcW6B_ry z(_W)huQfu@%!aB@9Gu=O9wuM^BJvyTNz0>bqq+Bu4ylir(}8fD$Z_l0##noU`sFc` zmeS5I?|KLZwm=;LkZIRwh&<;hK#fuXl8T3daz_kv#v#i0tzQa1GI<55)vL*Q=z#?Us*Y)B!b(5Tqptj&%j(7o=tP0uwku7JX ztma;|2HC~odFe(EWl2%8XVLo70qZ!ef+0~KRdn1J0yrgSdeS%+D+0=Qu-`VbmGWG& z)(ewiB+R$DN`@+s0m*j>t;{~SeU|?Kf(FLsF&6Z?ru{g&8m;!f(NV(Pg z`gU&Q701jkmJkrWGR7jsP_K}rpb1VinXQ0y+S};fbs8!BwJ|sG{+Y>gEd%77oO7SMUDw{%-jR|c3!5Ee`JMmb zoabopfgYY}oJ9d?oBzZsQLJ_-Dn|wLmIZG=B%*X)+xJWE?@r~g6bY+IN7bHk`iD26 z74V_=L^OkO*&j|F*#B+7zuwZ0;2Qe$n|8vRu zvKQ<9e@FdKhl9JxfahfN-*x1R#|wVv*)-fvufhJS@x<4;xp%OHWQH5rYv~U&HMZ(r zg!!>&GDa^C_;Ye~NpNyd@o0C>qN?cMTWb2bx%LAN*bb2T-F#(l{onMqC3Fwp{=Yd) zkjs(OJo~GWl`&l6z>DnX8x}4zd9~M9Ua;Tk_;n=kC(tsx8LRq9;~JY+b@C`s)(-)d zK7qUx?Ce~9e{2Z@R_R_f+I=GBI_l>)>f z!+{zQtwuT($I&=<9bJqj&*&fiqB(gydS1HDZ}XVzmq28$gKSGu?RnoKs}cn!esxm& zm;b&9KzO`(@Vfpe-nCTTo^4Oxx18GY1=O~m#pB-Wt)heQe=IUX=QWIhCVP(FU!F=1 zpX{%DOY1G`%k0()dQM*!^VENSzP&QiifTow`IUhCROLZ4);3Ki!^sLGc>8#eoN%2N z|9de`M?qR#tRE!5YC(n?9awOxIY%`0c8I%3srN>dBmBtP4?}g86mKh4yC@lQ|GzB>Mtbv|nX z&PT{D(5t~Ae#nO5S?t;C`@2^0gx3vi|Jj>Pu_0uIZY)i7Shafb12Dfk5W+RsH# zasJWG&hQf8Aw9JT|HIy>nwS9KIcSB@nsnOEQpoaNtiP*JXPD>m@oRAT&6Q%3cy}7%(9su&a zH*$jbJ>tvcoGg;jO%@lu#c<`up1s*UmLk6GIc2h?7H06*cvqikoTZIKxtg$G!cy4Y z^uu|2%~x)r9t3&|-p3y-v;!Vuob?qkIV?W(n`ili2cSw`d1gcP*rQ4y^;Pdc#)lSmBtAT;`TN`K+isg=aVF!u4d$?gvqD`^9^T2!%;Vvo z_eYLf5}brywMKpUHmobrF+VLHVDwIwA4iFn_)02IeJ9CmdN>za-q>QZ^a?72a4Bt@ ziR)4GXs)57NuffZK9=`tx^%^Nm+DoA2U@=F8o!^Rt(R5>DaUtUM7!P?_8sZpZTPh` z%nEBRi=JeP?$U(7W*L;EFiG_5$FSR9i^l9eI%JWhTGvknyZ%EQxSrH9;#z*B<`Zu7 z;igXgy@!EcYWxSUOYf0K9A zU#Fj=Ui)}Dq*FUprRHRbWX1Q~HOMxPgubcGsxy#>9$~TB#9mmCglDxft3T~^| zQ6PVp$_*SIKqn@NOc~mB>m-PJ=9bCcYrdFkwn>q7It!4Wv`dQO$TrOpy;c8V-_n4;85lDM23YF(Di#Bu7#8brXD#X#i4l3#d5{Q%)Kh#YjRX4) zUY83dAiQvuMLn9BMwhbDps0?RK+DMVfw1>K?SiC~@Lk42NsQDkqoqoGyKlIvd(`<9 zFET@dsg=6d_#mi-fI=~J73SkOHfi#@_RGss=os934IL?8*(vmBAkE++re#Do z5@+@23DJrgSCUP{COVoNDIx2xj=K#5_+eHnr7XWpceSZvzV>(}LF=tRXVJvvE!C*j zgMpnvn|=9Z^ZyB5XnzDWBP<38rYCd!25(zi>>m0G+#{4v7xy>ftcTspY4P%2;+lP!yD&; z1r2=onB#rEo*nXu4@C>{wbg*n)m|0reJ%-mAyUWcV1fCduG`HzpYPAx^;A-K$!sp2 zsT82En9YZ5XO0Bta+koO+{kZioKq&>J$~VG=_W9yVh(_{3tLc$BeF6Uin9RFT!uAc;pzL@}j+vB#sKRcCc zZOZ68xQ^)S@;7<&)h|QHwYq2yh>%7}QK$u)d`-XpBX+mbaEk-E#m0N0SZKJiYH4Wt z+(J!GPM^2$Z}L2@Ij0{Xa$j*S4a#BVSeaIXVu~Qz_pfgcT0VXyz)D(M`Z!p<)aBXX zJPiaheL-bLUOHSf$bJF{bhdCqKjS2M71segu~I(1mk0lD%O3cdK3punw`6Cv-{GEl z>AIsDLFcGQV_`!lV>#l(&p?`<#?6>T->-sgI}>dUnIs@saGJ|rT^zOe=Y5!7ajAXj zk8f8GOQ6#RHx~K`l8aJr#dWZ2sjnldbA&;fK@-!LoClW*3^!Tzhfoe~(rX=0F_x*POTIT$UHg0eT zsWY=~Cq7Empmj6rcC_3twb{!970(6Q43G1jQIi&`5is_D0 z`|K3!JRE9{T44C!V1YYRBB=v^qPnn?YnfL>5xr|Jw99|utIw6?@BuvvMbC`GvNd!& zo8*DnU%sxY9^dY@WJBMuQ_)L5knxBgSnOm4j@Yo|!o1|P^cREtSUYq5Q$?Q*NcEY9 z++1gY4)%tbF+jzSEKX#^4vRBJk|=r$QG4*`)-SY631de=35HViP9g&Y^$ZMjj8K(sDj@^H3zM9N?fKrSsuCwa!Etuh8xtS z{UYPV34i%@5ucC%({e&Zb80*4fwoS5BHo4QO!g9s7gdc{z%O-mSJlaA#(MO1aSC!B zuxC)Xhje8fZZ+dkmu*rW{R;6$U8b5k?6}HM{#4LnVC5MUI|fle9%-lw|2PXq)cuc| z&^i)@y!&$lYN_~Mz0IHbD~esgDtQnSjmzOTC~OEJHlT6QkEfj!{c)ZOXI6X|R(>xW z7PV4w>{vU-SHtUhPT6!>i1Z@vTv!q-TzjoYR?ig&8r2LedhH)e>|=+Y9sdQ*JFB?e z4Ai8@ZgYb6%Bv{CL1k%hWI6qznBK2L3l^am;p;qV$FUffjL8k%NAvIeU+6PUGyQ=w zv!vke)@=VD1M2>*>in0+wmPEHPVbjc1{260As4O2ma~daS1k6!A-mtBKhIE$2~424 zutL|KNi%xrdT6zc*yq14$*k<8+>G=dD9@3FCb)riNiejw5AM7n@y`>g+|;veW}~$A zF3+?0J~hEOnpArNiu6d^v)%?}uRM&VxH%X_jmO`9)xW`nhXvEI2D!$TJ)d?XA1INUd2?0=nJma#;6_ETZKg-spaGM$h!muTFn*$ z_CMLZtA4uG8ybd+D?eO(f1vPB1-I$91QPN|%L@94B7vScw?`R+tu)Z_6uR};0lgGH zi|K3B_YN7Mq(seCqmIz+LkFJ1Z?gwI+XUu5&q|qYHOu{RcBl&#F(4?TV}(248&vS} zw=Ef#6br$xKQQQSfR-9)>~RMe7aMjW0JkEySnoNqM=GL~yJz7F-HK^uxx@2yw^)1( zsakkd`lNqBR9jGE5J8+hT=JPGXEES@dTd3_+ukQ(092pf7TPI!?nX7x&Wm`r8;MIQ zK~{7~qo%g~`#Us&ET0CsKpi47)%dbC06QuJB@N^NLI*yEW&C2;*spe6b7*g_t0bTeR>OJQ5#q+3}d%Bh9u1HF&d#Uf2X7V$?#32Hs-Kh_) zOjilSF&__NofL0(>jGGN-o& zs)}3lvsQ6DJUW`wERZQ#rp(BODqABX#v1$6E=0o7ioS2E8XBtz_xz(fb24Mf{)WQK zLJ%9L8`P2eUtTxj3?h^OSF5`?>PGsP-=LNSU+7uI+Z=L2b;)(*=!mb?2@faQMGx#E zen7C!il+9}6ea~=sNf z4UH^t5>|{Qg)7o)2M<;GMCaX!2BPV=_eoKYhH$#0Ijw)P5Q#0sXL_kNHRb61MJi{W z{RP7rA%z%*VLcmxpt-EpBd&hogRU60+#mPnQ58Aw@Rh2)*_$=u4qS^4V3=C&`w;qhd>{(k$#-2QtI;F@Bsdw_G946?EA zRkLE-n+kH9?8vtM&Q$!*ux(8ihQO=P+&8bu(`PhWJ8;Hcx)LRLYsf1VhA`lLhn3&r0S<@?9Y+;2C3uo9XpMzGkB>Suew?ng;oPmT!gZ@l1uD9|075pig z!YWgtkLUA4wAlj4t`k0W@0VPfD>{?5UPGpYnC#7K@X#{4Y_!r1?W@W>^N!b3dwV1g?WF0x3RwGLoxV0TV# zF3$Go$Dewo>mAX#Kd;SH{#Y0+8~x?QZT{*|{(dP$G3@$Srs%HzRksuReqWwiYbbj} zhC;5xw4cC#K2X7$256{z=!SH^l7I3=@lwoePIf;vsdZ2bn2A!>FKtBH}h$Gn2in?q6rIJKqQ31XYwge7^qg0I z1@_|5;sWUCEbK`DosoONhTH^C=5cq`N2*=a-kmDekep_#!Iq zAEF@bv}&TEx^985a*40RnK#G3!m|GHa{h5oCJXt(2*~ES2&yq45ecVuIblr_bK!k( zu#+87WJ9P?r!@tjVKa7&h(oIwC1j}wb()dZ2cOJ>GK%OB&P+tVi&n8L?ua?o-2HHu zd9}TUFcs~ULIj9<86{RAEtiw>WScqLE9_3SBbJ}I!n`~UyDIT(wma)ayGM?Dh_`5M z`@k1UY=Wcop2GdLux#{iEAYJHTVZrtXTrRF-#&oY?6i=+4=^y404#ej-9_Q zN!aSz&^?!0dvLCi6mG%tQtc7obDF|*G!+;de}CF1_s~`1epT2mSNADN@-C9C`W_}+ z{L2)-j}gWHE~2(_}? zElo$*ra0sBka7g9=)SGK;0tJ5y$f;7T>Ky(FC@Bs!TP&L?X>|s&f@8WIr>3m_AQpx z(+@v7!@f+E@tHofm$)FG9p>dS`ZS2Y4DtC4P5vqvxRx&FtCq)y`s;1@3k^us!DxAB z?H4NN6+cxW*!7pjusT`qMxWi~C|clWRR|(xu3(Ecz1Gz=hvj#*7wU`-(ewLMsM%Mq zpt5Vym^yuiu8xo@ulnLLFdE8C$Rvlfu6exH%oDcwDSSJCFq0V{ev~Eu>)UEyyQo9K zQ5rc`9)9Yl{)eaY_L85U-r&;*7w!GVmJ*ieDqFORvR;Ues;Dl7M+V@Tfm;~ zCzUDLt-4du)}o>bxyZ)6795&bEu{J_ot=9mq-$iY3AqQYu$O7ZM9eWtD5ACTgk!# z(^j=sCW@!iih6}1UGGZl9(%(&HroBAGNaq7_~*x6Gq_KEw%ZM<{ftysgvY-)KM#`= zsX`uzgtCi;u2?M3jy@g}-W4?s@2*p5=xh@vG!i*ZX>s}6BJMFt(vu1 z*3GBgMb49!s;3%kZv+?J37;&PTf>XW*dJaR&FVed&QVN_y6E2yv|T90>KJpCgUYT4 z^$sfcKowO^mu!`NkYIQP8PEqaD-&B;hAJRet82TQFQ7$5&Is+qdxd^Sgj9BO zKSpNOiFV!Z^gUfIty)sKh{Be#e?g;&Oe2`^bT#x^h`yy)`hD)lq&6bjYAEd}Jt~pN zCSfZM?kIj;f}eMbbkA;V2NnMMcbp0JBqhEFUKSFjTSA>3+;$_x|ESUwrUi+`@|p@8 z5Zx4}{)co1BSa`*DQ2d)CjGWxhPIMVFnYn*$IaQ<>B?RS+or7G8;KH~ndv$~dNXV9 zz~Ue=7G%^l9uwm{_Uzkx?<**~$=|D(4}1f#*uoY|uCw1%k8jpkWog7^kElN|!@yw5WLKal{oN#z|(^1#zPyv6ni1nJU z>dq}GcGP_cX1oE5%q8H>uI6NlYFDUjr!YZ?olDL#9!_dGi=5`Op}2 z^@@?QB^`bEqLp2l7_-4X)qhuKyrSy_iyz3zMh}ltE?IMbxK_-VYkQlXO%klVvC;gH z?Gq^E&JOw7w|9ehYd1Q-Sk(B&y=Zz}ItClR#ai_B_tM z3EiFPDbk;`i5>x8w z*xqJ&ra8CSC2Ca7Az`@1L01&NvxqcvoRfXYxjIfPK>rE3)gc|P-}x{R%eoQY+a@H{ zE`9tl*^)XvNU?lP4P6#XxcB`=>*z=yyBLAs`t_Rj3J0RhSG}aSyAK6Nt&5SPf9#G{ zp3-_enSSuaHoo$Gp!Z+7FCQR~W`-C#P17}lFmD@~ku02Vpa7gm3ZZ15MzY0)t94bpFhv+p&$BGT$w##Nq z#mqd>qm{pHFehO9BqodQZ2;C4W&T#rtAYHZA4*ZM9>^{06~|x+z~;UIyN8M~fet>z2krZ}BJ3v?hxYo;e-1R8u>-;xqD>ZlaZNp*zuP2L-O|rgA6fBtM8eDUoSts8;!k#VhsUF*%izsriD*ArSscAj zbgHirR9Aa^)04egC(WoG*4$Fu;tHkZNAB>eRh>xHJ6-zQs-d@DIfCA2ZqtV4A0R)~ZELh!r8^{w*! zrhXx=InMPND|yMk|GevPrmE>>_2uzZ+6Z;a6>6VaBZyzF)Ho!Siwib(St|cU(Hw8Y ziQ%_u>o%66H6vqO1lPY!TvpWwJ~_{agb5OUx*~5-S$>p25utmuYt-Ud4NuFkq?uIY zpbZUbegQZ0BY8d$`5msfk13>QkD4F0I`UgYEnknLX+Y3K)2%s z3|Hi9O+Gf4OHHvmfGEhm)l7RTNIrg{f2iM(*sCEz0xKvoXDroAN;O|GPpaRj>Pzly zaZQcVTSD=Jp^2&i=we!5Ny&Fk{rYWD?f#>_TFk@Ekp-o;NgiX>aN1~}6+6-rgAdfj z82uSs#hR2{BpJM_GIM&qxQ$N*Rt&$bFW6r!`M9w?>)@^iHH}{fzoM#q_z|d*cjuS9 zM*HE;j@22kqq3zHWxLia1$?UElesf4lIQdsMyhaTdyAxnj9O{3?wz@0r0x?#o3WK< z`AJ2ik}DeOw|>D2?1?aZ#YoQ>PFBl-J*aa~(_0kzW4(wG^^Evo)^h=Z>;)7e=CvBh ztG@aQtGLY1HtXzIb)Q#BHi#jkk@QezVS7I7w!Tr)PCnUryI8r4&YAK_kRv{TJ6pXg z!AR!adb|8%i@X9T0_KD=JCDVBos%=3Q|Roo_pH5g+Dvq%Il-e55;8N(!H<@Oe5^`& zC0bY)$@m$Ex+mEe^XdHOHBYW30!1#mU;lgw#9H~4OTB&p*AuQvpZ$Pf+CWPRynsV2 zb==&yXtr=K6{h;w>$z4#z773Mx$LU}oOQ^}=;x>? zTVo%P_k7Fhxc~Fl_aIRt$YH`f{yh!sxlb3$LkHrCLXVwsrTiUn%ee{oC5-gUp?JxP zfs0m(lX*UCMO&>RT^N0$xDRHfhb;4?d|K|fd9=)$f5Ca*t-EYU3`AnP z9gU_yOY`A(_=Eg@$dZm;n(Q%0Nm;HI%DVth&!3`XQ z0-w?dCEr5&@m4ubIwvJvSSwcYRehl|5uU?JNqQf9LZ@^FOjesrS_c}{@a39s<1O*m zdTL~ip&DraK8*k+ty9uNDP4-YO6ht7tBQk&yBoDiq&QHGj7 zp`X930F)D8=}Ewa(iz21^k@;4A>GMEi#+xuP}I-IB#*TQ++gi*KK< z=EdjTIR+gix9a%AZ*G|Zw4iaMU@N^}>!U&Yd+PYVlSJ6Dr!n6UpEsUa;?M?B{3T+B zcEjL)>WN!W^L>oZb|p&C1R~AK-L6ukPxr6IprrrvH1b5Z{Y%md9bQQB(UL7!>36Hq@3M~W4KYweB zZGbH@d1=xyt=23(;7v@M>ti${)3ie2p?C(>73C(;DG_~)Rm9Bf399@6;4lxbvU6qp zeUAlciv<*eSU$e~IPg5<49pdq@~@fT&OB{BZCh#5{HHhf<&x*}Oa*$TyD|PFK19~y#lT))Xf)~+l5Lp)@a_)0YSHnfa#rf~h1O!?qZulE-M%Po!Phe8K2cHg1% zr;7C|1Uo?8sraKxBgS7mj_-jhPjl%`JiaKlEb1qJ^X2f~%ChXXu7D`Pq(BZS+*0HX zld;=X2?ZV)@1?p8><{zouZEzc87!Q<3$o+(y7hd@{}A=pCU4AF9XICj zHuAUr)zr#;NjYfs7y?RWlpn@h_at!vj^AMEpPZRgH`KRx*O2_S6&2U`CA#-C{6 z$mbm1b)_b6-mqvm{nPB{VgnZ@A=`0tz!{36!8pV-QT7> z{GYom*K1EYsb84~KK)IrTF9{ksXCHd;Z#dROA z04qnPz~lA{5TG|-lbM-xd~ETa_h~s*lEyM%iS@_+ZLK4ZD7QAgA8a&`=C$2*&CQ1y z2WW$D^ms)KspbFaO5j9#xI8mc7jxw?Smh1v?#_2>7rB!(Vl=JmT@?0r&|iZ}%4I6a zkc!7eHgVU0_-GBWcgfJO;2bkAbO$qcNxmFVJ$N(^aALmf(zoZ719(of2eZ!!8aocM zg_yJQ7mH#h#uZ8m2Q4cf4D+-LcflYZm2A$QTATLm%g?a#8yf!XCrm2co7Eo8Gf<2Q z_{!m^15?|+C1m_tZQ-}H_TZp$O4n) z8j@+m(UphTfFOso$N5(4rK<4;B>kEtF#@KlZ>0*;LehRYS>+X4Q*WB95)8e6U6Qpt z&5YLC8mgj_j#zWXpTBzFCa=3tG3Iy48ADfBcm-uW1?I&N zz>FJ~h;1KR-M)R~YcY)E|6cU4N8)Rk59s_hJ*dC(5@rKmZ#ji+toodFJhyBaRvo_B z*7pRj`=$Y-dvJS2(AO|S8#>x^z!z81;wDt!LeE>A4uMBLcZE@ZN9&n>BQ#n@}GYcT{_h-GgAx8c=Q$-76!(Sr734XY|QK z=j216zG4E{`-a`I=L`pcrENzJ#v!E&v(=-))z`Ohqnctl7!$n@ zFyjj%p#$k8L4z$onGvYSO7AC9;rr?&wf7{s^OpgISa#X676MzfAMTceC`PskeFi3$ z$)!~g!NwHD^aVpW^`;A%1BK}NlaS*ySyj?v!HJymss&}J1Bi0IM>CTmuw4+kGU9=&0z&TKtcclca>op9vTQEne8C^LK|YkR~%2?Z+qO!p9faTUhk zyUph$1lq}x%GT%p*6&bMG$2GSaF-xoq&I$W#~$60B*Uy^y`rMX+}V}6!^MN`2KVUd zC6_n@*pWXnrI}X7#gVgzcrFOGmX$zjjpkmwaH5+@5A9^%zMmL|>5VGZjcxIC()m8j z?B>~QcePbuBS$x{cXN|wDl!BM(OxEbI701SwSdA*>>K{c7J9%Ct=S-~Q$)Iew6v}L z(O+$z3&%FsF}^|@#fLuEW#*i&1^S_@#IZ!sgLGW}Lv8~LGymVEo7bQIUwhkscYa*@ zzOHmbJMZFC>%08He}QnYix?XA)EloYf&Jj>ERzV;82zXbT{e?DHRHG7SnMa@C}q4- z_9a2UUpb5J&B`_Xpr`(Z*00mBLYbW}(weL>A7zoYahNX@oHLy9(Kx~QlVi>X=Y_dF~O7uc! zDZiN5j_$|>s&F;*stT6n=KDL4JQ9_Ib-fv`_>N~8r+vhPTDdd?I4eVM`xa_|9BHj( zUs=F)NfY8fDxCk=uV3WFL}urp^-+u*<@HkQEu&XT>?TU)Id~3}?Z|l}%m=9sQnXXK zLxpnZ4|vn<@iqTm8Y?w0XRWc*t?KkyF)eiFu;Zz*f(aK?sb?)cD^q!yLg1`K8}E)` zY>6>&LC*8G22nS$^PFwb+^KC(7b0i%HTc?Y05>Y$I;2=r)tazf@(MAXw6aD0wVVzi z!n-d0{E`81#k>30-WvYi%=C4;=yp(6X+A!cE1ngE)@P?Qo@I;)5Pq~PB$(CV=nOv;)M%H=hmt~V!R5VQ!#_`dc!1{*__u(_>YV1z(MiwZ60@nq0VlGk9mzhZ` z87n#4_@1d}WTezW(NSc3f4ao~@y&Mza5HY{1J^DzXTc83UdLU4DhfHwuwswzomevuP5I(#Z-k}Ovz5#hMH9X5G0^W3UY468k!vjF$_t&7vCW>`TYUt*%? zee;&0R|q(>EG*y6tv8WxE?@kx61du<%?q%fm1e9nKtxgy8E(uEDIU!bSWd~pE zT?-!#YZ_iz(0e{dOV49>1vy4eE$JCQGaQVSUVkr)@ljy(?B2bb9bXqV4xDBBxuda< zXk*CXCkpP6C|lvFQWUAn=ykoCfFO=;_mfPmIc;h{Lg?RIP>^`i?wsut`8e=hU~`1R zY0hVT{V_vwfk{qPZ@qkKr0TuJ7V&GZziP#*Y9-x>UJBF~7BdOi?IQMwe^4xaD;&6+ zE|%QN`$NEnuqkzVM()VxnPZYk#40(lu5El~%0N4X??Fp(WYdr5j*zeh2Xa)eQHiKv9<9lo{TQ>A zEc<^VF^f5qCx_8@XOEj@5_XbKR6wLV-Mtpud7wPN-?=svXkkUUo4T)TLp zGb4+WC;SK3^B(mIQmCNnkkYr^2rE2YZxx*er9IT~b07K|H-P*cl`^b4x)46vc#L!9 zVSY;&)zaR6Pq1Ow_3r8O1ux4!Qip*((cAb=4_jnA0x?+ws`HlhfioMs`})ehd0Q%M zZ)Vk~{?$bEq=<->c%N-v4;fgxJ!BlPavSYga@M~hn7{}_lEv_eMNT21doPG1+Xb5u zI+V-$T8Slj6QH#$bAN&eR>acNGqy%j`WojEGa;?&aM-M()vrPL+p(ma72)WOf1hKe zZ}sT>(Ql_E;}8Fgs)yr_pf26aRg-}Csszu^9e>PD`BUR{qezw%s^E3g1@NCA_rYS9 zBLd2VL;EI(?Zs%Wcj}n z79QPYYZhO#nmv$_w0=t;#L$ffHI%K9d@ymqQZt1Tx`#oa8}#SkpKOCOo+qLoBSuktUFo^+13IA)c8j{ymxu%d7l!^w+T;3G$VaYBg}T3G z`aWp7=Scr$L&YltQshxuT zFk1kG>&I7KHz*TenMw61`+pT)wG!i>5J?MGky!&PKSz$c?^Xyv=nLLi;Qql^Fwa@c zg{U8&l&$0121R??s+u}uW4sb($6^~*?u^xD-b#NSnD^#rS%|YyMM*!}#<@DiOV=lU zx5M!8P~$+;C3Uf!GKo1;uI0ap!M}fqdw+hhM_SZ&5RNzs9F&bG^q}U&WF+-4y8-4D zc3N|?l04{w=#%H3v);n|T_b_kI?4YDpr=RzJSE378J8esRkw$3ODmtfi#h+sjyi@; z>;qHC14~fl02W0XZYMF~cQ9DR&L7>YDtKXm_0yiy9eUB_m`HzI47JS5CYygXpix{c zGb5hE)wA23UnL}dYn@+VY8k7*5Ulb)>=5Gnt48)Nv`PF)F1; zdwye`Sh}cx-{45tEPbr(L{4GM1mhz1M?JZKzweAL*UfP?iIX-(T;J~LZN=^tl&(e% zJjU%dKT%ndUZrsBq8H|BzrlM^3U|hon!hG+ShU zP**FH$QtL{m?n(Q5dB*h6qT|BgvPhHd2Bh)m}S4z#x6q8hJf&>%D5U` zBJ%VJq+8V8=lgvXJS_C|$9Qs{W@LT{W999w!c3g611^#!Pb+xxi1xNZ>3Yjgu!AW^ zAs9wgP9Vc!`PYZxytqUE+#W|>i9$Xr~KGJ_SSbxNc& zm%mR$9yPOyeFV2i7<=hxO|4YvW%k>L1YNu!d7Ic%UD0*U(@PM@JQ5!waAd1of7}s! zeZc9^`^0>+l~el1T2YZRx5Vv*q_t1 zmV80}tg0qz^swI`-q<>*FJkoPg@$O)^XFj3O8J=t8y57_G;x_U9k=)|DuQV81?fT@ zSsC*J=1$c)%@YZqWBllK&;19g*3BX1>yow#`*8!jq)6qr&1xl(>+Ipf$(XTrDhgUv z3Y5rSrc$E|bZ&s-CaxaSLFphOF1>Go4x z34N@~bSl_7wDo6>0gv`Y>`pv1o*(+f72$?8f2y7Oc-RtC{2*)qHQG5`^2u|)!DsbR zDPy)P>XAW10;FUTQIPo+bz~3mZW7iw#ts@%0EM72k6W0+Mv7Q|si};c;zNuuC+*PhnN^gxq)$%S;9%U7ceuvJ z9dr{`@%d*}Rlq01BgGTKyH%@&P`=x*i9r<%n7c5h~`h`R1QSo0OUqsK&UWt6}*0td@Z{P)t5Pun(;%*)~ zbf_fJzS=jWSsmHs0VxZ+PMe-yGY-{muq2kbOb>3uX8Pt{gnR^Zx6YmG?~hR1Kh(sb zpD=23a)eprqL%fqZ)1*jzg1+%x*QjfaFh@`)VO5Yx8L* z{5KjC6nCUpLOc=YED-2qwV|#=%#Q%oqCe|CfJ9)m-E&V|;`DVn39^n&U@0u(@s^61 zy9{m8T98t8KR2~F^qZR7O)iOfMyVE5sdZjX;w9ppwq_I0z&c~P``kv*Q#9MhFj;GH z0o$(_5?!Q63hT0k;OhCyE+1;MZ2J1M<-KB~ZSi0Ib*X#?;OK<+y!|E^A!hvOmamm< zqcEJk!uk730|vukF^h|P^MMh|$-j{)?^AsLHx}=IbCAYEyd(^F)+&6Mm)PNb+zIt} z{WktePo+l(+vA#YQcI9RLk-bo?Oa~at5WkCCwRqpsdxY43fBNc*)}vuRbH3mhEHLl{dixY^0t?k0e&Vlb7mvgO>I@Lb$#t~ zg=sAnU{=Q*@Y4d0Ol*4i+FTr&(9aZxV68r(iE8X0o^OeLOA308{oIDhT=FVGnY&8t zz?u_Wbn_hil%r4c2ULuJBkMJ{4k{Q+;!GE8 zsM$u>qc>7l#2ZOK;seC=(fYNE`YabD%`yaKx!Z`y{%z>!t4jz@7)^0l7-BUJ8;*qT zky?8ljiFpDzP5FAA(Ey4OoIznb57Z+9pagxPs4D}ko*H-Lq}b22xT2ZKo7*4KlMVK z=FAB>`Z1%|(mB->^-Wxhv)R&e{I3NML4K5QmL1VH`v84TnxXk@3yU$%*!}!fxOpNgf1TqOC|*)E`n^ zccS*;N2pYC$=a(?wM=pTlB757C6j@PD9*qjcW%j1j$pY3lXy52-38>n4{`NEF!U?k z(>UkZg!-BWR*Ak+R+2-mi^pl(=#2x>&wfh;>Mb6^b5=v~(>{$34uF3P@QhWXHh?A_ zx^2k_1KByv3@j)OERW=z*!9^%_sXBhV zb6Z)0-!KR$bWR}&Wr5}o5dPG_0Z7RXsBA%R4TsZ-77|$D_2TwDKw^{6>3SeLODN(3 zWyu`b8h_g{SAH;+V)iV=>vuvN8BP$qH!yPXAqq+=xwh3FnK~fas<7ZTMSt!xlS*HA zj5;4E(RkzX{z}vdK+rzcJsxPaT#f4)f{%vsz7%BgijbX{&bpF^C~V&Z4~=bF{X2!V zTd>*=uKO;6H1 zP7SmvLmiKK-`I{iNgl$=Dr&elWM}37olOogdE>Py^>+3uPiNZFY#a@0E`1}IX=5w9 zji?rP&5islO}3v)lkpRU8pZ0Sy3C9Y2}YUMcSpAkbDQuHpGty`p_WPSBhv7@&4vDA zMKmEE7IEk{iUKu4ZT(uQwx=^&uNk)E=Ic(~s}G7?)F^7sAcZRZ^ogR%Gs?X}-AfKe zf83|^HGU9#J8CeOiO>+NG_p;Z&d#w7CnNqsmrWW(o-&Ek^`bvkW3vs>e&nZ<-YlAF zb(Rk6gDqzWO(zb|O{A{SvQ~))9R}X&tjLYd zkNx<9t-WmBhUR|@E5?|cvWNA->@^&>kLJKx9A!pEg8Wp9KDO+`nW=hUl?NGwpIU@% ziyuQ!Gv^j;yB18KB#8f@D66Kp)+${02`aS^)dQ|3Xp^`OhHH ze88d=eQlujy`?bhE~d9?e>rcQHkY*wxkBqv%1f=nk-Y*#YToql$o+5hv~I%Ad`K`) zJA>D(89>?O<6Ud4)S z)}{PE+UqP6&nSpU?Rhrq6U&=UoxDmlr1&s3cA5bkFkN^Mx literal 0 HcmV?d00001 diff --git a/docs/conf.py b/docs/conf.py index d1bb09ffed..07d7aba416 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -69,7 +69,6 @@ "getting_started/custom-airflow-properties": "../run_dbt/airflow-worker/custom-airflow-properties.html", "getting_started/docker": "../guides/run_dbt/container/docker.html", "getting_started/execution-modes-local-conflicts": "../reference/troubleshooting/execution-modes-local-conflicts.html", - "getting_started/execution-modes": "../guides/run_dbt/execution-modes.html", "getting_started/gcp-cloud-run-job": "../guides/run_dbt/container/gcp-cloud-run-job.html", "getting_started/kubernetes": "../guides/run_dbt/container/kubernetes.html", "getting_started/operators": "../guides/run_dbt/operators/operators.html", diff --git a/docs/getting_started/execution-modes.rst b/docs/getting_started/execution-modes.rst new file mode 100644 index 0000000000..580a250cbf --- /dev/null +++ b/docs/getting_started/execution-modes.rst @@ -0,0 +1,106 @@ +.. _execution-modes: + +Choose an execution mode +======================== + +The ```ExecutionConfig`` defines your execution mode, which determines where and how dbt commands run in Cosmos. + +There are two types of execution modes: + +1. **Execute dbt commands on the Airflow worker or triggerer.** These execution modes offer faster +execution times, since no extra container needs to be spun up, but no or limited environment isolation. +There are four options for this type of execution mode: ``watcher``, ``local``, ``virtualenv``, and ``airflow_async``. +``airflow_async`` is available for BigQuery as of Cosmos 1.9 and ``watcher`` is available as of Cosmos 1.11. + +2. **Execute dbt commands in a container outside of your Airflow environment.** This type of execution mode offers high levels of environment isolation, and also allows you to spin up the Docker or +Kubernetes container in various cloud or on-premises environments + +The following diagram shows a decision tree to help you select the right execution mode for your project needs. + +.. image:: ../_static/execution_mode_map.png + :alt: A diagram illustrating the details about each execution mode in the two categories, "On the Airflow worker or trigger" and "in a container". + + +On the Airflow worker or triggerer +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +These execution modes offer faster execution times, since your don't need to spin up any extra containers. You can also use Airflow connections via the ``ProfileConfig``. But, these execution modes do not have any, or offer limited, environment isolation. There are four execution mode options that run on the Airflow worker: + +- `local <../guides/run_dbt/airflow-worker/local-execution-mode.html>`_: Default execution mode, but provides no environment isolation. Run ``dbt`` commands using a local ``dbt`` installation (default) +- `watcher <../guides/run_dbt/airflow-worker/watcher-execution-mode.html>`_: (Experimental since Cosmos 1.11.0) Optimized for execution speed. Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. +- `virtualenv <../guides/run_dbt/airflow-worker/cosmos-managed-venv.html>`_: Allows you to address package conflicts and an inability to create a venv at build time. Run ``dbt`` commands from Python virtual environments managed by Cosmos. This +- `airflow_async <../guides/run_dbt/airflow-worker/async-execution-mode.html>`_: (Stable since Cosmos 1.9.0) Optimized for worker efficiency if you have long-running dbt commands. Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ + +In a container +~~~~~~~~~~~~~~ + +You can also execute dbt commands in a container outside of the Airflow environment. Choosing these kinds of execution modes provides a high degree of isolation, but requires that you can only create Airflow connections with the dbt ``profiles.yml`` file, requires a pre-existing Docker image, and has slower run times, because of container provisioning. + +- `docker <../guides/run_dbt/container/docker.html>`_ : Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) +- `kubernetes <../guides/run_dbt/container/kubernetes.html>`_: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) +- `watcher_kubernetes <../guides/run_dbt/container/watcher-kubernetes-execution-mode.html>`_: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. +- `azure_container_instance <../guides/run_dbt/container/azure-container-instance.html>`_: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) +- `aws_ecs <../guides/run_dbt/container/aws-container-run-job.html>`_: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) +- `aws_eks <../guides/run_dbt/container/aws-eks.html>`_: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) +- `gcp_cloud_run_job <../guides/run_dbt/container/gcp-cloud-run-job.html>`_: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image). + +.. _execution-modes-comparison: + +Execution modes comparison +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The type of execution mode that you choose directly affects how fast your Cosmos Dag runs. + +.. list-table:: Execution Modes Comparison + :widths: 25 25 25 25 + :header-rows: 1 + + * - Execution Mode + - Task Duration + - Environment Isolation + - Cosmos Profile Management + * - Local + - Fast + - None + - Yes + * - Watcher + - Very Fast + - None + - Yes + * - Virtualenv + - Medium + - Lightweight + - Yes + * - Airflow Async + - Very Fast + - Medium + - Yes + * - Docker + - Slow + - Medium + - No + * - Kubernetes + - Slow + - High + - No + * - AWS_EKS + - Slow + - High + - No + * - Azure Container Instance + - Slow + - High + - No + * - GCP Cloud Run Job Instance + - Slow + - High + - No + * - AWS ECS + - Slow + - High + - No + * - Watcher Kubernetes + - Fast + - High + - No + diff --git a/docs/getting_started/index.rst b/docs/getting_started/index.rst index 59e07698a5..586f2e1dd2 100644 --- a/docs/getting_started/index.rst +++ b/docs/getting_started/index.rst @@ -5,6 +5,7 @@ :hidden: :caption: Cosmos Fundamentals + Choosing an execution mode Similar dbt and Airflow concepts .. toctree:: @@ -45,15 +46,6 @@ Execution Methods For more customization, check out the different execution modes that Cosmos supports on the `Execution Modes `__ page. -For specific guides, see the following: - -- `Executing dbt DAGs with DockerOperators <../../guides/run_dbt/container/docker.html>`__ -- `Executing dbt DAGs with KubernetesPodOperators <../../guides/run_dbt/container/kubernetes.html>`__ -- `Executing dbt DAGs with Watcher Kubernetes Mode <../../guides/run_dbt/container/watcher-kubernetes-execution-mode.html>`__ -- `Executing dbt DAGs with AzureContainerInstancesOperators <../../guides/run_dbt/container/azure-container-instance.html>`__ -- `Executing dbt DAGs with GcpCloudRunExecuteJobOperators <../../guides/run_dbt/container/gcp-cloud-run-job.html>`__ - - Concepts Overview ----------------- diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 9b7e233814..712119236d 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -24,7 +24,6 @@ Cosmos offers a number of configuration options to customize its behavior. For m :hidden: :caption: How Cosmos runs dbt - run_dbt/execution-modes run_dbt/airflow-worker/index run_dbt/container/index run_dbt/callbacks/callbacks diff --git a/docs/guides/run_dbt/execution-modes.rst b/docs/guides/run_dbt/execution-modes.rst deleted file mode 100644 index 261c8e63c3..0000000000 --- a/docs/guides/run_dbt/execution-modes.rst +++ /dev/null @@ -1,88 +0,0 @@ -.. _execution-modes: - -Execution Modes -=============== - -*Execution modes* describe *where* and *how* Cosmos runs dbt commands. - -On the Airflow worker or triggerer -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- `local `_: Run ``dbt`` commands using a local ``dbt`` installation (default) -- `watcher `_: (experimental since Cosmos 1.11.0) Run a single ``dbt build`` command from a producer task and have sensor tasks to watch the progress of the producer, with improved DAG run time while maintaining the tasks lineage in the Airflow UI, and ability to retry failed tasks. -- `virtualenv `_: Run ``dbt`` commands from Python virtual environments managed by Cosmos -- `airflow_async `_: (stable since Cosmos 1.9.0) Run the dbt resources from your dbt project asynchronously, by submitting the corresponding compiled SQLs to Apache Airflow's `Deferrable operators `__ - -In a container -~~~~~~~~~~~~~~ - -You can also execute dbt commands in a container outside of the Airflow environment. - -- `docker `_ : Run ``dbt`` commands from Docker containers managed by Cosmos (requires a pre-existing Docker image) -- `kubernetes `_: Run ``dbt`` commands from Kubernetes Pods managed by Cosmos (requires a pre-existing Docker image) -- `watcher_kubernetes `_: (experimental since Cosmos 1.13.0) Combines the speed of the watcher execution mode with the isolation of Kubernetes. Check the :ref:`watcher-kubernetes-execution-mode` for more details. -- `azure_container_instance `_: Run ``dbt`` commands from Azure Container Instances managed by Cosmos (requires a pre-existing Docker image) -- `aws_ecs `_: Run ``dbt`` commands from AWS ECS instances managed by Cosmos (requires a pre-existing Docker image) -- `aws_eks `_: Run ``dbt`` commands from AWS EKS Pods managed by Cosmos (requires a pre-existing Docker image) -- `gcp_cloud_run_job `_: Run ``dbt`` commands from GCP Cloud Run Job instances managed by Cosmos (requires a pre-existing Docker image) - -The choice of the ``execution mode`` can vary based on your needs and concerns. - -.. _execution-modes-comparison: - -Execution modes comparison -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. list-table:: Execution Modes Comparison - :widths: 25 25 25 25 - :header-rows: 1 - - * - Execution Mode - - Task Duration - - Environment Isolation - - Cosmos Profile Management - * - Local - - Fast - - None - - Yes - * - Watcher - - Very Fast - - None - - Yes - * - Virtualenv - - Medium - - Lightweight - - Yes - * - Airflow Async - - Very Fast - - Medium - - Yes - * - Docker - - Slow - - Medium - - No - * - Kubernetes - - Slow - - High - - No - * - AWS_EKS - - Slow - - High - - No - * - Azure Container Instance - - Slow - - High - - No - * - GCP Cloud Run Job Instance - - Slow - - High - - No - * - AWS ECS - - Slow - - High - - No - * - Watcher Kubernetes - - Fast - - High - - No - From 469df2c535c5b1bca09b605995f8cca58c138598 Mon Sep 17 00:00:00 2001 From: L Zdanski <25642903+lzdanski@users.noreply.github.com> Date: Wed, 4 Mar 2026 15:19:12 -0500 Subject: [PATCH 48/48] Add guide overview landing page --- docs/guides/index.rst | 58 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 9 deletions(-) diff --git a/docs/guides/index.rst b/docs/guides/index.rst index 712119236d..fe4a6744bd 100644 --- a/docs/guides/index.rst +++ b/docs/guides/index.rst @@ -3,25 +3,52 @@ Guides ====== -Cosmos offers a number of configuration options to customize its behavior. For more info, check out the links on the left or the table of contents below. +.. toctree:: + :maxdepth: 0 + :hidden: + + self + +Cosmos offers a number of configuration options to customize how Airflow dags and dbt commands run. + +To set up a project, you follow the same general set of steps. + + +1. Set up dbt with Airflow +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You must make your dbt projects available to Airflow and install dbt into the environment where your dbt code runs. .. toctree:: :maxdepth: 1 - :hidden: :caption: Set up dbt with Airflow dbt_setup/dbt-fusion +2. Connect to your dbt database +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Configure your Cosmos project to allow Airflow Dags to initiate dbt commands, and make data transformations and udpates in your data warehouses. You can creae these connections with your ``profiles.yml`` file in the dbt project, using profile mappings, or customizing ``ProfileConfig`` per dbt configuration. + +3. Translate your dbt code into Airflow Dags +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can customize how Cosmos parses your dbt workflows into Airflow Dags. Choosing how you want your dbt nodes to map to Airflow tasks within Dags can affect the time required for Cosmos to parse the dbt workflows and for Airflow to execute the resulting Dags. + .. toctree:: - :maxdepth: 1 - :hidden: + :maxdepth: 2 :caption: Translating dbt into Airflow translate_dbt_to_airflow/index + +4. Run dbt +~~~~~~~~~~~~~ + +You can specify more details about how Cosmos runs both dbt commands and Airflow Dags. This includes `choosing an execution mode <../getting_started/execution-modes.html>`_ , either one that runs dbt on an Airflow worker node or one that runs in a container. You can customize additional aspects of how your dbt code runs, like using particular operators that correspond to dbt commands. And, you can leverage Airflow's scheduling capabilities in your Cosmos Dags. + .. toctree:: - :maxdepth: 3 - :hidden: + :maxdepth: 2 :caption: How Cosmos runs dbt run_dbt/airflow-worker/index @@ -30,24 +57,37 @@ Cosmos offers a number of configuration options to customize its behavior. For m run_dbt/operators/operators run_dbt/customization/index +Multi-project Setups +~~~~~~~~~~~~~~~~~~~~ + +If you have a multi-project architecture where you have multiple dbt projects that reference each others' models, you can set up ``dbt-loom`` with Cosmos to handle cross-project references. + .. toctree:: :maxdepth: 1 - :hidden: :caption: Multi-project Setups multi_project/multi-project +Add your dbt documentation +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Cosmos supports dbt's documetnation capabilities. + .. toctree:: :maxdepth: 1 - :hidden: :caption: Documentation dbt_docs/generating-docs dbt_docs/hosting-docs +Cosmos DevEx +~~~~~~~~~~~~ + +You can configure Cosmos to improve your development experience. + .. toctree:: :maxdepth: 1 - :hidden: :caption: Cosmos DevEx cosmos_devex/index +