Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs/getting_started/async-execution-mode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -197,4 +197,13 @@ Create an Airflow connection with following configurations
The ``run`` tasks will run asynchronously via the deferrable operator, freeing up worker slots while waiting on I/O or long-running tasks.

.. note::
The deferrable operator is currently supported for dbt models only when using BigQuery. Adding support for other adapters is on the roadmap.

1. The deferrable operator is currently supported for dbt models only when using BigQuery. Adding support for other adapters is on the roadmap.

2. By default, the ``SetupAsyncOperator`` creates and executes within a new isolated virtual environment for each task run, which can cause performance issues. To reuse an existing virtual environment, use the ``virtualenv_dir`` parameter within the ``operator_args`` of the ``DbtDag``. We have observed that for ``dbt-bigquery``, the ``SetupAsyncOperator`` executes approximately 30% faster when reusing an existing virtual environment, particularly for transformations that take around 10 minutes to complete.

Example:

.. code-block:: python

DbtDag(..., operator_args={"virtualenv_dir": "dbt_venv"})