diff --git a/docs/guides/connect_database/profile-customise-per-node.rst b/docs/guides/connect_database/profile-customise-per-node.rst index 43969943c9..089171b246 100644 --- a/docs/guides/connect_database/profile-customise-per-node.rst +++ b/docs/guides/connect_database/profile-customise-per-node.rst @@ -26,10 +26,10 @@ Let's say the user configures the profile at a ``DbtDag`` or ``DbtTaskGroup`` le But that for a specific node or group of nodes, the user would like to replace: -* ``profile_name`` to be "non_default_profile" as opposed to "default_profile" -* ``target_name`` to be "stage" as opposed to "default_target" -* ``conn_id`` to be "non_default_connection" as opposed to "default_conn" -* ``schema`` to be "non_default_schema" as opposed to "default_schema" +- ``profile_name`` to be "non_default_profile" as opposed to "default_profile" +- ``target_name`` to be "stage" as opposed to "default_target" +- ``conn_id`` to be "non_default_connection" as opposed to "default_conn" +- ``schema`` to be "non_default_schema" as opposed to "default_schema" They could apply this different configuration to all the project seeds by doing: diff --git a/docs/guides/connect_database/use-profile-mapping.rst b/docs/guides/connect_database/use-profile-mapping.rst index 4b6ac2bcc3..edb490e372 100644 --- a/docs/guides/connect_database/use-profile-mapping.rst +++ b/docs/guides/connect_database/use-profile-mapping.rst @@ -10,8 +10,8 @@ a class in Cosmos for each Airflow connection to dbt profile mapping. You can find the available profile mappings on the left-hand side of this page. Each profile mapping is imported from ``cosmos.profiles`` and takes two arguments: -* ``conn_id``: the Airflow connection ID to use. -* ``profile_args``: a dictionary of additional arguments to pass to the dbt profile. This is useful for specifying +- ``conn_id``: the Airflow connection ID to use. +- ``profile_args``: a dictionary of additional arguments to pass to the dbt profile. This is useful for specifying values that are not in the Airflow connection. This also acts as an override for any values that are in the Airflow connection but should be overridden. diff --git a/docs/guides/cosmos_devex/lineage.rst b/docs/guides/cosmos_devex/lineage.rst index 4d9755651f..ae711a9264 100644 --- a/docs/guides/cosmos_devex/lineage.rst +++ b/docs/guides/cosmos_devex/lineage.rst @@ -22,16 +22,16 @@ and virtualenv execution methods (read `execution modes <../run_dbt/execution-mo Additionally, since Cosmos uses the open-source `openlineage-integration-common `_, it relies on this library to support specific dbt adapters. As of 27 December 2024, the version 1.26.0 of this package supports: -* Athena -* BigQuery -* Databricks -* DuckDB -* Dremio -* Postgres -* Redshift -* Snowflake -* Spark -* SQLServer +- Athena +- BigQuery +- Databricks +- DuckDB +- Dremio +- Postgres +- Redshift +- Snowflake +- Spark +- SQLServer Contributions are also welcome in the `OpenLineage project `_ to support more adaptors. diff --git a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst index b90545d6e2..58f76ad518 100644 --- a/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst +++ b/docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst @@ -41,17 +41,17 @@ Concept: ``ExecutionMode.WATCHER`` It is built on two operator types: -* ``DbtProducerWatcherOperator`` (`#1982 `_) +- ``DbtProducerWatcherOperator`` (`#1982 `_) Runs dbt **once** across the entire pipeline, register to `dbt event callbacks `_ and sends model progress updates via Airflow **XComs**. -* ``DbtConsumerWatcherSensor`` (`#1998 `_) +- ``DbtConsumerWatcherSensor`` (`#1998 `_) Watches those XComs and marks individual Airflow tasks as complete when their corresponding dbt models finish. Together, these operators let you: -* Run dbt as a single command (for speed) -* Retain model-level observability (for clarity) -* Retry specific models (for resilience) +- Run dbt as a single command (for speed) +- Retain model-level observability (for clarity) +- Retry specific models (for resilience) ------------------------------------------------------------------------------- @@ -59,9 +59,9 @@ Performance Gains +++++++++++++++++ We used a dbt project developed by Google, the `google/fhir-dbt-analytics `_ project, that interfaces with BigQuery. It contains: -* 2 seeds -* 52 sources -* 185 models +- 2 seeds +- 52 sources +- 185 models Initial benchmarks, using illustrate significant improvements: @@ -152,9 +152,9 @@ As it can be observed, the only difference with the default ``ExecutionMode.LOCA **How it works:** -* Cosmos executes your dbt project once via a producer task. -* Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. -* The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). +- Cosmos executes your dbt project once via a producer task. +- Model-level Airflow tasks act as watchers or sensors, updating their state as dbt completes each model. +- The DAG remains fully observable and retryable, with **dramatically improved runtime performance** (often 5× faster than ``ExecutionMode.LOCAL``). **How it looks like:** @@ -200,9 +200,9 @@ If your Airflow DAG includes multiple stages or integrations (e.g., data ingesti **Key advantages:** -* Integrates seamlessly into complex Airflow DAGs. -* Uses the same high-performance producer/consumer execution model. -* Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. +- Integrates seamlessly into complex Airflow DAGs. +- Uses the same high-performance producer/consumer execution model. +- Each ``DbtTaskGroup`` behaves independently — allowing modular dbt runs within larger workflows. .. image:: /_static/jaffle_shop_watcher_dbt_taskgroup_dag_run.png :alt: Cosmos DbtDag with `ExecutionMode.WATCHER` @@ -361,9 +361,9 @@ Individual dbt Operators '''''''''''''''''''''''' The ``ExecutionMode.WATCHER`` efficiently implements the following operators: -* ``DbtSeedWatcherOperator`` -* ``DbtSnapshotWatcherOperator`` -* ``DbtRunWatcherOperator`` +- ``DbtSeedWatcherOperator`` +- ``DbtSnapshotWatcherOperator`` +- ``DbtRunWatcherOperator`` However, other operators that are available in the ``ExecutionMode.LOCAL`` mode are not implemented. @@ -373,10 +373,10 @@ Additionally, since the ``dbt build`` command does not run ``source`` nodes, the Finally, the following features are not implemented as operators under ``ExecutionMode.WATCHER``: -* ``dbt ls`` -* ``dbt run-operation`` -* ``dbt docs`` -* ``dbt clone`` +- ``dbt ls`` +- ``dbt run-operation`` +- ``dbt docs`` +- ``dbt clone`` You can still invoke these operators using the default ``ExecutionMode.LOCAL`` mode. @@ -493,9 +493,9 @@ To override the default logic, pass a ``freshness_callback`` via ``setup_operato **Known limitations:** -* Incompatible with ``selector`` in ``RenderConfig`` — ``--exclude`` is ignored by dbt when a YAML selector is active. -* ``dbt source freshness`` is always re-executed at runtime; ``LoadMode.DBT_MANIFEST`` freshness data is not consulted. -* Not supported for ``ExecutionMode.WATCHER_KUBERNETES``. +- Incompatible with ``selector`` in ``RenderConfig`` — ``--exclude`` is ignored by dbt when a YAML selector is active. +- ``dbt source freshness`` is always re-executed at runtime; ``LoadMode.DBT_MANIFEST`` freshness data is not consulted. +- Not supported for ``ExecutionMode.WATCHER_KUBERNETES``. ------------------------------------------------------------------------------- @@ -605,10 +605,10 @@ Summary ``ExecutionMode.WATCHER`` represents a significant leap forward for running dbt in Airflow via Cosmos: -* ✅ Up to **5× faster** dbt DAG runs -* ✅ Maintains **model-level visibility** in Airflow -* ✅ Enables **smarter resource allocation** -* ✅ Built on proven Cosmos rendering techniques +- ✅ Up to **5× faster** dbt DAG runs +- ✅ Maintains **model-level visibility** in Airflow +- ✅ Enables **smarter resource allocation** +- ✅ Built on proven Cosmos rendering techniques This is an experimental feature, and we are looking for feedback from the community. diff --git a/docs/guides/run_dbt/callbacks/callbacks.rst b/docs/guides/run_dbt/callbacks/callbacks.rst index c179a0db5b..808f7793f7 100644 --- a/docs/guides/run_dbt/callbacks/callbacks.rst +++ b/docs/guides/run_dbt/callbacks/callbacks.rst @@ -13,9 +13,9 @@ alongside the artifacts created by dbt. Many users care about those artifacts and want to perform additional actions after running the dbt command. Some examples of usage: -* Upload the artifacts to an object storage; -* Run a command after the dbt command runs, such as `montecarlo `_; or -* Define other custom behaviours based on a specific artifact. +- Upload the artifacts to an object storage; +- Run a command after the dbt command runs, such as `montecarlo `_; or +- Define other custom behaviours based on a specific artifact. To support these use cases, Cosmos allows users to define functions called callbacks that can run as part of the task execution before deleting the target's folder. @@ -24,8 +24,8 @@ These functions illustrate how to upload the generated dbt artifacts to remote c There are two ways users can leverage using Cosmos auxiliary callback functions: -* When instantiating a Cosmos operator; -* When using ``DbtDag`` or ``DbtTaskGroup`` (users can define a callback that will apply to all tasks). +- When instantiating a Cosmos operator; +- When using ``DbtDag`` or ``DbtTaskGroup`` (users can define a callback that will apply to all tasks). Example: Using Callbacks with a Single Operator @@ -57,12 +57,12 @@ An example of how the data uploaded to GCS looks like when using ``upload_to_gcp The path naming convention is: -* Bucket configured by the user -* Name of the DAG -* DAG Run identifier -* Task ID -* Task retry identifier -* Target folder with its contents +- Bucket configured by the user +- Name of the DAG +- DAG Run identifier +- Task ID +- Task retry identifier +- Target folder with its contents If users are unhappy with this structure or format, they can implement similar methods, which can be based (or not) on the Cosmos standard ones. diff --git a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst index f731a9deed..6e1ebb9443 100644 --- a/docs/guides/run_dbt/container/gcp-cloud-run-job.rst +++ b/docs/guides/run_dbt/container/gcp-cloud-run-job.rst @@ -33,12 +33,12 @@ Prerequisites 5. GCP account with: 1. A GCP project (`setup guide `_) 2. IAM roles: - * Basic Role: `Owner `_ (control over whole project) or - * Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) + - Basic Role: `Owner `_ (control over whole project) or + - Predefined Roles: `Artifact Registry Administrator `_, `Cloud Run Developer `_ (control over specific services) 3. Enabled service APIs: - * Artifact Registry API - * Cloud Run Admin API - * BigQuery API + - Artifact Registry API + - Cloud Run Admin API + - BigQuery API 4. A service account with BigQuery roles: `JobUser `_ and `DataEditor `_ 6. Docker image built with required dbt project and dbt DAG 7. dbt DAG with Cloud Run Job operators in the Airflow DAGs directory to run in Airflow diff --git a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst index 7b7a8c8f41..11f0292a5e 100644 --- a/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst +++ b/docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst @@ -10,9 +10,9 @@ The ``ExecutionMode.WATCHER_KUBERNETES`` combines the **speed of the** :ref:`wat This execution mode is ideal for users who: -* Want to leverage the performance benefits of the watcher execution mode -* Need to run dbt in isolated Kubernetes pods -* Prefer not to install dbt in their `Apache Airflow® `_ deployment +- Want to leverage the performance benefits of the watcher execution mode +- Need to run dbt in isolated Kubernetes pods +- Prefer not to install dbt in their `Apache Airflow® `_ deployment ------------------------------------------------------------------------------- @@ -56,9 +56,9 @@ The following example shows how to configure a ``DbtDag`` with ``ExecutionMode.W **Key differences from** ``ExecutionMode.KUBERNETES``: -* The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` -* The producer task runs the entire ``dbt build`` command in a single Kubernetes pod -* Consumer tasks (sensors) watch for the completion of their corresponding dbt models +- The ``execution_mode`` is set to ``ExecutionMode.WATCHER_KUBERNETES`` instead of ``ExecutionMode.KUBERNETES`` +- The producer task runs the entire ``dbt build`` command in a single Kubernetes pod +- Consumer tasks (sensors) watch for the completion of their corresponding dbt models For the complete setup including Kubernetes secrets, Docker image configuration, and profile setup, refer to the :ref:`kubernetes` documentation. @@ -81,9 +81,9 @@ This represents approximately a **63% reduction** in total DAG runtime. The performance improvement comes from: -* Running dbt as a single command (reducing Kubernetes pod startup overhead) -* Leveraging dbt's native threading capabilities -* Eliminating repeated dbt initialization for each model +- Running dbt as a single command (reducing Kubernetes pod startup overhead) +- Leveraging dbt's native threading capabilities +- Eliminating repeated dbt initialization for each model ------------------------------------------------------------------------------- @@ -163,11 +163,11 @@ Other Inherited Limitations The following limitations from ``ExecutionMode.WATCHER`` also apply to ``ExecutionMode.WATCHER_KUBERNETES``: -* **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. +- **Individual dbt Operators**: Only ``DbtSeedWatcherKubernetesOperator``, ``DbtSnapshotWatcherKubernetesOperator``, and ``DbtRunWatcherKubernetesOperator`` are implemented. The ``DbtTestWatcherKubernetesOperator`` is currently a placeholder. -* **Test behavior**: Unlike ``ExecutionMode.WATCHER`` (which fully supports ``TestBehavior.AFTER_EACH`` since Cosmos 1.14.0), ``ExecutionMode.WATCHER_KUBERNETES`` does not yet support ``TestBehavior.AFTER_EACH``. Tests are run as part of the ``dbt build`` command by the producer task, and test tasks are rendered as ``EmptyOperator`` placeholders. This is tracked in `#1974 `_. +- **Test behavior**: Unlike ``ExecutionMode.WATCHER`` (which fully supports ``TestBehavior.AFTER_EACH`` since Cosmos 1.14.0), ``ExecutionMode.WATCHER_KUBERNETES`` does not yet support ``TestBehavior.AFTER_EACH``. Tests are run as part of the ``dbt build`` command by the producer task, and test tasks are rendered as ``EmptyOperator`` placeholders. This is tracked in `#1974 `_. -* **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. +- **Source freshness nodes**: The ``dbt build`` command does not run source freshness checks. For more details on these limitations, refer to the :ref:`watcher-execution-mode` documentation. @@ -203,9 +203,9 @@ Summary ``ExecutionMode.WATCHER_KUBERNETES`` provides: -* ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` -* ✅ **Isolation** between dbt and Airflow dependencies -* ✅ **Model-level visibility** in Airflow -* ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` +- ✅ **~63% faster** dbt DAG runs compared to ``ExecutionMode.KUBERNETES`` +- ✅ **Isolation** between dbt and Airflow dependencies +- ✅ **Model-level visibility** in Airflow +- ✅ **Easy migration** from ``ExecutionMode.KUBERNETES`` This execution mode is ideal for teams who want the performance benefits of the watcher mode while maintaining the isolation provided by Kubernetes execution. diff --git a/docs/guides/run_dbt/customization/partial-parsing.rst b/docs/guides/run_dbt/customization/partial-parsing.rst index a7a3873674..9d44cc04d2 100644 --- a/docs/guides/run_dbt/customization/partial-parsing.rst +++ b/docs/guides/run_dbt/customization/partial-parsing.rst @@ -13,8 +13,8 @@ Profile configuration To respect the dbt requirement of having the same profile to benefit from partial parsing, Cosmos users should either: -* If using Cosmos profile mapping (``ProfileConfig(profile_mapping=...``), disable using mocked profile mappings by setting ``render_config=RenderConfig(enable_mock_profile=False)`` -* Declare their own ``profiles.yml`` file, via ``ProfileConfig(profiles_yml_filepath=...)`` +- If using Cosmos profile mapping (``ProfileConfig(profile_mapping=...``), disable using mocked profile mappings by setting ``render_config=RenderConfig(enable_mock_profile=False)`` +- Declare their own ``profiles.yml`` file, via ``ProfileConfig(profiles_yml_filepath=...)`` If users don't follow these guidelines, Cosmos will use different profiles to parse the dbt project and to run tasks, and the user won't leverage dbt partial parsing. Their logs will contain multiple ``INFO`` messages similar to the following, meaning that Cosmos is not using partial parsing: diff --git a/docs/guides/run_dbt/operators/operators.rst b/docs/guides/run_dbt/operators/operators.rst index 5559d002e4..70367d361a 100644 --- a/docs/guides/run_dbt/operators/operators.rst +++ b/docs/guides/run_dbt/operators/operators.rst @@ -13,8 +13,8 @@ Clone Requirement -* Cosmos >= 1.8.0 -* dbt-core >= 1.6.0 +- Cosmos >= 1.8.0 +- dbt-core >= 1.6.0 The ``DbtCloneLocalOperator`` implement `dbt clone `_ command. diff --git a/docs/optimize_performance/caching.rst b/docs/optimize_performance/caching.rst index c8f2e2e249..6984d5c2d2 100644 --- a/docs/optimize_performance/caching.rst +++ b/docs/optimize_performance/caching.rst @@ -51,24 +51,24 @@ in the remote store. Cosmos will refresh the cache in a few circumstances: -* if any files of the dbt project change -* if one of the arguments that affect the dbt ls command execution changes +- if any files of the dbt project change +- if one of the arguments that affect the dbt ls command execution changes To evaluate if the dbt project changed, it calculates the changes using a few of the MD5 of all the files in the directory. Additionally, if any of the following DAG configurations are changed, we'll automatically purge the cache of the DAGs that use that specific configuration: -* ``ProjectConfig.dbt_vars`` -* ``ProjectConfig.env_vars`` -* ``ProjectConfig.partial_parse`` -* ``RenderConfig.env_vars`` -* ``RenderConfig.exclude`` -* ``RenderConfig.select`` -* ``RenderConfig.selector`` +- ``ProjectConfig.dbt_vars`` +- ``ProjectConfig.env_vars`` +- ``ProjectConfig.partial_parse`` +- ``RenderConfig.env_vars`` +- ``RenderConfig.exclude`` +- ``RenderConfig.select`` +- ``RenderConfig.selector`` Finally, if users would like to define specific Airflow variables that, if changed, will cause the recreation of the cache, they can specify those by using: -* ``RenderConfig.airflow_vars_to_purge_dbt_ls_cache`` +- ``RenderConfig.airflow_vars_to_purge_dbt_ls_cache`` Example: @@ -96,19 +96,19 @@ When using ``DbtDag``, the keys use the DAG name. When using ``DbtTaskGroup``, t Examples: -* The ``DbtDag`` "cosmos_dag" will have the cache represented by "cosmos_cache__basic_cosmos_dag". -* The ``DbtTaskGroup`` "customers" declared inside the DAG "basic_cosmos_task_group" will have the cache key "cosmos_cache__basic_cosmos_task_group__customers". +- The ``DbtDag`` "cosmos_dag" will have the cache represented by "cosmos_cache__basic_cosmos_dag". +- The ``DbtTaskGroup`` "customers" declared inside the DAG "basic_cosmos_task_group" will have the cache key "cosmos_cache__basic_cosmos_task_group__customers". **Cache value** The cache values contain a few properties: -* ``last_modified`` timestamp, represented using the ISO 8601 format. -* ``version`` is a hash that represents the version of the dbt project and arguments used to run dbt ls by the time Cosmos created the cache -* ``dbt_ls_compressed`` represents the dbt ls output compressed using zlib and encoded to base64 so Cosmos can record the value as a compressed string in the Airflow metadata database. -* ``dag_id`` is the DAG associated to this cache -* ``task_group_id`` is the TaskGroup associated to this cache -* ``cosmos_type`` is either ``DbtDag`` or ``DbtTaskGroup`` +- ``last_modified`` timestamp, represented using the ISO 8601 format. +- ``version`` is a hash that represents the version of the dbt project and arguments used to run dbt ls by the time Cosmos created the cache +- ``dbt_ls_compressed`` represents the dbt ls output compressed using zlib and encoded to base64 so Cosmos can record the value as a compressed string in the Airflow metadata database. +- ``dag_id`` is the DAG associated to this cache +- ``task_group_id`` is the TaskGroup associated to this cache +- ``cosmos_type`` is either ``DbtDag`` or ``DbtTaskGroup`` Caching the YAML selectors ++++++++++++++++++++++++++ @@ -135,17 +135,17 @@ in the remote store. Cosmos will refresh the cache in a few circumstances: -* if any files of the dbt project change -* if the YAML selectors in the manifest file change -* if the implementation of the YAML selector parsing logic changes +- if any files of the dbt project change +- if the YAML selectors in the manifest file change +- if the implementation of the YAML selector parsing logic changes - * For new definitions of the dbt YAML selector specification. + - For new definitions of the dbt YAML selector specification. To evaluate if the dbt project changed, it calculates the changes using a few of the MD5 of all the files in the directory. Finally, if users would like to define specific Airflow variables that, if changed, will cause the recreation of the cache, they can specify those by using: -* ``RenderConfig.airflow_vars_to_purge_dbt_yaml_selectors_cache`` +- ``RenderConfig.airflow_vars_to_purge_dbt_yaml_selectors_cache`` Example: @@ -177,20 +177,20 @@ When using ``DbtDag``, the keys use the DAG name. When using ``DbtTaskGroup``, t Examples: -* The ``DbtDag`` "cosmos_dag" will have the cache represented by "cosmos_cache__basic_cosmos_dag". -* The ``DbtTaskGroup`` "customers" declared inside the DAG "basic_cosmos_task_group" will have the cache key "cosmos_cache__basic_cosmos_task_group__customers". +- The ``DbtDag`` "cosmos_dag" will have the cache represented by "cosmos_cache__basic_cosmos_dag". +- The ``DbtTaskGroup`` "customers" declared inside the DAG "basic_cosmos_task_group" will have the cache key "cosmos_cache__basic_cosmos_task_group__customers". **Cache value** The cache values contain a few properties: -* ``last_modified`` timestamp, represented using the ISO 8601 format. -* ``version`` is a hash that represents the version of the dbt project, the raw YAML selectors, and a hash of the YAML selector parser implementation version combined with the keys specified by ``airflow_vars_to_purge_dbt_yaml_selectors_cache`` -* ``raw_selectors_compressed`` represents the raw YAML selector definitions compressed using zlib and encoded to base64 -* ``parsed_selectors_compressed`` represents the parsed YAML selector definitions compressed using zlib and encoded to base64 -* ``dag_id`` is the DAG associated to this cache -* ``task_group_id`` is the TaskGroup associated to this cache -* ``cosmos_type`` is either ``DbtDag`` or ``DbtTaskGroup`` +- ``last_modified`` timestamp, represented using the ISO 8601 format. +- ``version`` is a hash that represents the version of the dbt project, the raw YAML selectors, and a hash of the YAML selector parser implementation version combined with the keys specified by ``airflow_vars_to_purge_dbt_yaml_selectors_cache`` +- ``raw_selectors_compressed`` represents the raw YAML selector definitions compressed using zlib and encoded to base64 +- ``parsed_selectors_compressed`` represents the parsed YAML selector definitions compressed using zlib and encoded to base64 +- ``dag_id`` is the DAG associated to this cache +- ``task_group_id`` is the TaskGroup associated to this cache +- ``cosmos_type`` is either ``DbtDag`` or ``DbtTaskGroup`` **Shared Cache Behavior** diff --git a/docs/optimize_performance/memory_optimization.rst b/docs/optimize_performance/memory_optimization.rst index 6d82aec704..b0e9d76a87 100644 --- a/docs/optimize_performance/memory_optimization.rst +++ b/docs/optimize_performance/memory_optimization.rst @@ -37,7 +37,7 @@ Cosmos provides various configuration options and execution modes to optimize me 2. Use DBT_RUNNER Invocation Mode +++++++++++++++++++++++++++++++++ -* (default for ``ExecutionMode.LOCAL`` since 1.4.0, default for ``RenderConfig.DBT_LS`` since Cosmos 1.9.0) +- (default for ``ExecutionMode.LOCAL`` since 1.4.0, default for ``RenderConfig.DBT_LS`` since Cosmos 1.9.0) **Impact**: Medium-High. Depends on the execution and load modes used. Can reduce subprocess overhead and memory usage compared to subprocess mode. diff --git a/docs/policy/contributors-roles.rst b/docs/policy/contributors-roles.rst index 79990f084c..084be276f3 100644 --- a/docs/policy/contributors-roles.rst +++ b/docs/policy/contributors-roles.rst @@ -18,12 +18,12 @@ Cosmos contributors can be found in the Astronomer Cosmos Github `insights page Contributors are responsible for: -* Fixing bugs -* Refactoring code -* Improving processes and tooling -* Adding features -* Improving the documentation -* Making/answering questions in the #airflow-dbt Slack channel +- Fixing bugs +- Refactoring code +- Improving processes and tooling +- Adding features +- Improving the documentation +- Making/answering questions in the #airflow-dbt Slack channel Committers @@ -36,8 +36,8 @@ Check :ref:`contributors` for the official list of Astronomer Cosmos committers. Committers have the same responsibilities as standard contributors and also perform the following actions: -* Reviewing & merging pull-requests -* Scanning and responding to GitHub issues, helping triaging them +- Reviewing & merging pull-requests +- Scanning and responding to GitHub issues, helping triaging them If you know you are not going to be able to contribute for a long time (for instance, due to a change of job or circumstances), you should inform other maintainers, and we will mark you as "emeritus". Emeritus committers will no longer have write access to the repo. diff --git a/docs/policy/contributors.rst b/docs/policy/contributors.rst index 925f8bb00b..32925c22e2 100644 --- a/docs/policy/contributors.rst +++ b/docs/policy/contributors.rst @@ -9,19 +9,19 @@ Learn more about the project contributors roles in :ref:`contributors-roles`. Committers ++++++++++ -* Daniel Reeves (`@dwreeves `_) -* Giovanni Corsetti Silva (`@corsettigyg `_) -* Julian LaNeve (`@jlaneve `_) -* Justin Bandoro (`@jbandoro `_) -* Pankaj Koti (`@pankajkoti `_) -* Pankaj Singh (`@pankajastro `_) -* Tatiana Al-Chueyr (`@tatiana `_) +- Daniel Reeves (`@dwreeves `_) +- Giovanni Corsetti Silva (`@corsettigyg `_) +- Julian LaNeve (`@jlaneve `_) +- Justin Bandoro (`@jbandoro `_) +- Pankaj Koti (`@pankajkoti `_) +- Pankaj Singh (`@pankajastro `_) +- Tatiana Al-Chueyr (`@tatiana `_) Emeritus Committers +++++++++++++++++++ -* Chris Hronek (`@chrishronek `_) +- Chris Hronek (`@chrishronek `_) Contributors