Skip to content

[Docs] Getting Started - Table of Contents restructure #2408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lzdanski wants to merge 11 commits into from
Closed

[Docs] Getting Started - Table of Contents restructure #2408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
13a1839
first pass architecture
lzdanski Feb 25, 2026
1383081
Merge branch 'astronomer:main' into getting-started-toc-draft
lzdanski Feb 25, 2026
51cc1f8
First pass reformat
lzdanski Feb 25, 2026
1c140fc
delete test pages and directory
lzdanski Feb 25, 2026
56cd870
make flat navigation
lzdanski Feb 25, 2026
2eeefae
separate execution modes
lzdanski Feb 25, 2026
d5ea1fd
🎨 [pre-commit.ci] Auto format from pre-commit.com hooks
pre-commit-ci[bot] Feb 25, 2026
fd288b3
Merge branch 'astronomer:main' into getting-started-toc-draft
lzdanski Feb 26, 2026
7cc2abf
edit titles and sidebar
lzdanski Feb 26, 2026
03abd0f
remove placeholder page
lzdanski Feb 26, 2026
2906d88
update toc nav
lzdanski Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
"image_dark": "cosmos-icon.svg",
},
"footer_start": ["copyright"],
"show_nav_level": 0,
}

generate_mapping_docs()
4 changes: 3 additions & 1 deletion docs/contributing.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,17 +155,19 @@ To run the checks manually, run:
Writing Docs
____________

`Hatch <https://hatch.pypa.io/latest/>`_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatchto declare the dependencies required for the project itself, as well as for tests and documentation builds.
`Hatch <https://hatch.pypa.io/latest/>`_ is a unified command-line tool for managing dependencies and environment isolation for Python developers. In Cosmos, we use a Hatch to declare the dependencies required for the project itself, as well as for tests and documentation builds.

If you don’t already have Hatch installed, please `install it <https://hatch.pypa.io/latest/install/>`_ before proceeding. As an example, on macOS, you can do so with:

.. code-block:: bash

brew install hatch


You can run the docs locally by running the following:

.. code-block:: bash

hatch run docs:serve


Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/astro.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _astro:

Getting Started on Astro
========================
Getting Started with Cosmos on Astro
====================================

While it is possible to use Cosmos on Astro with all :ref:`Execution Modes <execution-modes>`, we recommend using the ``local`` execution mode. It's the simplest to set up and use.

Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/dbt-airflow-concepts.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _dbt-airflow-concepts:

Similar dbt & Airflow concepts
==============================
Similar dbt and Airflow concepts
================================

While dbt is an open source tool for data transformations and analysis, using SQL, Airflow focuses on being a platform
for the development, scheduling and monitoring of batch-oriented workflows, using Python. Although both tools have many
Expand Down
28 changes: 21 additions & 7 deletions docs/getting_started/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,41 @@
.. _getting-started:

.. toctree::
:maxdepth: 1
:hidden:
:caption: Contents:
:caption: Cosmos Fundamentals

Similar dbt and Airflow concepts <dbt-airflow-concepts>

.. toctree::
:maxdepth: 1
:hidden:
:caption: Get started with Cosmos

Open-source Airflow <open-source>
Astro <astro>
MWAA <mwaa>
GCC <gcc>
Open-Source <open-source>
Google Cloud Composer (GCC) <gcc>
Amazon Managed Workflows for Apache Airflow (MWAA) <mwaa>


.. toctree::
:maxdepth: 1
:hidden:
:caption: Execution modes and customization

Execution Modes <execution-modes>
execution-modes-local-conflicts
Docker Execution Mode <docker>
Kubernetes Execution Mode <kubernetes>
Azure Container Instance Execution Mode <azure-container-instance>
AWS Container Run Job Execution Mode <aws-container-run-job>
AWS Elastic Container Services (ECS) Execution Mode <aws-container-run-job>
GCP Cloud Run Job Execution Mode <gcp-cloud-run-job>
Airflow Async Execution Mode <async-execution-mode>
Watcher Execution Mode <watcher-execution-mode>
Watcher Kubernetes Execution Mode <watcher-kubernetes-execution-mode>
Comment on lines 26 to 35
Copy link
Copy Markdown
Collaborator

@tatiana tatiana Feb 26, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This currently renders:

Execution Modes

  • Execution Modes
  • Airflow and dbt dependencies conflicts
  • Docker Execution Mode
  • Azure Container instance execution mode
  • AWS Container Run Job Execution Mode
  • GCP Cloud Run Job Execution Mode
  • Airflow Async Execution Mode
  • Watcher Execution Mode
  • Watcher Kubernetes Execution Mode

I think this part of the table of contents can cause confusion and reduce scannability for new users. A few concerns:

  1. Redundant Parent and Child Naming
  2. Overuse of the Phrase “Execution Mode”
  3. Inconsistent Naming Conventions
  4. Concept vs Implementation Mixing
  5. Not Beginner-Friendly in a “Getting Started” Context

On the inconsistent naming:

  • There’s an inconsistency in how platforms and concepts are presented:
  • Some are infrastructure-based (Docker, Azure Container Instance)
  • Some are cloud-job based (AWS Container Run Job, GCP Cloud Run Job)
  • Some are behaviour-based (Airflow Async, Watcher)
  • Some are dependency-related (Airflow and dbt dependency conflicts)
  • These are not all in the same conceptual category, yet they’re grouped under the same heading as if they were equivalent “modes.”

On (4), concept x implementation, the list mixes:

  • Conceptual topics (dependency conflicts)
  • Execution strategies (async, watcher)
  • Infrastructure backends (Docker, Cloud Run, etc.)

This makes the mental model unclear. A user might reasonably ask:

  • Are these all mutually exclusive modes?
  • Are there some infrastructure options?
  • Are some features layered on top of others?

Lastly, on (5), for someone just trying to run dbt in Airflow, seeing:

  • Watcher Kubernetes Execution Mode
  • AWS Container Run Job Execution Mode
    …is overwhelming and abstract.

Instead of a smooth getting-started experience, it introduces internal terminology before grounding the user in the task.

Maybe we should break it down into two parts:

dbt Installation

  • Installing dbt (that could cover: same virtualenv as Airflow via requirements.txt, dedicated user-managed virtualenv via Dockerfile, dedicated Cosmos-managed virtualenv using ExecutionMode.VIRTUALENV or by creating a container image that contains dbt
  • Installing dbt dependencies (ProjectConfig.install_dbt_deps)

Running dbt in Airflow

  • Run dbt in the Airflow worker environment
    • Standard Execution
    • Watcher Execution
    • Async Execution
  • Running dbt in a container
    • Kubernetes
    • Kubernetes Watcher
    • Docker (highlight in this section, it is not compatible with Astro due to Docker in Docker issues!)
    • AWS ECS
    • AWS EKS
    • Azure Container Instance
    • GCP Cloud Run

dbt and Airflow Similar Concepts <dbt-airflow-concepts>
Operators <operators>
Custom Airflow Properties <custom-airflow-properties>


Getting Started
===============

Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/mwaa.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _mwaa:

Getting Started on MWAA
=======================
Getting Started with Cosmos on Amazon Managed Workflows
=======================================================

Users can face Python dependency issues when trying to use the Cosmos `Local Execution Mode <execution-modes.html#local>`_ in Amazon Managed Workflows for `Apache Airflow® <https://airflow.apache.org/>`_ (MWAA).

Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/open-source.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _open-source:

Getting Started on Open Source Airflow
======================================
Getting Started with Cosmos on Open-source Airflow
==================================================

When running open-source Airflow, your setup may vary. This guide assumes you have access to edit the underlying image.

Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

.. toctree::
:hidden:
:maxdepth: 2
:maxdepth: 1
:caption: Contents:

Home <self>
Expand Down
Loading