Add a blocking lychee linkcheck CI job and fix broken docs links

.github/workflows/docs-build.yml

@@ -8,12 +8,14 @@ on:
       - "docs/**"
       - "cosmos/**"
       - ".github/workflows/docs-build.yml"
+      - "lychee.toml"
   pull_request:
     branches: [main, 'release-*']
     paths:
       - "docs/**"
       - "cosmos/**"
       - ".github/workflows/docs-build.yml"
+      - "lychee.toml"
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
@@ -41,3 +43,22 @@ jobs:
 
       - name: Build Sphinx documentation
         run: sphinx-build -b html docs docs/_build --fail-on-warning --fresh-env
+
+  linkcheck:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          ref: ${{ github.event.pull_request.head.sha || github.ref }}
+          persist-credentials: false
+
+      - name: Check links in documentation
+        # Uses the lychee link checker. Behaviour (which hosts to skip, retries,
+        # accepted status codes for throttling hosts) is configured in the
+        # repo-root lychee.toml. Fails the job on genuinely broken links.
+        uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 # v2.8.0
+        with:
+          args: "--no-progress --config lychee.toml 'docs/**/*.rst'"
+          fail: true

docs/getting_started/astro-cli-quickstart.rst

@@ -5,7 +5,7 @@
 Run Cosmos with the Astro CLI
 =============================
 
-Work locally with `Apache Airflow® <https://airflow.apache.org/>`_, dbt, and Astronomer Cosmos using the `Astro CLI <https://github.com/astronomer/astro-cli>`__. While Cosmos fully works with standard Airflow and independently of the Astro CLI, the Astro CLI can simplify creating and running Airflow projects. If you want to get started with Cosmos using only Airflow, see `Getting Started on Open Source Airflow <open-source.html>`_.
+Work locally with `Apache Airflow® <https://airflow.apache.org/>`_, dbt, and Astronomer Cosmos using the `Astro CLI <https://github.com/astronomer/astro-cli>`__. While Cosmos fully works with standard Airflow and independently of the Astro CLI, the Astro CLI can simplify creating and running Airflow projects. If you want to get started with Cosmos using only Airflow, see :doc:`Getting Started on Open Source Airflow </getting_started/open-source>`.
 
 This guide shows you how to run a simple Dag locally with Cosmos, using an example dbt project in Airflow.
 
@@ -136,7 +136,7 @@ Next steps
 ~~~~~~~~~~
 
 - Follow one of the Getting Started Guides where you can bring your own dbt projects and/or Dag code:
-    - `Getting Started on Open-Source <open-source.html>`__
-    - `Getting Started on Astro <astro.html>`__
-    - `Getting Started on MWAA <mwaa.html>`__
-    - `Getting Started on GCC <gcc.html>`__
+    - :doc:`Getting Started on Open-Source </getting_started/open-source>`
+    - :doc:`Getting Started on Astro </getting_started/astro>`
+    - :doc:`Getting Started on MWAA </getting_started/mwaa>`
+    - :doc:`Getting Started on GCC </getting_started/gcc>`

docs/getting_started/core-concepts.rst

@@ -82,7 +82,7 @@ The method that Cosmos uses to parse the dbt object, whether its by using the Py
 Rendering
 ~~~~~~~~~
 
-After parsing your dbt project, Cosmos *renders* the project as an Airflow Dag or Task Group. Depending on the specifics of your dbt project, you can choose customizations that speed up overall performance. You can find more information about rendering options in the `RenderConfig <../guides/translate_dbt_to_airflow/render-config.html>`_.
+After parsing your dbt project, Cosmos *renders* the project as an Airflow Dag or Task Group. Depending on the specifics of your dbt project, you can choose customizations that speed up overall performance. You can find more information about rendering options in the :doc:`RenderConfig </guides/translate_dbt_to_airflow/render-config>`.
 
 
 Testing Strategy

docs/getting_started/how-cosmos-works.rst

@@ -18,17 +18,17 @@ Cosmos creates an interface between a dbt project and Airflow, allowing you to t
 
 You have a number of configuration options, but fundamentally, Cosmos provides the following two functions:
 
-- **Parse your dbt project**: Cosmos parses your dbt project, and translates it into an Airflow Dag. This process uses the `ProjectConfig <../reference/configs/project-config.html>`_ and `RenderConfig <../guides/translate_dbt_to_airflow/render-config.html>`_ to customize specific behavior, allowing you to optimize how your dbt project is represented as a Dag.
+- **Parse your dbt project**: Cosmos parses your dbt project, and translates it into an Airflow Dag. This process uses the :doc:`ProjectConfig </reference/configs/project-config>` and :doc:`RenderConfig </guides/translate_dbt_to_airflow/render-config>` to customize specific behavior, allowing you to optimize how your dbt project is represented as a Dag.
 
-- **Execute the dbt commands**: Cosmos then executes the Dag, using the execution options in your `ExecutionConfig <../reference/configs/execution-config.html>`_ and `ProjectConfig <../reference/configs/project-config.html>`_ to run dbt commands with the appropriate dbt adapter, finally resulting in your dbt SQL running in your data warehouse. Cosmos uses a connection defined in the `ProfileConfig <../profiles/index.html>`_ to execute your SQL in your data warehouse.
+- **Execute the dbt commands**: Cosmos then executes the Dag, using the execution options in your :doc:`ExecutionConfig </reference/configs/execution-config>` and :doc:`ProjectConfig </reference/configs/project-config>` to run dbt commands with the appropriate dbt adapter, finally resulting in your dbt SQL running in your data warehouse. Cosmos uses a connection defined in the :doc:`ProfileConfig </reference/configs/profile-config>` to execute your SQL in your data warehouse.
 
 .. image:: ../_static/how_cosmos_works.png
    :alt: Diagram showing that Cosmos parses dbt projects, with options defined by the ProfileConfig and RenderConfig. Then, Cosmos executes the resulting Dag, with options defined by ExecutionConfig and ProjectConfig.
 
 Quickstart
 ~~~~~~~~~~
 
-Even though Cosmos is highly extensible, and you have many advanced customization options, you can run a demo with the `Astro CLI <astro-cli-quickstart.html>`_ in just a few minutes. This demo introduces you to the key elements required for Cosmos to parse dbt projects and run Dags.
+Even though Cosmos is highly extensible, and you have many advanced customization options, you can run a demo with the :doc:`Astro CLI </getting_started/astro-cli-quickstart>` in just a few minutes. This demo introduces you to the key elements required for Cosmos to parse dbt projects and run Dags.
 
 Bring your own project
 ~~~~~~~~~~~~~~~~~~~~~~

docs/getting_started/mwaa.rst

@@ -3,7 +3,7 @@
 Getting Started with Cosmos on Amazon Managed Workflows
 =======================================================
 
-Users can face Python dependency issues when trying to use the Cosmos `Local Execution Mode <execution-modes.html#local>`_ in Amazon Managed Workflows for `Apache Airflow® <https://airflow.apache.org/>`_ (MWAA).
+Users can face Python dependency issues when trying to use the Cosmos :ref:`Local Execution Mode <local-execution>` in Amazon Managed Workflows for `Apache Airflow® <https://airflow.apache.org/>`_ (MWAA).
 
 This step-by-step illustrates how to use the Local Execution Mode, together with the
 `MWAA's startup script <https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html>`_ and

docs/getting_started/oss-quickstart.rst

@@ -5,7 +5,7 @@ Run Cosmos in open-source Apache Airflow®
 
 Quickly get started working locally with `Apache Airflow® <https://airflow.apache.org/>`_, dbt, and Astronomer Cosmos using the `Open-source Airflow <https://airflow.apache.org/docs/apache-airflow/stable/index.html>`_.
 
-This quickstart guide shows you how to set up a simple demo project, run a simple Dag locally with Cosmos, and then view the results with an open-source database viewer. If you want to get started working with your own project and configurations, see `Get started with Open-source Airflow <open-source.html>`_.
+This quickstart guide shows you how to set up a simple demo project, run a simple Dag locally with Cosmos, and then view the results with an open-source database viewer. If you want to get started working with your own project and configurations, see :doc:`Get started with Open-source Airflow </getting_started/open-source>`.
 
 By the end of this quickstart, you will:
 

docs/guides/connect_database/index.rst

@@ -5,8 +5,8 @@ Connect to your database
 
 Cosmos supports two methods of authenticating with your database:
 
-- using your own `dbt profiles.yml <../../guides/connect_database/use-your-profiles-yml.html>`_ file
-- using `Apache Airflow® <https://airflow.apache.org/>`_ connections via Cosmos' `profile mappings <../../guides/connect_database/use-profile-mapping.html>`_
+- using your own :doc:`dbt profiles.yml </guides/connect_database/use-your-profiles-yml>` file
+- using `Apache Airflow® <https://airflow.apache.org/>`_ connections via Cosmos' :doc:`profile mappings </guides/connect_database/use-profile-mapping>`
 
 If you're already interacting with your database from Airflow and have a connection set up, it's recommended
 to use a profile mapping to translate that Airflow connection to a dbt profile. This is because it's easier to

docs/guides/cosmos_devex/lineage.rst

@@ -18,7 +18,7 @@ Known limitations
 ~~~~~~~~~~~~~~~~~
 
 This feature is only available for the local
-and virtualenv execution methods (read `execution modes <../run_dbt/execution-modes.html>`_ for more information).
+and virtualenv execution methods (read :doc:`execution modes </guides/run_dbt/execution-modes>` for more information).
 
 Additionally, since Cosmos uses the open-source `openlineage-integration-common <https://github.com/OpenLineage/OpenLineage/tree/main/integration/common>`_, it relies on this library to support specific dbt adapters. As of 27 December 2024, the version 1.26.0 of this package supports:
 

docs/guides/dbt_docs/generating-docs.rst

@@ -5,7 +5,7 @@ Generating Docs
 
 dbt allows you to generate static documentation on your models, tables, and more. You can read more about it in the `official dbt documentation <https://docs.getdbt.com/docs/building-a-dbt-project/documentation>`_. For an example of what the docs look like with the ``jaffle_shop`` project, check out `this site <http://cosmos-demo-dbt-docs.s3-website.eu-north-1.amazonaws.com/>`_.
 
-After generating the dbt docs, you can host them natively within `Apache Airflow® <https://airflow.apache.org/>`_ via the Cosmos Airflow plugin; see `Hosting Docs <hosting-docs.html>`__ for more information.
+After generating the dbt docs, you can host them natively within `Apache Airflow® <https://airflow.apache.org/>`_ via the Cosmos Airflow plugin; see :doc:`Hosting Docs </guides/dbt_docs/hosting-docs>` for more information.
 
 Alternatively, many users choose to serve these docs on a separate static website. This is a great way to share your data models with a broad array of stakeholders.
 

docs/guides/dbt_docs/hosting-docs.rst

@@ -138,7 +138,7 @@ Host from Cloud Storage
 
 For typical users, the recommended setup for hosting dbt docs would look like this:
 
-1. Generate the docs via one of Cosmos' pre-built operators for generating dbt docs (see `Generating Docs <generating-docs.html>`__ for more information)
+1. Generate the docs via one of Cosmos' pre-built operators for generating dbt docs (see :doc:`Generating Docs </guides/dbt_docs/generating-docs>` for more information)
 2. Wherever you dumped the docs, set your ``cosmos.dbt_docs_dir`` to that location.
 3. If you want to use a conn ID other than the default connection, set your ``cosmos.dbt_docs_conn_id``. Otherwise, leave this blank.
 

docs/guides/dbt_setup/dbt-fusion.rst

@@ -12,7 +12,7 @@ Context
 dbt Labs `launched <https://www.getdbt.com/blog/dbt-launch-showcase-2025-recap>`_ `dbt Fusion <https://github.com/dbt-labs/dbt-fusion>`_ on 28 May 2025. dbt Fusion is the next-generation dbt engine that enables real-time model validation, improved SQL parsing and state-aware orchestration.
 It is a unified approach that aims to merge dbt Core and dbt Cloud features via a completely new CLI,
 implemented in a different programming language (Rust, as opposed to Python).
-As part of this, dbt Labs are `rewriting all dbt adapters <https://github.com/dbt-labs/dbt-fusion/tree/main/crates/dbt-fusion-adapter/src/adapters>`_ (equivalent to `Apache Airflow® <https://airflow.apache.org/>`_ providers) in Rust, starting from Snowflake.
+As part of this, dbt Labs are `rewriting all dbt adapters <https://github.com/dbt-labs/dbt-fusion/tree/main/crates/dbt-adapter>`_ (equivalent to `Apache Airflow® <https://airflow.apache.org/>`_ providers) in Rust, starting from Snowflake.
 They are also changing the `licensing model <https://github.com/dbt-labs/dbt-fusion/blob/main/LICENSES.md>`_ to a hybrid Open-Source and commercial license. We're supporting dbt Fusion with a license-complaint integration. This integration enables teams to:
 
 - Use dbt Fusion locally for enhanced development experience with real-time validation

docs/guides/dbt_setup/execution-modes-local-conflicts.rst

@@ -8,8 +8,8 @@ When using the :ref:`local-execution` without defining a custom ``ExecutionConfi
 
 If you find errors, we recommend users isolating the installation of dbt from the Airflow installation.
 With the ``local`` execution mode, this can be accomplished by installing dbt in a separate
-Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../../reference/configs/execution-config.html>`_  and
-`RenderConfig.dbt_executable_path <../../guides/translate_dbt_to_airflow/render-config.html>`_ parameters.
+Python virtualenv and setting the :doc:`ExecutionConfig.dbt_executable_path </reference/configs/execution-config>`  and
+:doc:`RenderConfig.dbt_executable_path </guides/translate_dbt_to_airflow/render-config>` parameters.
 
 The page, :ref:`execution-modes` describes many other methods that support isolating dbt from Airflow.
 

docs/guides/translate_dbt_to_airflow/parsing-methods.rst

@@ -142,7 +142,7 @@ Users can force Cosmos to run ``dbt ls`` with subprocess and not ``dbtRunner``,
     )
 
 
-For more information, check the `RenderConfig docs <./render-config.html>`_.
+For more information, check the :doc:`RenderConfig docs </guides/translate_dbt_to_airflow/render-config>`.
 
 
 ``dbt_ls_file``

docs/guides/translate_dbt_to_airflow/render-config.rst

@@ -10,11 +10,11 @@ It does this by exposing a ``cosmos.config.RenderConfig`` class that you can use
 The ``RenderConfig`` class takes the following arguments:
 
 - ``emit_datasets``: whether or not to emit Airflow datasets to be used for data-aware scheduling. Defaults to True. This feature is only available for ``ExecutionMode.LOCAL``, ``ExecutionMode.VIRTUALENV``, ``ExecutionMode.WATCHER`` and ``ExecutionMode.AIRFLOW_ASYNC``.
-- ``test_behavior``: how to run tests. Defaults to running a model's tests immediately after the model is run. For more information, see the `Testing Behavior <testing-behavior.html>`_ section.
-- ``load_method``: how to load your dbt project. See `Parsing Methods <parsing-methods.html>`_ for more information.
+- ``test_behavior``: how to run tests. Defaults to running a model's tests immediately after the model is run. For more information, see the :doc:`Testing Behavior </guides/translate_dbt_to_airflow/testing-behavior>` section.
+- ``load_method``: how to load your dbt project. See :doc:`Parsing Methods </guides/translate_dbt_to_airflow/parsing-methods>` for more information.
 - ``invocation_mode``: (new in v1.9) how to run ``dbt ls``, when using ``LoadMode.DBT_LS``. Learn more about this below.
-- ``select`` and ``exclude``: which models to include or exclude from your DAGs. See `Selecting & Excluding <selecting-excluding.html>`_ for more information.
-- ``selector``: (new in v1.3) name of a dbt YAML selector to use for DAG parsing. Only supported when using ``load_method=LoadMode.DBT_LS``. See `Selecting & Excluding <selecting-excluding.html>`_ for more information.
+- ``select`` and ``exclude``: which models to include or exclude from your DAGs. See :doc:`Selecting & Excluding </guides/translate_dbt_to_airflow/selecting-excluding>` for more information.
+- ``selector``: (new in v1.3) name of a dbt YAML selector to use for DAG parsing. Only supported when using ``load_method=LoadMode.DBT_LS``. See :doc:`Selecting & Excluding </guides/translate_dbt_to_airflow/selecting-excluding>` for more information.
 - ``dbt_deps``: (deprecated in v1.9, use ``ProjectConfig.install_dbt_deps`` onwards) A Boolean to run dbt deps when using dbt ls for dag parsing. Default True
 - ``node_converters``: a dictionary mapping a ``DbtResourceType`` into a callable. Users can control how to render dbt nodes in Airflow. Only supported when using ``load_method=LoadMode.DBT_MANIFEST`` or ``LoadMode.DBT_LS``. Find more information below.
 - ``node_conversion_by_task_group``: (new in v1.12.0) A boolean to control if node_converters are used at the task group level (ex. converting models with test_behavior=AFTER_EACH means the entire task group is converted including the run task and the test task), or the individual task level (gives more granularity for converting just the run tasks or just the test tasks).  Defaults to True.
@@ -29,7 +29,7 @@ The ``RenderConfig`` class takes the following arguments:
 - ``source_pruning``: When set to ``True``, automatically removes (or "prunes") any dbt source nodes from your Airflow DAG that do not have any downstream dependencies within the selected portion of the dbt graph. Defaults to ``False``. See :doc:`Source Nodes Rendering <managing-sources>` for more information.
 - ``normalize_task_id``: A callable that takes a dbt node as input and returns the task ID. This function allows users to set a custom task_id independently of the model name, which can be specified as the task's display_name. This way, task_id can be modified using a user-defined function, while the model name remains as the task's display name. The display_name parameter is available in Airflow 2.9 and above. See :doc:`Task display name </guides/cosmos_devex/task-display-name>` for more information.
 - ``normalize_task_display_name``: This function allows users to set a custom user-defined function to alter the display name independently of the model name. This way, the task_id can be preserved while the model display name is modified.
-- ``should_detach_multiple_parents_tests``: A boolean to control if tests that depend on multiple parents should be run as standalone tasks. See `Testing Behavior <testing-behavior.html>`_ for more information.
+- ``should_detach_multiple_parents_tests``: A boolean to control if tests that depend on multiple parents should be run as standalone tasks. See :doc:`Testing Behavior </guides/translate_dbt_to_airflow/testing-behavior>` for more information.
 - ``enable_owner_inheritance``: (introduced in 1.10.2) A boolean to control if dbt owners should be imported as part of the airflow DAG owners. Defaults to True.
 - ``node_conversion_by_task_group``: A boolean to control if node_converters are used at the task group level (ex. converting models with test_behavior=AFTER_EACH means the entire task group is converted including the run task and the test task), or the individual task level (gives more granularity for converting just the run tasks or just the test tasks).  Defaults to True.
 - ``group_nodes_by_folder``: When enabled, groups nodes by folder structure, creating a ``TaskGroup`` per resource type and folder. Disabled by default.

docs/index.rst

@@ -89,7 +89,7 @@ License
 `Apache License 2.0 <https://github.com/astronomer/astronomer-cosmos/blob/main/LICENSE>`_
 
 .. Tracking pixel for Scarf
-.. See https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/security-privacy.rst for more info about Cosmos privacy policy
+.. See https://github.com/astronomer/astronomer-cosmos/blob/main/PRIVACY_NOTICE.rst for more info about Cosmos privacy policy
 
 .. raw:: html