diff --git a/.github/workflows/docs-build.yml b/.github/workflows/docs-build.yml index f356d6b8d3..46d9ca986f 100644 --- a/.github/workflows/docs-build.yml +++ b/.github/workflows/docs-build.yml @@ -8,12 +8,14 @@ on: - "docs/**" - "cosmos/**" - ".github/workflows/docs-build.yml" + - "lychee.toml" pull_request: branches: [main, 'release-*'] paths: - "docs/**" - "cosmos/**" - ".github/workflows/docs-build.yml" + - "lychee.toml" concurrency: group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} @@ -41,3 +43,22 @@ jobs: - name: Build Sphinx documentation run: sphinx-build -b html docs docs/_build --fail-on-warning --fresh-env + + linkcheck: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + ref: ${{ github.event.pull_request.head.sha || github.ref }} + persist-credentials: false + + - name: Check links in documentation + # Uses the lychee link checker. Behaviour (which hosts to skip, retries, + # accepted status codes for throttling hosts) is configured in the + # repo-root lychee.toml. Fails the job on genuinely broken links. + uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 # v2.8.0 + with: + args: "--no-progress --config lychee.toml 'docs/**/*.rst'" + fail: true diff --git a/docs/getting_started/astro-cli-quickstart.rst b/docs/getting_started/astro-cli-quickstart.rst index 288231e7ee..ebea64e67e 100644 --- a/docs/getting_started/astro-cli-quickstart.rst +++ b/docs/getting_started/astro-cli-quickstart.rst @@ -5,7 +5,7 @@ Run Cosmos with the Astro CLI ============================= -Work locally with `Apache Airflow® `_, dbt, and Astronomer Cosmos using the `Astro CLI `__. While Cosmos fully works with standard Airflow and independently of the Astro CLI, the Astro CLI can simplify creating and running Airflow projects. If you want to get started with Cosmos using only Airflow, see `Getting Started on Open Source Airflow `_. +Work locally with `Apache Airflow® `_, dbt, and Astronomer Cosmos using the `Astro CLI `__. While Cosmos fully works with standard Airflow and independently of the Astro CLI, the Astro CLI can simplify creating and running Airflow projects. If you want to get started with Cosmos using only Airflow, see :doc:`Getting Started on Open Source Airflow `. This guide shows you how to run a simple Dag locally with Cosmos, using an example dbt project in Airflow. @@ -136,7 +136,7 @@ Next steps ~~~~~~~~~~ - Follow one of the Getting Started Guides where you can bring your own dbt projects and/or Dag code: - - `Getting Started on Open-Source `__ - - `Getting Started on Astro `__ - - `Getting Started on MWAA `__ - - `Getting Started on GCC `__ + - :doc:`Getting Started on Open-Source ` + - :doc:`Getting Started on Astro ` + - :doc:`Getting Started on MWAA ` + - :doc:`Getting Started on GCC ` diff --git a/docs/getting_started/core-concepts.rst b/docs/getting_started/core-concepts.rst index 55d51cb66d..89a4d7055b 100644 --- a/docs/getting_started/core-concepts.rst +++ b/docs/getting_started/core-concepts.rst @@ -82,7 +82,7 @@ The method that Cosmos uses to parse the dbt object, whether its by using the Py Rendering ~~~~~~~~~ -After parsing your dbt project, Cosmos *renders* the project as an Airflow Dag or Task Group. Depending on the specifics of your dbt project, you can choose customizations that speed up overall performance. You can find more information about rendering options in the `RenderConfig <../guides/translate_dbt_to_airflow/render-config.html>`_. +After parsing your dbt project, Cosmos *renders* the project as an Airflow Dag or Task Group. Depending on the specifics of your dbt project, you can choose customizations that speed up overall performance. You can find more information about rendering options in the :doc:`RenderConfig `. Testing Strategy diff --git a/docs/getting_started/how-cosmos-works.rst b/docs/getting_started/how-cosmos-works.rst index f5b52bb367..cffe9cbfd7 100644 --- a/docs/getting_started/how-cosmos-works.rst +++ b/docs/getting_started/how-cosmos-works.rst @@ -18,9 +18,9 @@ Cosmos creates an interface between a dbt project and Airflow, allowing you to t You have a number of configuration options, but fundamentally, Cosmos provides the following two functions: -- **Parse your dbt project**: Cosmos parses your dbt project, and translates it into an Airflow Dag. This process uses the `ProjectConfig <../reference/configs/project-config.html>`_ and `RenderConfig <../guides/translate_dbt_to_airflow/render-config.html>`_ to customize specific behavior, allowing you to optimize how your dbt project is represented as a Dag. +- **Parse your dbt project**: Cosmos parses your dbt project, and translates it into an Airflow Dag. This process uses the :doc:`ProjectConfig ` and :doc:`RenderConfig ` to customize specific behavior, allowing you to optimize how your dbt project is represented as a Dag. -- **Execute the dbt commands**: Cosmos then executes the Dag, using the execution options in your `ExecutionConfig <../reference/configs/execution-config.html>`_ and `ProjectConfig <../reference/configs/project-config.html>`_ to run dbt commands with the appropriate dbt adapter, finally resulting in your dbt SQL running in your data warehouse. Cosmos uses a connection defined in the `ProfileConfig <../profiles/index.html>`_ to execute your SQL in your data warehouse. +- **Execute the dbt commands**: Cosmos then executes the Dag, using the execution options in your :doc:`ExecutionConfig ` and :doc:`ProjectConfig ` to run dbt commands with the appropriate dbt adapter, finally resulting in your dbt SQL running in your data warehouse. Cosmos uses a connection defined in the :doc:`ProfileConfig ` to execute your SQL in your data warehouse. .. image:: ../_static/how_cosmos_works.png :alt: Diagram showing that Cosmos parses dbt projects, with options defined by the ProfileConfig and RenderConfig. Then, Cosmos executes the resulting Dag, with options defined by ExecutionConfig and ProjectConfig. @@ -28,7 +28,7 @@ You have a number of configuration options, but fundamentally, Cosmos provides t Quickstart ~~~~~~~~~~ -Even though Cosmos is highly extensible, and you have many advanced customization options, you can run a demo with the `Astro CLI `_ in just a few minutes. This demo introduces you to the key elements required for Cosmos to parse dbt projects and run Dags. +Even though Cosmos is highly extensible, and you have many advanced customization options, you can run a demo with the :doc:`Astro CLI ` in just a few minutes. This demo introduces you to the key elements required for Cosmos to parse dbt projects and run Dags. Bring your own project ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/getting_started/mwaa.rst b/docs/getting_started/mwaa.rst index 5e28ac2b94..14470086d4 100644 --- a/docs/getting_started/mwaa.rst +++ b/docs/getting_started/mwaa.rst @@ -3,7 +3,7 @@ Getting Started with Cosmos on Amazon Managed Workflows ======================================================= -Users can face Python dependency issues when trying to use the Cosmos `Local Execution Mode `_ in Amazon Managed Workflows for `Apache Airflow® `_ (MWAA). +Users can face Python dependency issues when trying to use the Cosmos :ref:`Local Execution Mode ` in Amazon Managed Workflows for `Apache Airflow® `_ (MWAA). This step-by-step illustrates how to use the Local Execution Mode, together with the `MWAA's startup script `_ and diff --git a/docs/getting_started/oss-quickstart.rst b/docs/getting_started/oss-quickstart.rst index 65328e6af6..802cb8080f 100644 --- a/docs/getting_started/oss-quickstart.rst +++ b/docs/getting_started/oss-quickstart.rst @@ -5,7 +5,7 @@ Run Cosmos in open-source Apache Airflow® Quickly get started working locally with `Apache Airflow® `_, dbt, and Astronomer Cosmos using the `Open-source Airflow `_. -This quickstart guide shows you how to set up a simple demo project, run a simple Dag locally with Cosmos, and then view the results with an open-source database viewer. If you want to get started working with your own project and configurations, see `Get started with Open-source Airflow `_. +This quickstart guide shows you how to set up a simple demo project, run a simple Dag locally with Cosmos, and then view the results with an open-source database viewer. If you want to get started working with your own project and configurations, see :doc:`Get started with Open-source Airflow `. By the end of this quickstart, you will: diff --git a/docs/guides/connect_database/index.rst b/docs/guides/connect_database/index.rst index b12944e1d3..0572e59028 100644 --- a/docs/guides/connect_database/index.rst +++ b/docs/guides/connect_database/index.rst @@ -5,8 +5,8 @@ Connect to your database Cosmos supports two methods of authenticating with your database: -- using your own `dbt profiles.yml <../../guides/connect_database/use-your-profiles-yml.html>`_ file -- using `Apache Airflow® `_ connections via Cosmos' `profile mappings <../../guides/connect_database/use-profile-mapping.html>`_ +- using your own :doc:`dbt profiles.yml ` file +- using `Apache Airflow® `_ connections via Cosmos' :doc:`profile mappings ` If you're already interacting with your database from Airflow and have a connection set up, it's recommended to use a profile mapping to translate that Airflow connection to a dbt profile. This is because it's easier to diff --git a/docs/guides/cosmos_devex/lineage.rst b/docs/guides/cosmos_devex/lineage.rst index cc50d870a9..7ac8d6eb4e 100644 --- a/docs/guides/cosmos_devex/lineage.rst +++ b/docs/guides/cosmos_devex/lineage.rst @@ -18,7 +18,7 @@ Known limitations ~~~~~~~~~~~~~~~~~ This feature is only available for the local -and virtualenv execution methods (read `execution modes <../run_dbt/execution-modes.html>`_ for more information). +and virtualenv execution methods (read :doc:`execution modes ` for more information). Additionally, since Cosmos uses the open-source `openlineage-integration-common `_, it relies on this library to support specific dbt adapters. As of 27 December 2024, the version 1.26.0 of this package supports: diff --git a/docs/guides/dbt_docs/generating-docs.rst b/docs/guides/dbt_docs/generating-docs.rst index 3b6051187c..b5d7c2463f 100644 --- a/docs/guides/dbt_docs/generating-docs.rst +++ b/docs/guides/dbt_docs/generating-docs.rst @@ -5,7 +5,7 @@ Generating Docs dbt allows you to generate static documentation on your models, tables, and more. You can read more about it in the `official dbt documentation `_. For an example of what the docs look like with the ``jaffle_shop`` project, check out `this site `_. -After generating the dbt docs, you can host them natively within `Apache Airflow® `_ via the Cosmos Airflow plugin; see `Hosting Docs `__ for more information. +After generating the dbt docs, you can host them natively within `Apache Airflow® `_ via the Cosmos Airflow plugin; see :doc:`Hosting Docs ` for more information. Alternatively, many users choose to serve these docs on a separate static website. This is a great way to share your data models with a broad array of stakeholders. diff --git a/docs/guides/dbt_docs/hosting-docs.rst b/docs/guides/dbt_docs/hosting-docs.rst index 3a46d4bcab..ed3b74f73d 100644 --- a/docs/guides/dbt_docs/hosting-docs.rst +++ b/docs/guides/dbt_docs/hosting-docs.rst @@ -138,7 +138,7 @@ Host from Cloud Storage For typical users, the recommended setup for hosting dbt docs would look like this: -1. Generate the docs via one of Cosmos' pre-built operators for generating dbt docs (see `Generating Docs `__ for more information) +1. Generate the docs via one of Cosmos' pre-built operators for generating dbt docs (see :doc:`Generating Docs ` for more information) 2. Wherever you dumped the docs, set your ``cosmos.dbt_docs_dir`` to that location. 3. If you want to use a conn ID other than the default connection, set your ``cosmos.dbt_docs_conn_id``. Otherwise, leave this blank. diff --git a/docs/guides/dbt_setup/dbt-fusion.rst b/docs/guides/dbt_setup/dbt-fusion.rst index cb297da278..b6367a719b 100644 --- a/docs/guides/dbt_setup/dbt-fusion.rst +++ b/docs/guides/dbt_setup/dbt-fusion.rst @@ -12,7 +12,7 @@ Context dbt Labs `launched `_ `dbt Fusion `_ on 28 May 2025. dbt Fusion is the next-generation dbt engine that enables real-time model validation, improved SQL parsing and state-aware orchestration. It is a unified approach that aims to merge dbt Core and dbt Cloud features via a completely new CLI, implemented in a different programming language (Rust, as opposed to Python). -As part of this, dbt Labs are `rewriting all dbt adapters `_ (equivalent to `Apache Airflow® `_ providers) in Rust, starting from Snowflake. +As part of this, dbt Labs are `rewriting all dbt adapters `_ (equivalent to `Apache Airflow® `_ providers) in Rust, starting from Snowflake. They are also changing the `licensing model `_ to a hybrid Open-Source and commercial license. We're supporting dbt Fusion with a license-complaint integration. This integration enables teams to: - Use dbt Fusion locally for enhanced development experience with real-time validation diff --git a/docs/guides/dbt_setup/execution-modes-local-conflicts.rst b/docs/guides/dbt_setup/execution-modes-local-conflicts.rst index 5502926060..fa0a9c4ab6 100644 --- a/docs/guides/dbt_setup/execution-modes-local-conflicts.rst +++ b/docs/guides/dbt_setup/execution-modes-local-conflicts.rst @@ -8,8 +8,8 @@ When using the :ref:`local-execution` without defining a custom ``ExecutionConfi If you find errors, we recommend users isolating the installation of dbt from the Airflow installation. With the ``local`` execution mode, this can be accomplished by installing dbt in a separate -Python virtualenv and setting the `ExecutionConfig.dbt_executable_path <../../reference/configs/execution-config.html>`_ and -`RenderConfig.dbt_executable_path <../../guides/translate_dbt_to_airflow/render-config.html>`_ parameters. +Python virtualenv and setting the :doc:`ExecutionConfig.dbt_executable_path ` and +:doc:`RenderConfig.dbt_executable_path ` parameters. The page, :ref:`execution-modes` describes many other methods that support isolating dbt from Airflow. diff --git a/docs/guides/translate_dbt_to_airflow/parsing-methods.rst b/docs/guides/translate_dbt_to_airflow/parsing-methods.rst index eba8135f11..a27cb763bd 100644 --- a/docs/guides/translate_dbt_to_airflow/parsing-methods.rst +++ b/docs/guides/translate_dbt_to_airflow/parsing-methods.rst @@ -142,7 +142,7 @@ Users can force Cosmos to run ``dbt ls`` with subprocess and not ``dbtRunner``, ) -For more information, check the `RenderConfig docs <./render-config.html>`_. +For more information, check the :doc:`RenderConfig docs `. ``dbt_ls_file`` diff --git a/docs/guides/translate_dbt_to_airflow/render-config.rst b/docs/guides/translate_dbt_to_airflow/render-config.rst index da4c1b673e..c19ffb00e2 100644 --- a/docs/guides/translate_dbt_to_airflow/render-config.rst +++ b/docs/guides/translate_dbt_to_airflow/render-config.rst @@ -10,11 +10,11 @@ It does this by exposing a ``cosmos.config.RenderConfig`` class that you can use The ``RenderConfig`` class takes the following arguments: - ``emit_datasets``: whether or not to emit Airflow datasets to be used for data-aware scheduling. Defaults to True. This feature is only available for ``ExecutionMode.LOCAL``, ``ExecutionMode.VIRTUALENV``, ``ExecutionMode.WATCHER`` and ``ExecutionMode.AIRFLOW_ASYNC``. -- ``test_behavior``: how to run tests. Defaults to running a model's tests immediately after the model is run. For more information, see the `Testing Behavior `_ section. -- ``load_method``: how to load your dbt project. See `Parsing Methods `_ for more information. +- ``test_behavior``: how to run tests. Defaults to running a model's tests immediately after the model is run. For more information, see the :doc:`Testing Behavior ` section. +- ``load_method``: how to load your dbt project. See :doc:`Parsing Methods ` for more information. - ``invocation_mode``: (new in v1.9) how to run ``dbt ls``, when using ``LoadMode.DBT_LS``. Learn more about this below. -- ``select`` and ``exclude``: which models to include or exclude from your DAGs. See `Selecting & Excluding `_ for more information. -- ``selector``: (new in v1.3) name of a dbt YAML selector to use for DAG parsing. Only supported when using ``load_method=LoadMode.DBT_LS``. See `Selecting & Excluding `_ for more information. +- ``select`` and ``exclude``: which models to include or exclude from your DAGs. See :doc:`Selecting & Excluding ` for more information. +- ``selector``: (new in v1.3) name of a dbt YAML selector to use for DAG parsing. Only supported when using ``load_method=LoadMode.DBT_LS``. See :doc:`Selecting & Excluding ` for more information. - ``dbt_deps``: (deprecated in v1.9, use ``ProjectConfig.install_dbt_deps`` onwards) A Boolean to run dbt deps when using dbt ls for dag parsing. Default True - ``node_converters``: a dictionary mapping a ``DbtResourceType`` into a callable. Users can control how to render dbt nodes in Airflow. Only supported when using ``load_method=LoadMode.DBT_MANIFEST`` or ``LoadMode.DBT_LS``. Find more information below. - ``node_conversion_by_task_group``: (new in v1.12.0) A boolean to control if node_converters are used at the task group level (ex. converting models with test_behavior=AFTER_EACH means the entire task group is converted including the run task and the test task), or the individual task level (gives more granularity for converting just the run tasks or just the test tasks). Defaults to True. @@ -29,7 +29,7 @@ The ``RenderConfig`` class takes the following arguments: - ``source_pruning``: When set to ``True``, automatically removes (or "prunes") any dbt source nodes from your Airflow DAG that do not have any downstream dependencies within the selected portion of the dbt graph. Defaults to ``False``. See :doc:`Source Nodes Rendering ` for more information. - ``normalize_task_id``: A callable that takes a dbt node as input and returns the task ID. This function allows users to set a custom task_id independently of the model name, which can be specified as the task's display_name. This way, task_id can be modified using a user-defined function, while the model name remains as the task's display name. The display_name parameter is available in Airflow 2.9 and above. See :doc:`Task display name ` for more information. - ``normalize_task_display_name``: This function allows users to set a custom user-defined function to alter the display name independently of the model name. This way, the task_id can be preserved while the model display name is modified. -- ``should_detach_multiple_parents_tests``: A boolean to control if tests that depend on multiple parents should be run as standalone tasks. See `Testing Behavior `_ for more information. +- ``should_detach_multiple_parents_tests``: A boolean to control if tests that depend on multiple parents should be run as standalone tasks. See :doc:`Testing Behavior ` for more information. - ``enable_owner_inheritance``: (introduced in 1.10.2) A boolean to control if dbt owners should be imported as part of the airflow DAG owners. Defaults to True. - ``node_conversion_by_task_group``: A boolean to control if node_converters are used at the task group level (ex. converting models with test_behavior=AFTER_EACH means the entire task group is converted including the run task and the test task), or the individual task level (gives more granularity for converting just the run tasks or just the test tasks). Defaults to True. - ``group_nodes_by_folder``: When enabled, groups nodes by folder structure, creating a ``TaskGroup`` per resource type and folder. Disabled by default. diff --git a/docs/index.rst b/docs/index.rst index 08a378c87b..e7e16d5e75 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -89,7 +89,7 @@ License `Apache License 2.0 `_ .. Tracking pixel for Scarf -.. See https://github.com/astronomer/astronomer-cosmos/blob/main/docs/policy/security-privacy.rst for more info about Cosmos privacy policy +.. See https://github.com/astronomer/astronomer-cosmos/blob/main/PRIVACY_NOTICE.rst for more info about Cosmos privacy policy .. raw:: html diff --git a/docs/optimize_performance/caching.rst b/docs/optimize_performance/caching.rst index 9c6f2a201b..1afbd56adb 100644 --- a/docs/optimize_performance/caching.rst +++ b/docs/optimize_performance/caching.rst @@ -219,7 +219,7 @@ The cache values contain a few properties: **Shared Cache Behavior** When using Airflow variables as the backend to store cached cosmos artifacts, both the dbt ls output and the YAML selectors cache will use the same variable. It should not be possible -to have both artifacts occupy the cache at the same time due to their distinct `RenderConfig.load_mode <./render-config.html>`_ and switching from using one cache to the other will invalidate the cache on the next version check. +to have both artifacts occupy the cache at the same time due to their distinct :doc:`RenderConfig.load_mode ` and switching from using one cache to the other will invalidate the cache on the next version check. Caching the partial parse file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -237,7 +237,7 @@ Users can customize where to store the cache using the setting ``AIRFLOW__COSMOS It is possible to switch off this feature by exporting the environment variable ``AIRFLOW__COSMOS__ENABLE_CACHE_PARTIAL_PARSE=0``. -For more information, read the `Cosmos partial parsing documentation <./partial-parsing.html>`_ +For more information, read the :doc:`Cosmos partial parsing documentation ` Caching the profiles diff --git a/docs/reference/configs/profile-config.rst b/docs/reference/configs/profile-config.rst index 52899734b8..639fa2582f 100644 --- a/docs/reference/configs/profile-config.rst +++ b/docs/reference/configs/profile-config.rst @@ -5,9 +5,9 @@ Profile Config Cosmos supports two methods of authenticating with your database: -- Using your `dbt profiles.yml <../../guides/connect_database/use-your-profiles-yml.html>`_ file -- using `Apache Airflow® `_ connections with the Cosmos `profile_mapping <../../guides/connect_database/use-profile-mapping.html>`_ +- Using your :doc:`dbt profiles.yml ` file +- Using `Apache Airflow® `_ connections with the Cosmos :doc:`profile_mapping ` -You can also customize the profiles used in each `dbt node <../../guides/connect_database/profile-customise-per-node.html>`_. +You can also customize the profiles used in each :doc:`dbt node `. -The `Profiles reference <../profiles/index.html>`_ contains information about the different profiles available for you to use in your database connections. +The :doc:`Profiles reference ` contains information about the different profiles available for you to use in your database connections. diff --git a/docs/reference/index.rst b/docs/reference/index.rst index d4d4eb4bce..5ff4a5b8d5 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -37,14 +37,14 @@ Configurations There are different configurations and profiles that you can use to configure how Cosmos works. -- `ProjectConfig `_: The ``ProjectConfig`` contains information about which dbt project a Cosmos Dag or task group is going to execute, as well as configurations that apply to both rendering and execution. -- `ExecutionConfig `_: The ``ExecutionConfig`` determines where and how the dbt commands are run within Cosmos. -- `CosmosConfig `_: This page lists available `Apache Airflow® `_ configurations that affect ``astronomer-cosmos`` behavior. You can set them in the ``airflow.cfg`` file or using environment variables. -- `ProfileConfig `_: The ``ProfileConfig`` class determines which data warehouse Cosmos connects to when it executes the dbt SQL. These docs include reference documentation for connecting to popular data warehouses you might use in your dbt code. +- :doc:`ProjectConfig `: The ``ProjectConfig`` contains information about which dbt project a Cosmos Dag or task group is going to execute, as well as configurations that apply to both rendering and execution. +- :doc:`ExecutionConfig `: The ``ExecutionConfig`` determines where and how the dbt commands are run within Cosmos. +- :doc:`CosmosConfig `: This page lists available `Apache Airflow® `_ configurations that affect ``astronomer-cosmos`` behavior. You can set them in the ``airflow.cfg`` file or using environment variables. +- :doc:`ProfileConfig `: The ``ProfileConfig`` class determines which data warehouse Cosmos connects to when it executes the dbt SQL. These docs include reference documentation for connecting to popular data warehouses you might use in your dbt code. Profiles ~~~~~~~~ The **Profiles** reference provides information about the different kinds of profile mappings available in Cosmos. These profile mappings are Airflow operators that map Airflow connections to dbt profiles, allowing you to work with resources in your data warehouses. -- `Profile mapping <./profiles/index.html>`_ +- :doc:`Profile mapping ` diff --git a/lychee.toml b/lychee.toml new file mode 100644 index 0000000000..a26da5e4a0 --- /dev/null +++ b/lychee.toml @@ -0,0 +1,38 @@ +# Configuration for the lychee link checker used by the docs "linkcheck" CI job +# (see .github/workflows/docs-build.yml). Check the docs locally with: +# lychee --config lychee.toml 'docs/**/*.rst' + +# Only check web links. The docs include example connection strings +# (postgres://, s3://, gs://, databricks://, ...) that are not real URLs. +scheme = ["http", "https"] + +# Give slow or occasionally throttling hosts a few chances before failing, so a +# transient network hiccup doesn't make this blocking check flaky. A lower +# concurrency keeps us from tripping per-host rate limiters. +max_retries = 4 +retry_wait_time = 3 +timeout = 30 +max_concurrency = 8 + +# Fail only on genuinely broken links. A 429 (rate limited) or 503/504 +# (service unavailable / gateway timeout) means the host is up but throttling +# automated requests — common for CI runners hitting docs CDNs — not that the +# link is broken. 4xx such as 404/403/410 are still treated as failures. +accept = ["200..=299", "429", "503", "504"] + +# Don't treat unreachable loopback / example hosts (e.g. http://localhost:8080) +# as failures. +exclude_loopback = true + +# URLs an automated checker cannot verify, plus intentional placeholders. +# Genuinely broken links are fixed in the docs, not excluded here. +exclude = [ + # Auth wall — Slack invite links return 403 to unauthenticated requests. + 'https://join\.slack\.com', + # Flaky / rate-limited from CI — intermittent TLS failures. + 'https://ossrank\.com', + # Bot-blocked — the Bitnami Helm chart repo returns 403 to non-Helm clients. + 'https://charts\.bitnami\.com', + # Intentional placeholder in example snippets, not a real repository. + 'https://github\.com/your-org/', +]