Add cosmos/propagate_logs Airflow config support for disabling log pr… by agreenburg · Pull Request #648 · astronomer/astronomer-cosmos

agreenburg · 2023-11-03T21:33:53Z

…opagation if desired

Description

Add Airflow config check for cosmos/propagate_logs to allow override of default propagation behavior.
Expose entry-point so that Airflow can theoretically detect configuration default.

Related Issue(s)

closes #639

Breaking Change?

This is backward-compatible as it falls back to default behavior if the cosmos section or propagate_logs option don't exist.

Checklist

I have made corresponding changes to the documentation (if required)
I have added tests that prove my fix is effective or that my feature works

…opagation if desired

netlify · 2023-11-03T21:33:58Z

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name	Link
🔨 Latest commit	`eb5e64d`
🔍 Latest deploy log	https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/654ceac79f05e60008e86e25

codecov · 2023-11-06T15:57:14Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9cd46d2) 92.67% compared to head (eb5e64d) 92.70%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #648      +/-   ##
==========================================
+ Coverage   92.67%   92.70%   +0.02%     
==========================================
  Files          54       54              
  Lines        2212     2220       +8     
==========================================
+ Hits         2050     2058       +8     
  Misses        162      162

Files	Coverage Δ
cosmos/__init__.py	`100.00% <100.00%> (ø)`
cosmos/log.py	`100.00% <100.00%> (ø)`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tatiana

Thank you very much @agreenburg for making this improvement to Cosmos logging.

Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/

Could you give more details on the following?
"Expose entry-point so that Airflow can theoretically detect configuration default."

I'm happy to merge this change once I understand this a little bit more :)

agreenburg · 2023-11-08T15:28:31Z

Thank you very much @agreenburg for making this improvement to Cosmos logging.

Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/

Could you give more details on the following? "Expose entry-point so that Airflow can theoretically detect configuration default."

I'm happy to merge this change once I understand this a little bit more :)

HI @tatiana ! The entry-point stuff is based on the guidance here: https://airflow.apache.org/docs/apache-airflow-providers/howto/create-custom-providers.html#how-to-create-a-provider

I have not thoroughly tested that airflow is detecting it properly yet though.

tatiana · 2023-11-08T16:44:26Z

Thank you very much @agreenburg for making this improvement to Cosmos logging.
Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/
Could you give more details on the following? "Expose entry-point so that Airflow can theoretically detect configuration default."
I'm happy to merge this change once I understand this a little bit more :)

HI @tatiana ! The entry-point stuff is based on the guidance here: https://airflow.apache.org/docs/apache-airflow-providers/howto/create-custom-providers.html#how-to-create-a-provider

I have not thoroughly tested that airflow is detecting it properly yet though.

Ah, I see - so the motivation is to display the config in the Airflow UI (e.g. http://localhost:8080/configuration). If we didn't add this provider logic (both entry point and get_provider_info), you'd still be able to set this value via an environment variable or Airflow config file, but it would not display in the UI. Makes sense!

…opagation if desired (#648) Add Airflow config check for cosmos/propagate_logs to allow override of default propagation behavior. Expose entry-point so that Airflow can theoretically detect configuration default. Closes #639 ## Breaking Change? This is backward-compatible as it falls back to default behavior if the `cosmos` section or `propagate_logs` option don't exist. ## Checklist - [X] I have made corresponding changes to the documentation (if required) - [X] I have added tests that prove my fix is effective or that my feature works --------- Co-authored-by: Andrew Greenburg <agreenburg@vergeventures.net>

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 and #688 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728 * Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648 * Add operator_args full_refresh as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735 Enhancements * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730 * Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731 * Add aws_session_token for Athena mapping by @benjamin-awd in #663 Others * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 * Speed up integration tests by @jbandoro in #732

@perttus

Features * Add ProfileMapping for Vertica by @perttus in #540 and #688 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728 * Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648 * Add operator_args full_refresh as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735 Enhancements * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730 * Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731 * Add aws_session_token for Athena mapping by @benjamin-awd in #663 Others * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 * Speed up integration tests by @jbandoro in #732

@DanMawdsleyBA

**Features** * Add `ProfileMapping` for Snowflake encrypted private key path by @ivanstillfront in #608 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in #616 * Add support to select using (some) graph operators when using `LoadMode.CUSTOM` and `LoadMode.DBT_MANIFEST` by @tatiana in #728 * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in #648 * Add operator_args ``full_refresh`` as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in #735 **Enhancements** * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to `dbt_packages` when `dbt_deps` is False when using `LoadMode.DBT_LS` by @DanMawdsleyBA in #730 * Support no `profile_config` for `ExecutionMode.KUBERNETES` and `ExecutionMode.DOCKER` by @MrBones757 and @tatiana in #681 and #731 * Add `aws_session_token` for Athena mapping by @benjamin-awd in #663 **Others** * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 * Speed up integration tests by @jbandoro in #732

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in #733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in #728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in #755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in #540, #688 and #741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in #608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in #616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in #648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in #623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in #735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in #768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in #736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in #730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in #663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in #758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in #759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in #772 **Others** * Replace flake8 for Ruff by @joppevos in #743 * Reduce code complexity to 8 by @joppevos in #738 * Speed up integration tests by @jbandoro in #732 * Fix README quickstart link in by @RNHTTR in #776 * Add package location to work with hatchling 1.19.0 by @jbandoro in #761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in #766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in #757 * Update conflict matrix between Airflow and dbt versions by @tatiana in #731 and #779 * pre-commit updates in #775, #770, #762

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in astronomer#755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in astronomer#608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in astronomer#616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in astronomer#648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in astronomer#623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in astronomer#735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in astronomer#768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in astronomer#736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in astronomer#758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in astronomer#759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in astronomer#772 **Others** * Replace flake8 for Ruff by @joppevos in astronomer#743 * Reduce code complexity to 8 by @joppevos in astronomer#738 * Speed up integration tests by @jbandoro in astronomer#732 * Fix README quickstart link in by @RNHTTR in astronomer#776 * Add package location to work with hatchling 1.19.0 by @jbandoro in astronomer#761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in astronomer#766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757 * Update conflict matrix between Airflow and dbt versions by @tatiana in astronomer#731 and astronomer#779 * pre-commit updates in astronomer#775, astronomer#770, astronomer#762

zy-wiser · 2024-02-05T20:35:15Z

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

agreenburg · 2024-02-05T20:39:16Z

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

zy-wiser · 2024-02-05T20:42:50Z

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

Got it thanks.

I used the following command to check airflow config and it looks good to me (can get False as return)

airflow config get-value cosmos propagate_logs
False

But actual log still shows doubled

[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.522+0000�[0m] {�[34mlocal.py:�[0m253} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.523+0000�[0m] {�[34mlocal.py:�[0m254} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Outlets: []�[0m

agreenburg · 2024-02-05T21:29:10Z

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

Got it thanks.

I used the following command to check airflow config and it looks good to me (can get False as return)
airflow config get-value cosmos propagate_logs
False
But actual log still shows doubled
[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.522+0000�[0m] {�[34mlocal.py:�[0m253} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.523+0000�[0m] {�[34mlocal.py:�[0m254} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Outlets: []�[0m

Those are two separate log messages. One with the Inlets[] and the other with the Outlets[]. Before the change I was seeing logs like this:

[2023-12-04, 19:08:25 UTC] {log.py:232} WARNING - [�[34m2023-12-04T19:08:25.566+0000�[0m] {�[34mlocal.py:�[0m244} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2023-12-04, 19:08:25 UTC] {local.py:244} INFO - Inlets: []

zy-wiser · 2024-02-05T21:37:01Z

@agreenburg Thanks for the explanation. I guess I misunderstood the outcome with the setting.
Was thinking I can limit it to just showing

[2023-12-04, 19:08:25 UTC] {local.py:244} INFO - Inlets: []

and completely get rid of the cosmos log

[2023-12-04, 19:08:25 UTC] {log.py:232} WARNING - [�[34m2023-12-04T19:08:25.566+0000�[0m] {�[34mlocal.py:�[0m244} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m

as it's not that readable and format well...

agreenburg · 2024-02-05T21:39:25Z

@zy-wiser yes, unfortunately that would require a bigger change. I guess the custom logging wasn't designed with the celery executor in mind.

…opagation if desired (astronomer#648) Add Airflow config check for cosmos/propagate_logs to allow override of default propagation behavior. Expose entry-point so that Airflow can theoretically detect configuration default. Closes astronomer#639 ## Breaking Change? This is backward-compatible as it falls back to default behavior if the `cosmos` section or `propagate_logs` option don't exist. ## Checklist - [X] I have made corresponding changes to the documentation (if required) - [X] I have added tests that prove my fix is effective or that my feature works --------- Co-authored-by: Andrew Greenburg <agreenburg@vergeventures.net>

@DanMawdsleyBA

**Features** * Add `ProfileMapping` for Snowflake encrypted private key path by @ivanstillfront in astronomer#608 * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in astronomer#616 * Add support to select using (some) graph operators when using `LoadMode.CUSTOM` and `LoadMode.DBT_MANIFEST` by @tatiana in astronomer#728 * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in astronomer#648 * Add operator_args ``full_refresh`` as a templated field by @joppevos in astronomer#623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in astronomer#735 **Enhancements** * Make Pydantic an optional dependency by @pixie79 in astronomer#736 * Create a symbolic link to `dbt_packages` when `dbt_deps` is False when using `LoadMode.DBT_LS` by @DanMawdsleyBA in astronomer#730 * Support no `profile_config` for `ExecutionMode.KUBERNETES` and `ExecutionMode.DOCKER` by @MrBones757 and @tatiana in astronomer#681 and astronomer#731 * Add `aws_session_token` for Athena mapping by @benjamin-awd in astronomer#663 **Others** * Replace flake8 for Ruff by @joppevos in astronomer#743 * Reduce code complexity to 8 by @joppevos in astronomer#738 * Update conflict matrix between Airflow and dbt versions by @tatiana in astronomer#731 * Speed up integration tests by @jbandoro in astronomer#732

@woogakoki

**Features** * Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)). * Add support to select using (some) graph operators when using ``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude)) * Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in astronomer#755, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)). * Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741, as ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)). * Add ``ProfileMapping`` for Snowflake encrypted private key path by @ivanstillfront in astronomer#608, as ([documentation]( https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)). * Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in astronomer#649 * Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro in astronomer#616, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)). * Add cosmos/propagate_logs Airflow config support for disabling log propagation by @agreenburg in astronomer#648, ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)). * Add operator_args ``full_refresh`` as a templated field by @joppevos in astronomer#623 * Expose environment variables and dbt variables in ``ProjectConfig`` by @jbandoro in astronomer#735 ([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)). * Support disabling event tracking when using Cosmos profile mapping by @jbandoro in astronomer#768, ([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)). **Enhancements** * Make Pydantic an optional dependency by @pixie79 in astronomer#736 * Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730 * Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663 * Retrieve temporary credentials from ``conn_id`` for Athena by @octiva in astronomer#758 * Extend ``DbtDocsLocalOperator`` with static flag by @joppevos in astronomer#759 **Bug fixes** * Remove Pydantic upper version restriction so Cosmos can be used with Airflow 2.8 by @jlaneve in astronomer#772 **Others** * Replace flake8 for Ruff by @joppevos in astronomer#743 * Reduce code complexity to 8 by @joppevos in astronomer#738 * Speed up integration tests by @jbandoro in astronomer#732 * Fix README quickstart link in by @RNHTTR in astronomer#776 * Add package location to work with hatchling 1.19.0 by @jbandoro in astronomer#761 * Fix type check error in ``DbtKubernetesBaseOperator.build_env_args`` by @jbandoro in astronomer#766 * Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757 * Update conflict matrix between Airflow and dbt versions by @tatiana in astronomer#731 and astronomer#779 * pre-commit updates in astronomer#775, astronomer#770, astronomer#762

## Summary `cosmos/provider_info.py` currently claims that the `propagate_logs` config option was added in `1.3.0a1` and deprecated in `1.6.0a1`. Both references pre-date the changes they describe by weeks; switching them to the GA versions (`1.3.0` and `1.6.0`) makes the metadata accurate and matches the version style every other Airflow provider uses in `provider.yaml`. ## Evidence from the release history | Claim in `provider_info.py` | What actually happened | |---|---| | `"version_added": "1.3.0a1"` | `1.3.0a1` tagged 2023-10-27. The commit that introduced `propagate_logs` (`d063b5ed`, PR #648) merged on 2023-11-09 — **two weeks later**. First alpha containing it: `1.3.0a2`. First GA: `1.3.0` (2024-01-04). | | `"version_deprecated": "1.6.0a1"` | `1.6.0a1` tagged 2024-07-05. The "Simplify logging" change (`89f5999e`, PR #1108) that made the option redundant merged 2024-08-15 — **six weeks later**. First alpha with the deprecation: `1.6.0a7`. First GA: `1.6.0` (both on 2024-08-20). | The existing `deprecation_reason` string already refers to `Cosmos 1.6.0` (no alpha suffix), so the file's own narrative already assumes the GA boundary — this PR just makes the structured fields agree with that. ## Diff ```diff - "version_added": "1.3.0a1", - "version_deprecated": "1.6.0a1", + "version_added": "1.3.0", + "version_deprecated": "1.6.0", ``` ## Test plan - [x] `hatch -e tests.py3.11-2.10-1.9 run pre-commit run --files cosmos/provider_info.py` — all hooks Passed (including `mypy-python`) - [x] Diff is exactly the two string literals; no other changes - [ ] CI green ## Breaking change? No. The `version_added` / `version_deprecated` fields are documentation metadata surfaced through Airflow's `airflow config list` and the provider listing — they don't affect runtime behavior. The option itself remains deprecated either way. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Add cosmos/propagate_logs Airflow config support for disabling log pr…

f57d03d

…opagation if desired

agreenburg requested a review from a team as a code owner November 3, 2023 21:33

agreenburg requested a review from a team November 3, 2023 21:33

agreenburg had a problem deploying to external November 3, 2023 21:33 — with GitHub Actions Error

🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

b9f3ed0

pre-commit-ci Bot had a problem deploying to external November 3, 2023 21:34 Error

Merge branch 'main' into issue_639/propagate_logs_config

416f4fd

agreenburg temporarily deployed to external November 3, 2023 21:38 — with GitHub Actions Inactive

Code coverage

a919598

agreenburg temporarily deployed to external November 6, 2023 16:15 — with GitHub Actions Inactive

tatiana added this to the 1.3.0 milestone Nov 7, 2023

tatiana reviewed Nov 8, 2023

View reviewed changes

Merge branch 'main' into issue_639/propagate_logs_config

eb5e64d

tatiana temporarily deployed to external November 9, 2023 14:21 — with GitHub Actions Inactive

tatiana approved these changes Nov 9, 2023

View reviewed changes

tatiana added the area:logging Related to logging, like log levels, log formats, error logging, etc label Nov 9, 2023

tatiana merged commit 6a2c866 into astronomer:main Nov 9, 2023

tatiana mentioned this pull request Dec 7, 2023

Release 1.3.0a3 #753

Merged

tatiana mentioned this pull request Jan 4, 2024

Release 1.3.0 #781

Merged

pankajastro mentioned this pull request May 20, 2026

Use GA versions for propagate_logs in provider_info #2712

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cosmos/propagate_logs Airflow config support for disabling log pr…#648

Add cosmos/propagate_logs Airflow config support for disabling log pr…#648
tatiana merged 5 commits into
astronomer:mainfrom
agreenburg:issue_639/propagate_logs_config

agreenburg commented Nov 3, 2023 •

edited

Loading

Uh oh!

netlify Bot commented Nov 3, 2023 •

edited

Loading

Uh oh!

codecov Bot commented Nov 6, 2023 •

edited

Loading

Uh oh!

tatiana left a comment

Uh oh!

agreenburg commented Nov 8, 2023

Uh oh!

tatiana commented Nov 8, 2023

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

agreenburg commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Breaking Change?

Checklist

Uh oh!

netlify Bot commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Uh oh!

codecov Bot commented Nov 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tatiana left a comment

Choose a reason for hiding this comment

Uh oh!

agreenburg commented Nov 8, 2023

Uh oh!

tatiana commented Nov 8, 2023

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

zy-wiser commented Feb 5, 2024

Uh oh!

agreenburg commented Feb 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

agreenburg commented Nov 3, 2023 •

edited

Loading

netlify Bot commented Nov 3, 2023 •

edited

Loading

codecov Bot commented Nov 6, 2023 •

edited

Loading