Allow to push URIs as XCOM of cosmos tasks

[corsettigyg]

Description

At Getyourguide we were using airflow 2.9 and during the bump to airflow 2.11, we noticed that some of our auxiliary tasks that were reading the XCom and parsing the uri field started failing. On a quick investigation, this PR explains the change that took effect from airflow 2.10 >= with the change from Dataset to DatasetAlias.

We have been using the uri produced by the XCom field to parse the full table path, since it contains the catalog, schema and table name, greatly helping us to manage operations like VACUUM and OPTIMIZE on those tables after the dbt tasks were finished.

The idea of this PR is to re-enable the emission of the uri field to the task XCom so users can have this pre-formatted information available in case they want to consume it downstream.

I have tested it in my local Airflow and this implementation worked fine. Open for feedbacks from the community though !

Breaking Change?

Should not be since the default is False. Only remark is that I am using the walrus operator to avoid nested ifs and it is only supported from python 3.8>=, although I believe cosmos do not use python 3.7<= anymore.

Checklist

• I have made corresponding changes to the documentation (if required)
• I have added tests that prove my fix is effective or that my feature works

[netlify]

✅ Deploy Preview for astronomer-cosmos ready!

Name	Link
🔨 Latest commit	d08b18b
🔍 Latest deploy log	https://app.netlify.com/projects/astronomer-cosmos/deploys/69787e47c51c010008a3d1b8
😎 Deploy Preview	https://deploy-preview-2275--astronomer-cosmos.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

This pull request adds an opt-in feature to emit dataset URIs as XCom values from Cosmos tasks. This addresses a regression experienced when upgrading from Airflow 2.9 to 2.11, where the URI field that was previously available in XCom was no longer accessible due to the Dataset to DatasetAlias migration.

Changes:

• Added enable_uri_xcom configuration setting (default: False) to control URI emission to XCom
• Modified _handle_datasets method to push outlet URIs to XCom when the feature is enabled
• Added comprehensive test coverage for the new feature across different scenarios
• Added documentation explaining how to enable and use the feature

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
cosmos/settings.py	Adds new enable_uri_xcom boolean configuration setting (default: False)
cosmos/operators/local.py	Implements XCom push logic in _handle_datasets to emit outlet URIs when enabled
tests/operators/test_local.py	Adds 4 comprehensive tests covering enabled/disabled states and edge cases
docs/configuration/scheduling.rst	Documents the new feature with configuration examples and usage patterns

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tatiana]

HI @corsettigyg thanks a lot for adding this. I'll take a look on this on Friday

[corsettigyg]

@tatiana no worries 🍷 take your name ! it is a very simple change overall

[tatiana]

@corsettigyg I'm happy for us to ship this change in Cosmos 1.13.0, which we're aiming to have released on Thursday. Please address the Copilot feedback and have the checks pass; it would be great, and I'll approve, and we can go ahead and merge it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.02%. Comparing base (4134018) to head (699f9d0).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2275   +/-   ##
=======================================
  Coverage   98.02%   98.02%           
=======================================
  Files         100      100           
  Lines        6420     6424    +4     
=======================================
+ Hits         6293     6297    +4     
  Misses        127      127           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[corsettigyg]

@tatiana done 🚀

[tatiana]

Looks great, thanks a lot for making this a backwards-compatible feature, @corsettigyg! We can consider exposing these XComs as the default in Cosmos 2.0. Before this, though, it would be great to understand the long-term impact of storing this data in the Airflow metadata database - do you have any suggestions for us to monitor this?

[corsettigyg]

performance-wise, I'd expect low to no impact considering how simple the xcom is. We will have it enabled on Getyourguide so I can keep track of it here, but on a broader perspective, the only concern I can think about is that some companies might not want to expose this information globally due to data privacy.