Improve dataset/asset driven scheduling documentation by tatiana · Pull Request #1729 · astronomer/astronomer-cosmos

tatiana · 2025-04-30T13:16:06Z

Improve asset/dataset event scheduling after implementing #1713

Copilot

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)

docs/configuration/scheduling.rst: Language not supported

netlify · 2025-04-30T13:16:21Z

✅ Deploy Preview for sunny-pastelito-5ecb04 ready!

Name	Link
🔨 Latest commit	`700d60c`
🔍 Latest deploy log	https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/6813500cf1b3940008c061fd
😎 Deploy Preview	https://deploy-preview-1729--sunny-pastelito-5ecb04.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

cloudflare-workers-and-pages · 2025-04-30T13:17:40Z

Deploying astronomer-cosmos with Cloudflare Pages

Latest commit:	`700d60c`
Status:	✅ Deploy successful!
Preview URL:	https://3e0d790d.astronomer-cosmos.pages.dev
Branch Preview URL:	https://docs-alias-support.astronomer-cosmos.pages.dev

View logs

codecov · 2025-04-30T13:46:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.59%. Comparing base (bfd3662) to head (b0bb72f).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1729   +/-   ##
=======================================
  Coverage   97.59%   97.59%           
=======================================
  Files          83       83           
  Lines        5166     5166           
=======================================
  Hits         5042     5042           
  Misses        124      124

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tatiana · 2025-04-30T15:08:50Z

@pankajkoti let's wait for the investigation on
#1730

Before we merge this, the documentation can be updated after the release.

pankajkoti · 2025-04-30T15:27:01Z

@pankajkoti let's wait for the investigation on #1730

Before we merge this, the documentation can be updated after the release.

@tatiana Should we put it in draft?

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>

tatiana · 2025-05-01T10:45:48Z

@pankajkoti I finished the investigation and I'm merging this for 1.10.0. We'll align to discuss the changes tomorrow.

@tatiana

Features * Airflow 3 support * Support running ``dbt deps`` incrementally to pre-defined ``dbt_packages`` by @tatiana in #1668 and #1670 * Add ``DuckDB`` profile mapping by @prithvijitguha and @pankajastro in #1553 * Implement DBT exposure selector by ghjklw #1717 Bug Fixes * Fix ``test_indirect_selection`` flag to be propagated in case of ``TestBehavior.BUILD`` by @corsettigyg in #1663 * Fix ``select`` clause in the case of detached tests by @anyapriya in #1680 * Operator argument fixes by @johnhoran in #1648 Airflow 3 Support * Support rendering DbtDag in Airflow 3 by @tatiana and @ashb in #1657 * Refactor Rendered Task Instance Fields (RTIF) handling for Airflow 2.x and 3.x by @pankajkoti in #1661 * Run cosmos operator in Airflow 3 by @pankajastro in #1642 * Fix ``python_virtualenv.prepare_env`` top-level import for Airflow 3 by @pankajkoti in #1678 * Fix Variable not found issue in Airflow 3 by @tatiana in #1684 * Disable CosmosPlugin on Airflow 3 setup by @pankajkoti in #1692, #1698 * Use ``schedule`` param in example DAGs instead of the 2.10 deprecated and 3.0 removed ``schedule_interval`` by @pankajkoti in #1701 * Ensure ``virtualenv_dir`` path exists by @pankajkoti in #1724 * Support emitting Assets with Airflow 3 by @tatiana in #1713 * Add docs on Airflow 3 compatibility by @pankajkoti and @tatiana in #1731 * Introduce, test and document asset/dataset breaking change by @tatiana in #1672 * Improve dataset/asset driven scheduling documentation by @tatiana in #1729 Enhancements * Allow multiple callbacks by @corsettigyg #1693 * Refactor kubernetes warning callback handling by @canbekley in #1681 Documentation * Add documentation related to ``copy_dbt_packages`` by @tatiana in #1671 * Make wording and command consistent in the contributing doc by @pankajkoti in #1697 * Add MonteCarlo callback example for importing dbt artifacts by @corsettigyg #1695 * Change async feature to be non-experimental by @tatiana in #1732 Others * Add sample ``dbt_packages`` to validate incremental ``dbt deps`` by @tatiana in #1669 * Add kubernetes execution mode example in Airflow 3 by @pankajastro in #1667 * Check only major version until Airflow 3 stable release by @pankajastro in #1665 * Install Airflow from main branch by @pankajastro in #1660 * Add dev tool for Airflow 3 by @pankajastro and @tatiana in #1627 * Improve Airflow 3 tooling by @pankajastro in #1656 * Skip associating ``openlineage_events_completes`` to ``ti`` in Airflow 3 by @pankajkoti in #1662 * Add .gitignore file for the scripts/airflow3 directory by @pankajkoti in #1658 * Remove ``original_jaffle_shop`` dbt project by @pankajkoti in #1676 * Fix or ignore type check error by @pankajastro in #1687 * Run virtualenv example with Airflow 3 tooling by @pankajastro in #1686 * Enable running setup/teardown tasks with Async execution DAG with Airflow 3 tooling by @pankajastro in #1696 * Enable integration tests for the DuckDB adapter by @pankajastro in #1699 * Add Airflow 3 tests matrix entries in CI by @pankajkoti in #1646 * Use a different way to get tasks count for asserting test_perf_dag by @pankajkoti in #1714 * Reinstall Airflow 3 dependency on ``pydantic>=2.11`` for dbt adapter versions 1.6 & 1.9 by @pankajkoti in #1715 * Fix outdated ``echo`` in Airflow 3 tooling script #1700 * Add files not needed for git tracking to .gitignore by @pankajkoti in #1723 * Use latest minor versions for dbt adapters to get in compatibility fixes by @pankajkoti in #1719 * Fix Airflow 3 tests raising generate_run_id() takes 0 positional arguments by @tatiana in #1725 * Fix dataset tests failing in Airflow 3 by @tatiana in #1716 * Enable example DAGs to run in CI that were disabled in PR #1646 by @pankajkoti in #1726 * Pre-commit updates: #1666, #1653, #1641, #1682, #1720 Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com> Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com> --------- Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>

Copilot AI review requested due to automatic review settings April 30, 2025 13:16

Copilot AI reviewed Apr 30, 2025

View reviewed changes

dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Apr 30, 2025

tatiana temporarily deployed to internal April 30, 2025 13:16 — with GitHub Actions Inactive

dosubot Bot added area:datasets Related to the Airflow datasets feature/module area:docs Relating to documentation, changes, fixes, improvement labels Apr 30, 2025

tatiana temporarily deployed to internal April 30, 2025 13:18 — with GitHub Actions Inactive

tatiana temporarily deployed to internal April 30, 2025 13:21 — with GitHub Actions Inactive

pankajkoti approved these changes Apr 30, 2025

View reviewed changes

Comment thread docs/configuration/scheduling.rst Outdated

tatiana temporarily deployed to internal April 30, 2025 14:56 — with GitHub Actions Inactive

tatiana marked this pull request as draft April 30, 2025 15:36

tatiana added this to the Cosmos 1.10.1 milestone Apr 30, 2025

tatiana temporarily deployed to internal May 1, 2025 10:27 — with GitHub Actions Inactive

pre-commit-ci Bot temporarily deployed to internal May 1, 2025 10:28 Inactive

tatiana temporarily deployed to internal May 1, 2025 10:32 — with GitHub Actions Inactive

tatiana commented May 1, 2025

View reviewed changes

Comment thread docs/configuration/scheduling.rst

tatiana temporarily deployed to internal May 1, 2025 10:33 — with GitHub Actions Inactive

tatiana and others added 7 commits May 1, 2025 11:35

Improve dataset scheduling documentation

c3d6b12

Improve docs

04fbd5a

Update docs/configuration/scheduling.rst

a1f99a5

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>

Update scheduling.rst

3e95e3a

🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

e6aa9c6

Add docs for AIRFLOW__COSMOS__USE_DATASET_AIRFLOW3_URI_STANDARD

86b803b

Add breaking change to AF3 compatibility

700d60c

tatiana marked this pull request as ready for review May 1, 2025 10:42

dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels May 1, 2025

tatiana force-pushed the docs-alias-support branch from bb98b08 to 700d60c Compare May 1, 2025 10:42

tatiana temporarily deployed to internal May 1, 2025 10:42 — with GitHub Actions Inactive

tatiana merged commit 69e08e2 into main May 1, 2025
87 of 90 checks passed

tatiana deleted the docs-alias-support branch May 1, 2025 10:45

tatiana mentioned this pull request May 1, 2025

Release 1.10.0 #1672

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve dataset/asset driven scheduling documentation#1729

Improve dataset/asset driven scheduling documentation#1729
tatiana merged 7 commits into
mainfrom
docs-alias-support

tatiana commented Apr 30, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

netlify Bot commented Apr 30, 2025 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages Bot commented Apr 30, 2025 •

edited

Loading

Uh oh!

codecov Bot commented Apr 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

tatiana commented Apr 30, 2025

Uh oh!

pankajkoti commented Apr 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

tatiana commented May 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tatiana commented Apr 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

netlify Bot commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for sunny-pastelito-5ecb04 ready!

Uh oh!

cloudflare-workers-and-pages Bot commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying astronomer-cosmos with Cloudflare Pages

Uh oh!

codecov Bot commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

tatiana commented Apr 30, 2025

Uh oh!

pankajkoti commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

tatiana commented May 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify Bot commented Apr 30, 2025 •

edited

Loading

cloudflare-workers-and-pages Bot commented Apr 30, 2025 •

edited

Loading

codecov Bot commented Apr 30, 2025 •

edited

Loading

pankajkoti commented Apr 30, 2025 •

edited

Loading