Skip to content

Add dbt Core 1.11 to test matrix#2230

Merged
tatiana merged 8 commits into
mainfrom
support-dbt-1.11
Dec 29, 2025
Merged

Add dbt Core 1.11 to test matrix#2230
tatiana merged 8 commits into
mainfrom
support-dbt-1.11

Conversation

@tatiana
Copy link
Copy Markdown
Collaborator

@tatiana tatiana commented Dec 29, 2025

There have been a few releases of dbt Core 1.11 since December 19th:
Screenshot 2025-12-29 at 10 17 54

This PR aims to validate the latest dbt version against Cosmos tests to identify potential issues.

We temporarily enabled running tests against this branch to confirm that GH Actions Jobs were working. Example of CI job run:
https://github.com/astronomer/astronomer-cosmos/actions/runs/20570466441

@netlify
Copy link
Copy Markdown

netlify Bot commented Dec 29, 2025

Deploy Preview for astronomer-cosmos canceled.

Name Link
🔨 Latest commit d96a38e
🔍 Latest deploy log https://app.netlify.com/projects/astronomer-cosmos/deploys/69526f96872ece00081113a4

@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.98%. Comparing base (ebc74f9) to head (3532f08).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2230   +/-   ##
=======================================
  Coverage   97.98%   97.98%           
=======================================
  Files          95       95           
  Lines        6190     6190           
=======================================
  Hits         6065     6065           
  Misses        125      125           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tatiana tatiana marked this pull request as ready for review December 29, 2025 10:49
Copilot AI review requested due to automatic review settings December 29, 2025 10:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds dbt-core 1.11 to the testing matrix to validate compatibility with the latest dbt version. The changes update both the project configuration and CI workflow to include dbt 1.11 in automated testing.

Key Changes:

  • Added dbt 1.11 to the test matrix in project configuration
  • Updated CI workflow to test against dbt 1.11 across different Python and Airflow versions
  • Temporarily enabled CI runs on the support branch for validation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
pyproject.toml Added "1.11" to the dbt version matrix for comprehensive testing across all supported versions
.github/workflows/test.yml Updated dbt-version parameters to "1.11" in multiple test jobs and added temporary branch trigger for CI validation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated
@tatiana tatiana changed the title Add dbt-core 1.11 to test matrix Add dbt Core 1.11 to test matrix Dec 29, 2025
Copy link
Copy Markdown
Contributor

@pankajkoti pankajkoti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Inline suggestions to also preserve 1.10, along with 1.11, to run the Unit Integration and Kubernetes tests since it's still a supported version. WDYT?

Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml
Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
tatiana and others added 2 commits December 29, 2025 12:03
Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated
@tatiana
Copy link
Copy Markdown
Collaborator Author

tatiana commented Dec 29, 2025

Thanks for the quick review, @pankajkoti !

Looks good. Inline suggestions to also preserve 1.10, along with 1.11, to run the Unit Integration and Kubernetes tests, since it's still a supported version. WDYT?

For unit tests, it does not make sense because they should not rely on dbt being installed.

For integration tests, it would be too expensive to run them against all supported versions of Python and Airflow (currently, the matrix spans 32 jobs). By having two dbt versions, we'd have 64 - if we were to add all dbt versions that didn't reach EOL. It would be 128).

In most cases, Cosmos does not rely on specific features of dbt versions - except for two scenarios:

  • the async implementation
  • dbt 1.5
    Both of which have dedicated integration test jobs.

I'm happy for us to have a more extensive dbt version testing in the CI. That said, I don't believe the solution is to multiply the existing Py & Airflow matrix by also having multiple dbt versions.

WDYT of logging a dedicated ticket for us to review and develop a better strategy? We could run a subset of integration tests with a varying matrix of AF & dbt, but pin the Python version. It feels this is outside of the scope of the current ticket.

@pankajkoti
Copy link
Copy Markdown
Contributor

Sounds all good to me @tatiana 👍🏽

@tatiana
Copy link
Copy Markdown
Collaborator Author

tatiana commented Dec 29, 2025

Logged: #2234

@tatiana tatiana merged commit d7fa4cb into main Dec 29, 2025
85 checks passed
@tatiana tatiana deleted the support-dbt-1.11 branch December 29, 2025 12:35
@tatiana tatiana added this to the Cosmos 1.13.0 milestone Jan 29, 2026
@pankajastro pankajastro mentioned this pull request Jan 29, 2026
tatiana added a commit that referenced this pull request Jan 30, 2026
Features

* Support cross-referencing models across dbt projects using dbt-loom by
@pankajkoti in #2271
* Support use of YAML selectors when using ``LoadMode.DBT_MANIFEST`` by
@YourRoyalLinus in #2261
* Introduce ``ExecutionMode.WATCHER_KUBERNETES`` to use the watcher with
``KubernetesPodOperator`` by @tatiana in #2207
* Add support for StarRocks profile mapping by @kurkim0661 in #2256
* Allow pushing URIs as XComs for Cosmos tasks by @corsettigyg in #2275
* Support defining custom callbacks alongside the ``WATCHER_KUBERNETES``
callback by @johnhoran in #2307

Enhancements

* Refactor: remove duplicate ``_construct_dest_file_path`` by @jx2lee in
#2077
* Leverage Airflow ``::group::`` to group logs associated with DAG
parsing by @tatiana in #2235
* Refactor ``DbtConsumerWatcherSensor`` for reusability by @tatiana in
#2245
* Restore plain text output when using ``ExecutionMode.WATCHER`` by
@tiovader in #2241

Bug Fixes

* Fix running empty models or ephemeral nodes in
``ExecutionMode.WATCHER`` by @tatiana in #2279
* Improve watcher producer task priority in scheduling and the UI by
@tatiana in #2237
* Fix typos and formatting issues in documentation by @pankajkoti in
#2259
* Allow watcher producer retries without erroring by @tatiana in #2283
* Fix ``TestBehavior.AFTER_ALL`` is missing project_name information
when loading project using manifest file by @tuantran0910 in #2242
* Fix duplicate log lines in watcher subprocess execution and format
timestamps by @pankajkoti in #2301

Docs

* Add Watcher Kubernetes documentation by @tatiana in #2303
* Document newly added telemetry metrics in the privacy notice by
@pankajkoti in #2249
* Add compatibility policy document by @pankajastro in #2251
* Improve watcher documentation related to dbt threads by @tatiana in
#2273
* Fix link in watcher execution mode documentation by @jedcunningham in
#2277
* Update Apache Airflow minimum compatibility policy by @tatiana in
#2285
* Clarify Cosmos runtime support until "End of Basic Support" by
@jedcunningham in #2286
* Update watcher docs by @tatiana in #2298
* Update watcher kubernetes documentation by @tatiana in #2306

Others

* Add Airflow 3 DAG versioning tests for Cosmos by @michal-mrazek in
#2177
* Add dbt Core 1.11 to the test matrix by @tatiana in #2230
* Add integration tests using InvocationMode.SUBPROCESS and validate
output by @tatiana in #2287
* Fix main branch failing tests by @tatiana in #2296
* Update pre-commit hooks to the latest versions by @jedcunningham in
#2289
* Pre-commit autoupdates by @pre-commit in #2222, #2264, #2274 and #2290
* Dependabot updates by @dependabot in #2218, #2219, #2220, #2280 and
#2284
* Add Scarf metrics to understand Cosmos feature usage patterns
- Add telemetry tracking for dbt docs plugin usage by @pankajkoti in
#2240
- Add DAG run telemetry metrics for load mode, invocation, and
render_config parameters by @pankajkoti in #2223
  - Collect profile metrics for DAG runs by @pankajastro in #2228
- Compress telemetry metadata to reduce serialized DAG size by
@pankajkoti in #2252
- Skip storing telemetry metadata when emission is disabled by
@pankajkoti in #2278
- Hide telemetry metadata parameters from the Airflow trigger UI by
@pankajkoti in #2247

closes:
astronomer/oss-integrations-private#317

---------

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants