Add CI job to test multiple dbt versions for the async DAG

[pankajkoti]

This PR introduces a new CI job named Run-Integration-Tests-DBT-Async to ensure compatibility of the async example DAG with multiple dbt versions. It achieves this by adding a third dimension to the pyproject.toml matrix, enabling the CI to run the DAG across a list of dbt versions.

Additionally, this PR includes a new test file: tests/test_async_example_dag.py. While we already have tests/test_example_dags.py, certain dbt versions have shown parsing issues with some example DAGs, which can cause CI failures unrelated to the async DAG. To prevent this, the new file is a modified copy that exclusively tests simple_async_dag, with other DAGs ignored via .airflowignore. This ensures that the CI job focuses on validating the async DAG without being affected by unrelated parsing errors.

closes: #1489

[netlify]

✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name	Link
🔨 Latest commit	9f1a85b
🔍 Latest deploy log	https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/67d2df51ec489800080a0a49

[cloudflare-workers-and-pages]

Deploying astronomer-cosmos with    Cloudflare Pages

Latest commit:	e0b60a1
Status:	✅  Deploy successful!
Preview URL:	https://3c1c7322.astronomer-cosmos.pages.dev
Branch Preview URL:	https://run-async-on-dbt-versions.astronomer-cosmos.pages.dev

View logs

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.36%. Comparing base (dd8b6c7) to head (0f00368).
Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1535   +/-   ##
=======================================
  Coverage   97.36%   97.36%           
=======================================
  Files          80       80           
  Lines        4934     4934           
=======================================
  Hits         4804     4804           
  Misses        130      130           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tatiana]

Hi @pankajkoti , I believe this is great, thank you very much!

I left some comments inline and, please, could you log a ticket to cover this part of the original ticket:
3. Validate that the monkeypatched behavior is consistent and functional across all versions.

So we can actually confirm that:

• The setup task does not actually perform the queries in BigQuery
  Making sure the mock works as expected for the desired dbt versions?

Also, if you could log a ticket for us to add a compatibility document, explaining which versions of dbt/airflow/python are tested with Cosmos, it could be great.

[pankajkoti]

Hi @pankajkoti , I believe this is great, thank you very much!

I left some comments inline and, please, could you log a ticket to cover this part of the original ticket: 3. Validate that the monkeypatched behavior is consistent and functional across all versions.

So we can actually confirm that:

• The setup task does not actually perform the queries in BigQuery
  Making sure the mock works as expected for the desired dbt versions?

Also, if you could log a ticket for us to add a compatibility document, explaining which versions of dbt/airflow/python are tested with Cosmos, it could be great.

Thanks. Created a couple of tickets:

1. Add a CI test to validate that the mock behaves correctly during async execution #1605
2. Extend compatibility information in Cosmos documentation #1606