Add MySQL profile mapping by Lee2532 · Pull Request #1977 · astronomer/astronomer-cosmos

Lee2532 · 2025-09-16T07:22:15Z

Description

Mysql Connection mapping with airflow cosmos

dbt-mysql==1.7.0
apache-ariflow-providers-mysql==6.0.0
airflow-version 2.10.5

Related Issue(s)

Closes: #1976

Checklist

I have made corresponding changes to the documentation (if required)
I have added tests that prove my fix is effective or that my feature works

netlify · 2025-09-16T07:22:33Z

✅ Deploy Preview for sunny-pastelito-5ecb04 ready!

Name	Link
🔨 Latest commit	`8580cc8`
🔍 Latest deploy log	https://app.netlify.com/projects/sunny-pastelito-5ecb04/deploys/68eccc4ee12d790008a7a0c5
😎 Deploy Preview	https://deploy-preview-1977--sunny-pastelito-5ecb04.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2025-09-16T10:21:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (4845f24) to head (8580cc8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1977      +/-   ##
==========================================
+ Coverage   97.83%   97.84%   +0.01%     
==========================================
  Files          87       89       +2     
  Lines        5591     5617      +26     
==========================================
+ Hits         5470     5496      +26     
  Misses        121      121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tatiana

Hi @Lee2532 , thank you very much for your contribution!

I gave some minor feedback inline.

Please, could you give more information in the PR description about how you tested this profile mapping? Examples of useful information:

version of the MySQL dbt adaptor
version of the MySQL Airflow provider
Airflow version

This feature seems close to completion - I'm happy to merge after the feedback is addressed and the checks are passing. It will be part of Cosmos 1.11.

Lee2532 · 2025-09-18T11:31:43Z

Hi @Lee2532 , thank you very much for your contribution!

I gave some minor feedback inline.

Please, could you give more information in the PR description about how you tested this profile mapping? Examples of useful information:

version of the MySQL dbt adaptor

version of the MySQL Airflow provider

Airflow version

This feature seems close to completion - I'm happy to merge after the feedback is addressed and the checks are passing. It will be part of Cosmos 1.11.

it's my testing environment

dbt-mysql==1.7.0
apache-ariflow-providers-mysql==6.0.0
airflow-version 2.10.5

pankajastro

Looks great! 👍
The Codecov job is failing. @Lee2532, could you take a look?
It's complaining that the mock_profile method is not covered by tests.

Lee2532 · 2025-09-30T07:03:32Z

Looks great! 👍 The Codecov job is failing. @Lee2532, could you take a look? It's complaining that the mock_profile method is not covered by tests.

Of course. I've added a related test code

Lee2532 · 2025-09-30T08:33:20Z

Hi, @pankajastro
In my opinion, this test failure appears to be unrelated to my code changes

pankajastro · 2025-09-30T08:39:24Z

Hi, @pankajastro In my opinion, this test failure appears to be unrelated to my code changes

yes, it is unrelated to your changes

@tatiana

**Features** * Introduce ``ExecutionMode.WATCHER`` to reduce DAG run time by 1/5 in several PRs. Learn more about it [here](https://astronomer.github.io/astronomer-cosmos/getting_started/watcher-execution-mode.html#watcher-execution-mode). This feature was implemented via multiple PRs, including: * Expose new execution mode by @tatiana @pankajastro @pankajkoti in #1999 * Add ``DbtProducerWatcherOperator`` for the proposed ``ExecutionMode.WATCHER`` by @pankajkoti in #1982 * Add ``DbtConsumerWatcherSensor`` for the proposed ``ExecutionMode.WATCHER`` by @pankajastro in #1998 * Push producer's task completion status to XCOM by @pankajkoti in #2000 * Add default priority_weight for ``DbtProducerWatcherOperator`` by @pankajkoti in #1995 * Add sample dbt events for the dbt watcher execution mode by @pankajkoti in #1952 * Add ``compiled_sql`` as a template fields on ```ExecutionMode.WATCHER``` when using ``run_results.json`` by @pankajastro in #2070 * Set ``push_run_results_to_xcom`` kwargs correctly for invocation mode subprocess and Watcher mode by @pankajastro in #2067 * Store compiled SQL as template field for dbt callback events in ``ExecutionMode.WATCHER`` by @pankajkoti in #2068 * Add initial documentation for ``ExecutionMode.WATCHER`` by @tatiana in #2046 * Support running ``State.UPSTREAM_FAILED`` tasks when WATCHER consumer upstream tasks fail by @tatiana in #2062 * Fail sensor tasks immediately if the ``ExecutionMode.WATCHER`` producer task fails by @pankajastro in #2040 * Add ``WATCHER``` to GitHub issue template by @tatiana in #2056 * Add support for ``TestBehavior.AFTER_ALL`` with ``ExecutionMode.WATCHER`` by @pankajastro in #2049 * Add support for ``TestBehavior.NONE`` with ``ExecutionMode.WATCHER`` by @pankajastro in #2047 * Fix ``ExecutionMode.WATCHER`` behaviour with ``DbtTaskGroup`` by @tatiana in #2044 * Fix Cosmos behaviour when using watcher with ``InvocationMode.DBT_RUNNER`` by @tatiana in #2048 * Add Airflow 3 plugin for dbt docs with multiple dbt projects support by @pankajkoti in #2009, check the [documentation](https://astronomer.github.io/astronomer-cosmos/configuration/hosting-docs.html). * Initial support to ``dbt Fusion`` by @tatiana in #1803. More details [here](https://astronomer.github.io/astronomer-cosmos/configuration/dbt-fusion). * Support to prune sources without downstream references in dbt projects by @corsettigyg in #1988 * Allow to set task display name as a user-defined function by @corsettigyg in #1761 * Add dbt project's hash to dag docs to support dag versioning in Airflow 3 by @pankajkoti in #1907 * feat: Add Jinja templating support for ``dbt_cmd_flags`` by @skillicinski in #1899 * Add Scarf metric to collect the execution mode uses by @pankajastro in #1981 * Support Airflow 3.1 by @tatiana in #1980 * Add MySQL profile mapping by @Lee2532 in #1977 * Add sqlserver profile mapping by @pankajastro in #1737 **Enhancement** * Use XCom to store sql when using ``ExecutionMode.AIRFLOW_ASYNC`` by @pankajastro in #1934 * Refactor ``AIRFLOW_ASYNC`` teardown so it doesn't install the virtualenv by @pankajastro in #1938 * Reuse the virtual env for ``AIRFLOW_ASYNC`` setup task by @pankajastro in #1939 * Improve dataset/asset experience in Cosmos by @tatiana in #2030 * Add ``downstreams`` to ``DbtNode`` by @wornjs in #2028 **Bug fixes** * Fix tags extraction by @ms32035 in #1915 * Fix task flow operator args by @anyapriya in #2024 **Documentation** * Add documentation for Airflow 3 Plugin supporting dbt docs for multiple dbt projects by @pankajkoti in #2063 * Add Cosmos Deferrable Operator Guide by @pankajastro in #1922 * Add dbt Fusion documentation by @tatiana in #1824 #1830 * Update dbt-fusion.rst to explicitly highlight it is in alpha by @tatiana in #1838 * Fix a bunch of docs build errors and warnings by @pankajkoti in #1886 * Add docs note for param virtualenv_dir for async execution mode by @pankajastro in #1969 * Use pepy.tech downloads badge in README by @pankajkoti in #1920 * Correct the default value of ``cache_dir`` by @seokyun.ha in #2027 **Others** * Promote @corsettigyg to committer by @tatiana in #1985 * Add @pankajkoti and @pankajastro to ``contributors.rst`` by @tatiana in #1983 * Update setup script for airflow3 script by @dwreeves in #2023 * Prevent pytest from trying to test classes that aren't actually tests by @anyapriya in #2032 * Fix ``dag.test()`` for Airflow 3.1+ by syncing DAG to database bby @kaxil in #2037 * Disable Scarf in CI by @pankajastro in #2016 * Fix failing dbt Fusion tests when run in parallel in CI by @pankajkoti in #1896 * Fix MyPy issues related to ``ObjectStoragePath`` in main branch by @tatiana in #2012 * Cleanup example dbt event JSON dictionaries kept for XCOM referencby @pankajkoti in #1997 * Bump min hatch version that includes fixes for click>=8.3.0 by @pankajkoti in #1996 * Use official postgres image from Docker hub for kubernetes setup by @pankajkoti in #1986 * Use click<8.3.0 for hatch as click 8.3 breaks hatch by @pankajkoti in #1987 * Pin Airflow version in type check CI job by @pankajastro in #2003 * Improve comments after feedback on #1948 by @tatiana in #1963 * Fix running tests with dbt Fusion 2.0.0 preview versions by @tatiana in #1948 * Test hardening of dbt node having tags as unset or missing by @pankajkoti in #1918 * Fix Sphinx issue in the main branch by @tatiana in #2064 * pre-commit autoupdate in #2065, #2043, #2033, #2019, #1990, #2019, #2008, #1941, #1935, #1924 * GitHub dependabot update in #2051, #2050, #2038, #2022, #1947, #1955, #1946, #1944, #1945, #1928, #1921, #1917 Co-authored-by: Pankaj Koti <pankaj.koti@astronomer.io> Co-authored-by: Pankaj Singh <pankaj.singh@astronomer.io> Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>

Lee2532 temporarily deployed to external September 16, 2025 07:22 — with GitHub Actions Inactive

pankajastro reviewed Sep 18, 2025

View reviewed changes

Comment thread cosmos/profiles/mysql/user_pass.py

Lee2532 temporarily deployed to external September 18, 2025 09:14 — with GitHub Actions Inactive

pankajastro added this to the Cosmos 1.11.0 milestone Sep 18, 2025

tatiana reviewed Sep 18, 2025

View reviewed changes

Comment thread tests/profiles/mysql/test_mysql_user_pass.py Outdated

tatiana reviewed Sep 18, 2025

View reviewed changes

Comment thread tests/profiles/mysql/test_mysql_user_pass.py Outdated

tatiana reviewed Sep 18, 2025

View reviewed changes

Comment thread tests/profiles/mysql/test_mysql_user_pass.py Outdated

tatiana reviewed Sep 18, 2025

View reviewed changes

Lee2532 temporarily deployed to external September 18, 2025 11:59 — with GitHub Actions Inactive

tatiana changed the title ~~Mysql profile mapping~~ Add MySQL profile mapping Sep 18, 2025

Lee2532 added 6 commits September 29, 2025 22:53

add mysql lib

0669562

add test case

3bf485c

add mysql profile mapping

d3e348e

Update __init__.py

cb81ba0

add is_community attr

c6d6a0a

Modify for higher version compatibility

c80bf7f

pankajastro force-pushed the mysql-profile-mapping branch from 93fff71 to c80bf7f Compare September 29, 2025 17:23

pankajastro temporarily deployed to external September 29, 2025 17:23 — with GitHub Actions Inactive

pankajastro approved these changes Sep 29, 2025

View reviewed changes

Add mock_profile test code

571bd73

Lee2532 temporarily deployed to external September 30, 2025 07:02 — with GitHub Actions Inactive

Merge branch 'main' into mysql-profile-mapping

8580cc8

tatiana temporarily deployed to external October 13, 2025 09:54 — with GitHub Actions Inactive

tatiana merged commit e931354 into astronomer:main Oct 13, 2025
96 checks passed

Lee2532 deleted the mysql-profile-mapping branch October 22, 2025 14:09

tatiana mentioned this pull request Oct 29, 2025

Release 1.11.0 #2020

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MySQL profile mapping#1977

Add MySQL profile mapping#1977
tatiana merged 8 commits into
astronomer:mainfrom
Lee2532:mysql-profile-mapping

Lee2532 commented Sep 16, 2025 •

edited

Loading

Uh oh!

netlify Bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

codecov Bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tatiana left a comment

Uh oh!

Lee2532 commented Sep 18, 2025

Uh oh!

pankajastro left a comment

Uh oh!

Lee2532 commented Sep 30, 2025

Uh oh!

Lee2532 commented Sep 30, 2025

Uh oh!

pankajastro commented Sep 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Lee2532 commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Checklist

Uh oh!

netlify Bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for sunny-pastelito-5ecb04 ready!

Uh oh!

codecov Bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tatiana left a comment

Choose a reason for hiding this comment

Uh oh!

Lee2532 commented Sep 18, 2025

Uh oh!

pankajastro left a comment

Choose a reason for hiding this comment

Uh oh!

Lee2532 commented Sep 30, 2025

Uh oh!

Lee2532 commented Sep 30, 2025

Uh oh!

pankajastro commented Sep 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Lee2532 commented Sep 16, 2025 •

edited

Loading

netlify Bot commented Sep 16, 2025 •

edited

Loading

codecov Bot commented Sep 16, 2025 •

edited

Loading