Skip to content

feat: Add support for StarRocks profile mapping#2256

Merged
tatiana merged 3 commits into
astronomer:mainfrom
kurkim0661:feature/add-starrocks-profile
Jan 9, 2026
Merged

feat: Add support for StarRocks profile mapping#2256
tatiana merged 3 commits into
astronomer:mainfrom
kurkim0661:feature/add-starrocks-profile

Conversation

@kurkim0661
Copy link
Copy Markdown
Contributor

@kurkim0661 kurkim0661 commented Jan 8, 2026

Description

Introduces the StarrocksUserPasswordProfileMapping class, enabling Cosmos to automatically map Airflow connections to dbt profiles for StarRocks.

Test environment

  • dbt-starrocks==1.11.0
  • apache-airflow-providers-mysql==5.7.4
  • airflow==2.10.4
  • python==3.12

Concerns

  1. When using get_automatic_profile_mapping, it compares airflow_connection_type with conn_type. For StarRocks, we use the mysql type, so it overlaps with regular MySQL connections. Since MySQL is searched first, existing MySQL users are fine—but StarRocks won’t be found via the automatic search as things stand.
  2. The same issue happens with get_env_var_name: it also relies on airflow_connection_type, so it ends up colliding with MySQL as well.

Both issues shouldn’t affect existing users, but they could cause problems for users who want to use StarRocks.

Related Issue(s)

Closes: #2255

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

This commit introduces the StarrocksUserPasswordProfileMapping class, enabling Cosmos to automatically map Airflow connections to dbt profiles for StarRocks.

Key changes:

Added StarrocksUserPasswordProfileMapping to support starrocks connection type.
Registered the new profile mapping in
cosmos/profiles/
init
.py
.
Added dbt-starrocks to dbt-all and dbt-starrocks optional dependencies in
pyproject.toml
.
Added comprehensive unit tests in
tests/profiles/starrocks/test_starrocks_user_pass.py
.
Copilot AI review requested due to automatic review settings January 8, 2026 13:36
@netlify
Copy link
Copy Markdown

netlify Bot commented Jan 8, 2026

Deploy Preview for astronomer-cosmos canceled.

Name Link
🔨 Latest commit 38bad82
🔍 Latest deploy log https://app.netlify.com/projects/astronomer-cosmos/deploys/695fba5f9d19550008359c85

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for mapping Airflow MySQL connections to dbt StarRocks profiles. StarRocks uses the MySQL protocol, so the integration leverages the existing MySQL connection type while mapping to dbt's StarRocks adapter.

Key changes:

  • Implements StarrocksUserPasswordProfileMapping class for StarRocks profile generation
  • Adds dbt-starrocks as an optional dependency in the project configuration
  • Includes comprehensive test coverage for the new profile mapping functionality

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cosmos/profiles/starrocks/user_pass.py Core implementation of StarRocks profile mapping using MySQL connection type
cosmos/profiles/starrocks/init.py Package initialization exposing the StarRocks profile mapping class
cosmos/profiles/init.py Registers StarRocks profile mapping in the automatic profile discovery system
pyproject.toml Adds dbt-starrocks to optional dependencies
tests/profiles/starrocks/test_starrocks_user_pass.py Test suite validating connection claiming, profile generation, and environment variables

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cosmos/profiles/starrocks/__init__.py Outdated
Comment thread cosmos/profiles/starrocks/user_pass.py Outdated
Good.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Good suggestion

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.04%. Comparing base (c251c36) to head (38bad82).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2256   +/-   ##
=======================================
  Coverage   98.03%   98.04%           
=======================================
  Files          98      100    +2     
  Lines        6377     6401   +24     
=======================================
+ Hits         6252     6276   +24     
  Misses        125      125           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kurkim0661, this looks great, thank you very much for the contribution! We'll be releasing it as part of Cosmos 1.13. The implementation, tests, and PR description are excellent, including the limitations bit. Excellent work!

@tatiana tatiana added this to the Cosmos 1.13.0 milestone Jan 9, 2026
@tatiana tatiana merged commit b34fb32 into astronomer:main Jan 9, 2026
85 checks passed
@pankajastro pankajastro mentioned this pull request Jan 29, 2026
tatiana added a commit that referenced this pull request Jan 30, 2026
Features

* Support cross-referencing models across dbt projects using dbt-loom by
@pankajkoti in #2271
* Support use of YAML selectors when using ``LoadMode.DBT_MANIFEST`` by
@YourRoyalLinus in #2261
* Introduce ``ExecutionMode.WATCHER_KUBERNETES`` to use the watcher with
``KubernetesPodOperator`` by @tatiana in #2207
* Add support for StarRocks profile mapping by @kurkim0661 in #2256
* Allow pushing URIs as XComs for Cosmos tasks by @corsettigyg in #2275
* Support defining custom callbacks alongside the ``WATCHER_KUBERNETES``
callback by @johnhoran in #2307

Enhancements

* Refactor: remove duplicate ``_construct_dest_file_path`` by @jx2lee in
#2077
* Leverage Airflow ``::group::`` to group logs associated with DAG
parsing by @tatiana in #2235
* Refactor ``DbtConsumerWatcherSensor`` for reusability by @tatiana in
#2245
* Restore plain text output when using ``ExecutionMode.WATCHER`` by
@tiovader in #2241

Bug Fixes

* Fix running empty models or ephemeral nodes in
``ExecutionMode.WATCHER`` by @tatiana in #2279
* Improve watcher producer task priority in scheduling and the UI by
@tatiana in #2237
* Fix typos and formatting issues in documentation by @pankajkoti in
#2259
* Allow watcher producer retries without erroring by @tatiana in #2283
* Fix ``TestBehavior.AFTER_ALL`` is missing project_name information
when loading project using manifest file by @tuantran0910 in #2242
* Fix duplicate log lines in watcher subprocess execution and format
timestamps by @pankajkoti in #2301

Docs

* Add Watcher Kubernetes documentation by @tatiana in #2303
* Document newly added telemetry metrics in the privacy notice by
@pankajkoti in #2249
* Add compatibility policy document by @pankajastro in #2251
* Improve watcher documentation related to dbt threads by @tatiana in
#2273
* Fix link in watcher execution mode documentation by @jedcunningham in
#2277
* Update Apache Airflow minimum compatibility policy by @tatiana in
#2285
* Clarify Cosmos runtime support until "End of Basic Support" by
@jedcunningham in #2286
* Update watcher docs by @tatiana in #2298
* Update watcher kubernetes documentation by @tatiana in #2306

Others

* Add Airflow 3 DAG versioning tests for Cosmos by @michal-mrazek in
#2177
* Add dbt Core 1.11 to the test matrix by @tatiana in #2230
* Add integration tests using InvocationMode.SUBPROCESS and validate
output by @tatiana in #2287
* Fix main branch failing tests by @tatiana in #2296
* Update pre-commit hooks to the latest versions by @jedcunningham in
#2289
* Pre-commit autoupdates by @pre-commit in #2222, #2264, #2274 and #2290
* Dependabot updates by @dependabot in #2218, #2219, #2220, #2280 and
#2284
* Add Scarf metrics to understand Cosmos feature usage patterns
- Add telemetry tracking for dbt docs plugin usage by @pankajkoti in
#2240
- Add DAG run telemetry metrics for load mode, invocation, and
render_config parameters by @pankajkoti in #2223
  - Collect profile metrics for DAG runs by @pankajastro in #2228
- Compress telemetry metadata to reduce serialized DAG size by
@pankajkoti in #2252
- Skip storing telemetry metadata when emission is disabled by
@pankajkoti in #2278
- Hide telemetry metadata parameters from the Airflow trigger UI by
@pankajkoti in #2247

closes:
astronomer/oss-integrations-private#317

---------

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support starrocks profile.

3 participants