Collect cosmos profile metrics#2198
Conversation
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2198 +/- ##
==========================================
+ Coverage 97.89% 97.98% +0.08%
==========================================
Files 95 95
Lines 6171 6190 +19
==========================================
+ Hits 6041 6065 +24
+ Misses 130 125 -5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
2636bd2 to
c020061
Compare
✅ Deploy Preview for astronomer-cosmos canceled.
|
There was a problem hiding this comment.
Pull request overview
This PR adds telemetry collection for Cosmos profile configuration metrics to track how users configure dbt profiles (via mapping or yaml file), which profile mapping classes are used, and which database types are deployed.
Key changes:
- Adds a new
get_profile_metricsfunction to extract profile strategy, mapping class, and database type from task instances - Integrates profile metrics into the task telemetry payload
- Adds early return optimization in
_get_profile_pathwhen profiles_yml_filepath is provided
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| cosmos/listeners/task_instance_listener.py | Implements get_profile_metrics function and integrates profile metrics (strategy, mapping class, database) into task telemetry |
| cosmos/config.py | Optimizes _get_profile_path by returning early when profiles_yml_filepath is set, avoiding unnecessary profile mapping operations |
| tests/listeners/test_task_instance_listener.py | Adds comprehensive tests for profile metrics collection covering both mapping and yaml file strategies |
Comments suppressed due to low confidence (1)
tests/listeners/test_task_instance_listener.py:172
- The test should verify that the new profile metrics fields (profile_strategy, profile_mapping_class, database) are set to None when profile_config is not provided, to ensure consistent behavior when operators don't have profile configuration.
def test_build_task_metrics_records_core_fields():
operator = DummyDbtOperator()
ti = _make_task_instance(operator)
metrics = task_instance_listener._build_task_metrics(ti, status="success")
assert metrics["operator_name"] == "DummyDbtOperator"
assert metrics["dbt_command"] == "run"
assert metrics["install_deps"] is True
assert metrics["invocation_mode"] == InvocationMode.DBT_RUNNER.value
assert metrics["execution_mode"] == "local"
assert metrics["is_cosmos_operator_subclass"] is False
assert metrics["dag_run_id"] == "run-1"
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
tatiana
left a comment
There was a problem hiding this comment.
Looks good, @pankajastro !
Since we'll be extending this to also emit profile when parsing the DAG, we can cover the missing line by the time we do this follow up PR:
Do we have a ticket to do this follow-up work?
Breaking changes * Introduced in the PR #2080. The following functions are expected to be used internally only to Cosmos, so we hope these won't impact end-users, but we are documenting the changes just in case: - ``generate_task_or_group`` receives ``render_config`` instead of its individual configurations, such as ``test_behavior``, ``source_rendering_behavior`` and ``enable_owner_inheritance`` - ``create_task_metadata`` receives ``render_config`` instead of its individual configurations, such as ``test_behavior``, ``source_rendering_behavior`` and ``enable_owner_inheritance`` - ``create_task_metadata`` now expects the ``node_converters`` argument * Drop Python 3.9 support by @pankajastro in #2118 * Drop Airflow 2.4 support by @pankajastro in #2161 * Drop Airflow 2.5 support by @pankajastro in #2165 Features * Support applying ``node_converter`` at a task level instead of task group level by @anyapriya in #1759 * Allow overriding ``DbtProducerWatcherOperator`` parameters via ``ExecutionConfig.setup_operator_args`` by @pankajastro in #2133 * Use deferrable sensors by default in ``ExecutionMode.WATCHER`` by @pankajastro in #2084 * Support real-time consumer updates when using ``ExecutionMode.WATCHER`` and ``InvocationMode.SUBPROCESS`` by @pankajastro in #2152 * Update telemetry to v3 format with query parameters by @pankajkoti in #2192 * Add initial set of telemetry task listener metrics for Cosmos operators by @pankajkoti in #2195 Enhancements * Unify Airflow version handling into ``constants.py`` by @tatiana in #2089 * Refactor ``airflow/graph.py`` to simplify the code base by @tatiana in #2080 * Force watcher producer retries to zero by @pankajkoti in #2114 * Fail ``ExecutionMode.WATCHER`` consumer sensors immediately when the producer fails using Airflow context by @pankajkoti in #2126 * ``ExecutonMode.WATCHER``: fetch producer status asynchronously from the Airflow runtime so deferrable sensors fail immediately when the producer task fails by @pankajkoti in #2144 * Refactor ``ExecutionMode.WATCHER`` ``InvocationMode.SUBPROCESS`` log parser by @tatiana in #2183 * Replace map_index with is_mapped_task boolean in task telemetry metrics by @pankajkoti in #2210 * Collect cosmos profile metrics in task telemetry metrics by @pankajastro in #2198 * Remove unnecessary information from telemetry by @tatiana in #2211 Bug fixes * Clarify ``ExecutionMode.WATCHER`` deferrable failure messaging by @pankajkoti in #2124 * Remove empty test tasks when all tests are detached by @anyapriya in #2010 * Fix forwarding ``DbtProducerWatcherOperator`` ``dbt build`` flags by @michal-mrazek in #2127 * Add databricks oauth mock profile by @fjmacagno in #2164 * Register listeners in Airflow 3 plugin implementation by @pankajastro in #2187 * Fix resolution of ``packages-install-path`` when it uses ``env_var`` by @tatiana in #2194 * Fix ``template_fields`` in ``DbtConsumerWatcherSensor`` to include ``DbtRunLocalOperator`` template_fields`` by @tiovader and @emanuel-luis in #2209 * Emit asset events in ExecutionMode.AIRFLOW_ASYNC mode by @pankajastro in #2184 * Remove dag_run_id from telemetry tests by @tatiana in #2213 Docs * Document dataset-event limitation when using ``ExecutionMode.AIRFLOW_ASYNC`` by @varaprasadregani in #2143 * Expand ``ExecutionMode.KUBERNETES`` guidance by @tatiana in #2139 * Add docs for deferrable ``DbtConsumerWatcherSensor`` by @pankajastro in #2115 * Fix reStructuredText formatting by @dnskr in #2132 * Add docs for ``setup_operator_args`` param by @pankajastro in #2136 * Remove experimental flag for ``ExecutionMode.AIRFLOW_ASYNC`` by @pankajastro in #2153 * Clarify ``ExecutionMode.AIRFLOW_ASYNC`` dataset limits by @pankajkoti in #2167 * Update PRIVACY_NOTICE.rst by @tatiana in #2212 Others * Drop Python 3.9 support by @pankajastro in #2118 * Drop Airflow 2.4 support by @pankajastro in #2161 * Drop Airflow 2.5 support by @pankajastro in #2165 * Improve example DAG ``jaffle_shop_kubernetes.py`` by @tatiana in #2140 * Enable tests for Python 3.13 by @pankajastro in #2154 * Add Python 3.12 to CI integration tests matrix by @pankajastro in #2168 * Retry flaky Telemetry success test to stabilise CI by @pankajkoti in #2138 * Drop unused producer state xcom handling in ``ExecutionMode.WATCHER`` by @pankajkoti in #2145 * Remove unused Python3.9 uses from Github action CI by @pankajastro in #2117 * Run pre-commit on ``ExecutionMode.WATCHER`` modules by @pankajkoti in #2150 * Refactor: Use shared airflow version constant by @pankajkoti in #2157 * Pin ``pydantic<2.0`` for Airflow 2.6 compatibility by @pankajastro in #2172 * Remove duplicate ``dbt-duckdb`` dependency by @pankajastro in #2170 * Add targeted ``type: ignore`` for untyped decorators to fix ``mypy`` errors by @pankajastro in #2174 * Replace Legacy typing Aliases with Built-in Types for Python 3.10+ by @pankajastro in #2175 * Refactor to reuse ``load_method_from_module`` from ``_utils/importer.py`` by @pankajastro in #2176 * Remove try except block for cache import and unused python_version variable by @pankajastro in #2186 * Unpin Airflow to satisfy GitHub Security tab requirements by @pankajastro in #2171 * Update Python version for ``pyupgrade`` in ``pre-commit`` config by @pankajastro in #2190 * Add cooldown config in ``dependabot`` config by @pankajastro in #2189 * Adjust pre-commit so Python 3.10 or higher can be used by @tatiana in #2196 * Remove empty variables emission from telemetry metrics by @pankajkoti in #2197 * Reformat documented comments for historical URL formats by @pankajkoti in #2199 * Bump ``actions/checkout`` from ``5.0.0`` to ``5.0.1`` by @dependabot in #2135 * Bump ``actions/checkout`` to ``6.0.0`` in GitHub workflows by @dependabot in #2147 * Bump ``zizmorcore/zizmor-action`` from ``0.2.0`` to ``0.3.0`` by @dependabot in #2156 * Bump ``actions/checkout`` from ``5.0.1`` to ``6.0.0`` by @dependabot in #2155 * Bump ``actions/checkout`` from ``6.0.0`` to ``6.0.1`` by @dependabot in #2178 * Bump ``codecov/codecov-action`` from ``5.5.1`` to ``5.5.2`` by @dependabot in #2208 * pre-commit autoupdate by @pre-commit-ci[bot] in #2134, #2162, #2173, #2191, #2202 closes: astronomer/oss-integrations-private#275
|
Address: #2198 (review) This pull request adds a test to verify that profile metrics are correctly handled when dealing with non-Cosmos operators. The test ensures that when `_build_task_metrics` is called on a task instance with a regular Airflow operator (not a dbt/Cosmos operator), the profile-related metrics return appropriate `None` values. **Key changes:** - Added a `DummyOperator` class that extends `BaseOperator` without Cosmos-specific functionality - Added `test_profile_metrics_with_non_cosmos_operator()` to validate metrics behavior with non-Cosmos operators
Address: #2198 (review) This pull request adds a test to verify that profile metrics are correctly handled when dealing with non-Cosmos operators. The test ensures that when `_build_task_metrics` is called on a task instance with a regular Airflow operator (not a dbt/Cosmos operator), the profile-related metrics return appropriate `None` values. **Key changes:** - Added a `DummyOperator` class that extends `BaseOperator` without Cosmos-specific functionality - Added `test_profile_metrics_with_non_cosmos_operator()` to validate metrics behavior with non-Cosmos operators

closes: #2108
This PR adds telemetry collection for Cosmos profile configuration metrics to track how users configure dbt profiles (via mapping or yaml file), which profile mapping classes are used, and which database types are deployed.
Key changes:
get_profile_metricsfunction to extract profile strategy, mapping class, and database type from task instances_get_profile_pathwhen profiles_yml_filepath is provided