Skip to content

Add task owner to dbt operator#1082

Merged
tatiana merged 14 commits into
astronomer:mainfrom
wornjs:issue/1605
Jul 23, 2024
Merged

Add task owner to dbt operator#1082
tatiana merged 14 commits into
astronomer:mainfrom
wornjs:issue/1605

Conversation

@wornjs
Copy link
Copy Markdown
Contributor

@wornjs wornjs commented Jul 7, 2024

Set dbt meta owner as task owner in all Cosmos DbtxxxOperators.

Closes: #1065

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jul 7, 2024
@netlify
Copy link
Copy Markdown

netlify Bot commented Jul 7, 2024

Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name Link
🔨 Latest commit bb4e334
🔍 Latest deploy log https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/669ee62e4f7922000821cd6a

@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jul 7, 2024
@wornjs
Copy link
Copy Markdown
Contributor Author

wornjs commented Jul 7, 2024

here is the result in airflow web ui
스크린샷 2024-07-07 오후 11 18 59

스크린샷 2024-07-07 오후 11 19 24

@wornjs wornjs marked this pull request as draft July 8, 2024 03:22
Copy link
Copy Markdown
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great, @wornjs !

I left a minor comment (sorry for doing this while the PR is in draft) - we can finish reviewing once the checks are passing and the PR is ready for review.

Comment thread cosmos/dbt/graph.py Outdated
@wornjs wornjs marked this pull request as ready for review July 14, 2024 11:06
@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.48%. Comparing base (e61f3a3) to head (f008ab9).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1082   +/-   ##
=======================================
  Coverage   96.47%   96.48%           
=======================================
  Files          64       64           
  Lines        3288     3297    +9     
=======================================
+ Hits         3172     3181    +9     
  Misses        116      116           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wornjs
Copy link
Copy Markdown
Contributor Author

wornjs commented Jul 17, 2024

@tatiana
plz check this

@tatiana tatiana changed the title add task owner to dbt operator Add task owner to dbt operator Jul 17, 2024
Comment thread cosmos/airflow/graph.py Outdated
@tatiana tatiana added this to the Cosmos 1.6.0 milestone Jul 17, 2024
Comment thread tests/airflow/test_graph.py
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jul 23, 2024
@tatiana tatiana merged commit d383792 into astronomer:main Jul 23, 2024
@wornjs wornjs deleted the issue/1605 branch July 30, 2024 02:35
@pankajkoti pankajkoti mentioned this pull request Jul 31, 2024
dwreeves pushed a commit to dwreeves/astronomer-cosmos that referenced this pull request Jul 31, 2024
Set dbt meta owner as task owner in all Cosmos `DbtxxxOperator`s.

Closes: astronomer#1065

---------
Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
pankajkoti added a commit that referenced this pull request Aug 20, 2024
New Features

* Add support for loading manifest from cloud stores using Airflow
Object Storage by @pankajkoti in #1109
* Cache ``package-lock.yml`` file by @pankajastro in #1086
* Support persisting the ``LoadMode.VIRTUALENV`` directory by @tatiana
in #1079
* Add support to store and fetch ``dbt ls`` cache in remote stores by
@pankajkoti in #1147
* Add default source nodes rendering by @arojasb3 in #1107
* Add Teradata ``ProfileMapping`` by @sc250072 in #1077

Enhancements

* Add ``DatabricksOauthProfileMapping`` profile by @CorsettiS in #1091
* Use ``dbt ls`` as the default parser when ``profile_config`` is
provided by @pankajastro in #1101
* Add task owner to dbt operators by @wornjs in #1082
* Extend Cosmos custom selector to support + when using paths and tags
by @mvictoria in #1150
* Simplify logging by @dwreeves in #1108

Bug fixes

* Fix Teradata ``ProfileMapping`` target invalid issue by @sc250072 in
#1088
* Fix empty tag in case of custom parser by @pankajastro in #1100
* Fix ``dbt deps`` of ``LoadMode.DBT_LS`` should use
``ProjectConfig.dbt_vars`` by @tatiana in #1114
* Fix import handling by lazy loading hooks introduced in PR #1109 by
@dwreeves in #1132
* Fix Airflow 2.10 regression and add Airflow 2.10 in test matrix by
@pankajastro in #1162

Docs

* Fix typo in azure-container-instance docs by @pankajastro in #1106
* Use Airflow trademark as it has been registered by @pankajastro in
#1105

Others

* Run some example DAGs in Kubernetes execution mode in CI by
@pankajastro in #1127
* Install requirements.txt by default during dev env spin up by
@@CorsettiS in #1099
* Remove ``DbtGraph.current_version`` dead code by @tatiana in #1111
* Disable test for Airflow-2.5 and Python-3.11 combination in CI by
@pankajastro in #1124
* Pre-commit hook updates in #1074, #1113, #1125, #1144, #1154,  #1167

---------

Co-authored-by: Pankaj Koti <pankajkoti699@gmail.com>
Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com>
tatiana pushed a commit that referenced this pull request Sep 11, 2024
`dag.owner` is a computed property that joins owners of existing tasks.
We should rely on airflow's existing owner fallback in
[airflow.models.baseoperator.BaseOperator](https://github.com/apache/airflow/blob/9a5f27f6dc560dcf456cabb793a32a61eab98dea/airflow/models/baseoperator.py#L865),
as it was the case before #1082.

Fixes #1194
ags-de pushed a commit to ags-de/astronomer-cosmos that referenced this pull request Sep 24, 2024
`dag.owner` is a computed property that joins owners of existing tasks.
We should rely on airflow's existing owner fallback in
[airflow.models.baseoperator.BaseOperator](https://github.com/apache/airflow/blob/9a5f27f6dc560dcf456cabb793a32a61eab98dea/airflow/models/baseoperator.py#L865),
as it was the case before astronomer#1082.

Fixes astronomer#1194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] add task owner to dbt operator

2 participants