Skip to content

feat: handle tests for WATCHER_KUBERNETES mode#2529

Merged
tatiana merged 5 commits into
astronomer:mainfrom
vricciardulli:feat/handle-tests-for-k8s-watcher-mode
Apr 7, 2026
Merged

feat: handle tests for WATCHER_KUBERNETES mode#2529
tatiana merged 5 commits into
astronomer:mainfrom
vricciardulli:feat/handle-tests-for-k8s-watcher-mode

Conversation

@vricciardulli
Copy link
Copy Markdown
Contributor

Description

Following #2441 and #2447, this PR implements test handling for WATCHER_KUBERNETES mode.

Related Issue(s)

closes #2427

Breaking Change?

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@vricciardulli vricciardulli changed the title Feat/handle tests for k8s watcher mode feat: handle tests for WATCHER_KUBERNETES mode Apr 6, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements dbt test handling for ExecutionMode.WATCHER_KUBERNETES by turning the previously-placeholder test operator into a watcher sensor and updating docs/tests to reflect the supported TestBehavior options in this mode.

Changes:

  • Implement DbtTestWatcherKubernetesOperator as a DbtConsumerWatcherKubernetesSensor that watches aggregated test results via XCom.
  • Add/adjust unit and graph-construction tests to cover watcher-kubernetes test behavior (NONE, AFTER_EACH, AFTER_ALL).
  • Update watcher and watcher-kubernetes documentation to describe the supported test behaviors and operators.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cosmos/operators/watcher_kubernetes.py Replaces placeholder test operator with a real watcher sensor implementation.
tests/operators/test_watcher_kubernetes_unit.py Adds unit tests for the new test watcher sensor; simplifies producer callback test.
tests/airflow/test_graph.py Verifies task generation for watcher-kubernetes across TestBehavior modes.
docs/guides/run_dbt/container/watcher-kubernetes-execution-mode.rst Updates limitations/behavior docs to reflect test support in watcher-kubernetes.
docs/guides/run_dbt/airflow-worker/watcher-execution-mode.rst Updates watcher-mode test behavior documentation to match current behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cosmos/operators/watcher_kubernetes.py Outdated
Comment thread tests/operators/test_watcher_kubernetes_unit.py Outdated
Copy link
Copy Markdown
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @vricciardulli , this looks great!

@tatiana tatiana merged commit 9611fa5 into astronomer:main Apr 7, 2026
1 of 2 checks passed
@michal-mrazek
Copy link
Copy Markdown
Contributor

Awesome, thanks! :)

@pankajkoti
Copy link
Copy Markdown
Contributor

Not sure if it's due to this PR, but it's evident on the merge of this PR, that the kubernetes tests are getting stuck https://github.com/astronomer/astronomer-cosmos/actions/runs/24081355329/job/70242570483

pankajkoti added a commit that referenced this pull request Apr 7, 2026
pankajkoti added a commit that referenced this pull request Apr 7, 2026
)

This reverts commit 9611fa5.

Kubernetes Tests are getting stuck after having merged PR #2529 as
observed in the CI.
@tatiana
Copy link
Copy Markdown
Collaborator

tatiana commented Apr 7, 2026

@vricciardulli @michal-mrazek we unfortunately had to revert (#2535) this PR since it caused the K8S integration tests to get stuck for some reason, and we do not have time to fix it right now. I made the mistake of merging it without realising that those tests were stuck.

@pankajkoti created a PR to revert. Please, if you could open a new PR with the change and fix the integration tests - we can make sure this will be shipped as part of Cosmos 1.15.0.

@vricciardulli
Copy link
Copy Markdown
Contributor Author

@tatiana sorry for the inconvenience!

@tatiana
Copy link
Copy Markdown
Collaborator

tatiana commented Apr 7, 2026

@vricciardulli don't worry, it was my fault - I should have realised the tests weren't running as expected, and that would have avoided this situation.

It would be amazing if you could create a new PR with the feature, and we could support on checking what may be hanging. Did you test the feature locally, using Airflow standalone or some other way?

vricciardulli added a commit to vricciardulli/astronomer-cosmos that referenced this pull request Apr 8, 2026
@vricciardulli
Copy link
Copy Markdown
Contributor Author

@tatiana I've opened the PR again #2543

I only ran unit tests locally, I wasn't yet able to set up integration tests locally so I thought I'd wait for the CI to run them instead.

@tatiana tatiana added this to the Cosmos 1.15.0 milestone May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] dbt test node states for WATCHER kubernetes mode

5 participants