Skip to content

[kube-prometheus-stack] Keep KubeletDown cluster-scoped - #6906

Merged
jkroepke merged 2 commits into
prometheus-community:mainfrom
Zakharden:fix/kps-kubeletdown-aggregation
May 16, 2026
Merged

[kube-prometheus-stack] Keep KubeletDown cluster-scoped#6906
jkroepke merged 2 commits into
prometheus-community:mainfrom
Zakharden:fix/kps-kubeletdown-aggregation

Conversation

@Zakharden

@Zakharden Zakharden commented May 11, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

KubeletDown joins kube_node_info and kubelet up{} series to detect missing kubelet targets. When defaultRules.additionalAggregationLabels includes labels that do not exist consistently on both metric sources, such as namespace, the alert can fire even when kubelets are healthy.

This keeps KubeletDown scoped to cluster only, while leaving the existing additional aggregation label behavior unchanged for other generated rules.

Which issue this PR fixes

Fixes #6872.

Special notes for your reviewer

The generated rules file is updated through hack/sync_prometheus_rules.py; the follow-up commit only syncs the generated file formatting expected by CI.

Checklist

  • DCO signed
  • Chart Version bumped
  • Title of the PR starts with chart name

Validation

  • helm unittest -f unittests/prometheus/kubelet_down_aggregation_test.yaml charts/kube-prometheus-stack
  • ./venv/bin/python ./sync_prometheus_rules.py from charts/kube-prometheus-stack/hack
  • git diff --check

@Zakharden

Copy link
Copy Markdown
Contributor Author

Hi maintainers, could someone take a look at this PR?

It keeps KubeletDown cluster-scoped so defaultRules.additionalAggregationLabels do not introduce false positives when labels like namespace are not present consistently across kube_node_info and kubelet up{}.

I synced the generated rules and added focused unittest coverage; happy to revise if there is a preferred rule-generation pattern. Thanks.

@jkroepke

@Zakharden
Zakharden force-pushed the fix/kps-kubeletdown-aggregation branch from a3d691a to 6e1cd72 Compare May 13, 2026 06:54
@Zakharden

Copy link
Copy Markdown
Contributor Author

@jkroepke @gkarthiks @Xtigyro @gianrubio Rebased on the latest main; the PR is mergeable now and CI is green. I also updated the unit test to assert the KubeletDown rule by content rather than by a brittle array index. Could you please take another look?

@Zakharden
Zakharden force-pushed the fix/kps-kubeletdown-aggregation branch from 6e1cd72 to 41ae333 Compare May 13, 2026 11:32
Zakharden added 2 commits May 16, 2026 13:49
Signed-off-by: Zakhar Dvurechensky <72825626+Zakharden@users.noreply.github.com>
Signed-off-by: Zakhar Dvurechensky <72825626+Zakharden@users.noreply.github.com>
@Zakharden
Zakharden force-pushed the fix/kps-kubeletdown-aggregation branch from 41ae333 to def019a Compare May 16, 2026 10:51
@jkroepke
jkroepke merged commit 5735841 into prometheus-community:main May 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants