Skip to content

fix(observability): use the chart's key format to disable RecordingRulesNoData - #4199

Merged
Tanguille merged 1 commit into
mainfrom
fix/recordingrules-key
Jul 27, 2026
Merged

fix(observability): use the chart's key format to disable RecordingRulesNoData#4199
Tanguille merged 1 commit into
mainfrom
fix/recordingrules-key

Conversation

@Tanguille

Copy link
Copy Markdown
Owner

Follow-up to #4195, caught while verifying the merge in-cluster.

The alert was still firing after reconcile. The HelmRelease value applied cleanly (kubectl get hr shows recordingRulesNoData: {create: false}), but the rule remained in the rendered VMRule, because defaultRules.rules is keyed by the exact alert name with an enabled field. The chart says so in its own commented example:

rules: {}
  # CPUThrottlingHigh:
  #   enabled: false

So recordingRulesNoData: create: false matched nothing and was silently ignored. Corrected to RecordingRulesNoData: enabled: false.

The neighbouring etcdMemberCommunicationSlow entry is deliberately left alone: its key already matches its alert name verbatim, and that alert is absent from the rendered rules, so it is doing its job.

Rationale for disabling is unchanged from #4195: the rule flags recording rules that are empty by design. count:up0 is count(up == 0), which yields no samples precisely when every scrape target is healthy, as do the orphan-pod variants of namespace_workload_pod:kube_pod_owner:relabel.

…lesNoData

The rule survived the merge: defaultRules.rules is keyed by the exact alert
name with an 'enabled' field, per the chart's own commented example, so the
camelCased 'recordingRulesNoData: create: false' matched nothing and the alert
kept firing. Verified against the live VMRule after reconcile.

The neighbouring etcdMemberCommunicationSlow entry is left as-is: its key
already matches its alert name verbatim and that alert is absent from the
rendered rules.
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@Tanguille, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0dfc787e-bc89-4a7b-a229-f9491db10918

📥 Commits

Reviewing files that changed from the base of the PR and between db2be8b and f0678d7.

📒 Files selected for processing (1)
  • kubernetes/apps/observability/victoria-metrics/app/helmrelease.yaml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/recordingrules-key

Comment @coderabbitai help to get the list of available commands.

@tanguille-cluster

Copy link
Copy Markdown
@@ data.config.yaml @@
# v1/ConfigMap/observability/victoria-metrics-sync-job-config
! ± value change in multiline text (two inserts, two deletions)
  common:
    clusterLabel: "cluster"
    multicluster: false
  dashboards:
  
  [91 lines unchanged)]
  
        spec:
          annotations: {}
          labels: {}
    rules: 
-     etcdMemberCommunicationSlow:
+     RecordingRulesNoData:
        enabled: false
-     recordingRulesNoData:
+     etcdMemberCommunicationSlow:
        enabled: false
    groups: 
      kubernetes-apps:
        jobNamespaces:
  
  [35 lines unchanged)]
  
      - enabled: true
        url: https://raw.githubusercontent.com/VictoriaMetrics/VictoriaMetrics/master/deployment/docker/rules/alerts-single-node.yml
      - enabled: false
        url: https://raw.githubusercontent.com/VictoriaMetrics/VictoriaTraces/master/deployment/docker/rules/alerts-health.yml

@tanguille-cluster

Copy link
Copy Markdown
@@ spec.values.defaultRules.rules @@
# helm.toolkit.fluxcd.io/v2/HelmRelease/observability/victoria-metrics
! - one map entry removed:
- recordingRulesNoData:
-   create: false
! + one map entry added:
+ RecordingRulesNoData:
+   enabled: false

@Tanguille
Tanguille marked this pull request as ready for review July 27, 2026 15:14
@Tanguille
Tanguille merged commit 5f7b511 into main Jul 27, 2026
15 checks passed
@Tanguille
Tanguille deleted the fix/recordingrules-key branch July 27, 2026 15:14
@github-actions

Copy link
Copy Markdown

AI Automated Review

Analysis engine: qwen-3.6-fast@http://litellm.ai.svc.cluster.local/v1 (openai)

Recommendation

Approve. The PR correctly fixes a configuration bug in the victoria-metrics HelmRelease where an alert rule was not being disabled due to a mismatched key and field format. The change is minimal, targeted, and aligns with the chart's documented behavior.

Change-by-Change Findings

  • kubernetes/apps/observability/victoria-metrics/app/helmrelease.yaml:
    • Changed recordingRulesNoData: create: false to RecordingRulesNoData: enabled: false.
    • Verified that the new key matches the exact alert name (RecordingRulesNoData) and uses the correct boolean field (enabled) as per the chart's example comments cited in the PR body.
    • Confirmed that the neighboring etcdMemberCommunicationSlow entry remains unchanged, which is correct as its key already matched the alert name verbatim.

Standards Compliance

  • Follows Conventional Commit title format (fix(observability): ...).
  • Adheres to repository standards for small, focused changes.
  • No shell scripts or secrets involved.

Must Check Verification

  • Validate manifest against target cluster version: The change only modifies values within a HelmRelease spec for VictoriaMetrics. It does not introduce new API versions or kinds that would depend on a specific Kubernetes cluster version. The existing helm.toolkit.fluxcd.io/v2 API is standard for Flux v2 clusters. No validation errors expected.
  • Check for resource quota / limit changes: No resource requests or limits were modified in this diff. Only the defaultRules.rules configuration block was updated.

Unknowns or Needs Verification

  • None. The evidence provided in the PR body and diff is sufficient to verify the correctness of the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant