Skip to content

ROB-2714 more default crd permissions - #1204

Merged
RoiGlinik merged 9 commits into
masterfrom
ROB-2714-more-default-crd-permissions
Dec 24, 2025
Merged

RoiGlinik merged 9 commits into
masterfrom
ROB-2714-more-default-crd-permissions

Conversation

@RoiGlinik

@RoiGlinik RoiGlinik commented Dec 18, 2025 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Added granular CRD RBAC toggles (Argo, Flux, Kafka, KEDA, Crossplane, Istio, Gateway API, Velero) with read-only defaults and per-chart example configurations for Holmes and Robusta.
  • Documentation

    • Reworked in-cluster permissions docs: new Default CRD Permissions, expanded Custom Permissions and Common Scenarios, replaced ArgoCD example with Cert‑Manager, and updated deployment guidance and example values.

✏️ Tip: You can customize this high-level summary in your review settings.

Signed-off-by: Roi Glinik <groi.tech@gmail.com>
Signed-off-by: Roi Glinik <groi.tech@gmail.com>
Signed-off-by: Roi Glinik <groi.tech@gmail.com>
@coderabbitai

coderabbitai Bot commented Dec 21, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

Reworks in-cluster permissions docs and adds configurable per-CRD read permissions to the Holmes Helm chart via a new crdPermissions values block and conditional RBAC rules in the service-account/ClusterRole template. Documentation includes per-chart examples and custom-permissions guidance. (47 words)

Changes

Cohort / File(s) Summary
Documentation - Permissions Guide
docs/data-sources/permissions.md
Replaces "Common Scenarios" with "Default CRD Permissions"; adds per-chart examples (Holmes, Robusta) and a new "Adding Custom Permissions" section; swaps ArgoCD example for Cert‑Manager (certificates, certificaterequests, issuers, clusterissuers) and updates YAML snippets.
Helm values - CRD flags
helm/holmes/values.yaml
Adds a crdPermissions object with boolean flags: argo, flux, kafka, keda, crossplane, istio, gatewayApi, velero (inserted after customClusterRoleRules) to control per-operator CRD read permissions.
Helm RBAC template - conditional rules
helm/holmes/templates/holmesgpt-service-account.yaml
Adds conditional ClusterRole/Role rule blocks guarded by .Values.crdPermissions.*; each enabled flag injects get, list, watch rules for the corresponding CRD apiGroup(s); replaces an explicit Prometheus CRD list with a wildcard for monitoring.coreos.com.

Sequence Diagram(s)

(omitted — changes are documentation and Helm templating/configuration only)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Possibly related PRs

Suggested reviewers

  • moshemorad
  • arikalon1
  • pavangudiwada

Pre-merge checks

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: expanding default CRD permissions configuration across multiple Kubernetes operators.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between de9596f and 3a47e68.

📒 Files selected for processing (1)
  • helm/holmes/templates/holmesgpt-service-account.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: build (3.11)
  • GitHub Check: build (3.12)
  • GitHub Check: build (3.10)
  • GitHub Check: llm_evals
  • GitHub Check: build
🔇 Additional comments (3)
helm/holmes/templates/holmesgpt-service-account.yaml (3)

184-192: LGTM! Wildcard approach aligns with past feedback.

The wildcard resource pattern for Prometheus CRDs is cleaner and more maintainable than enumerating specific resources. This change matches the previous reviewer's suggestion and future-proofs the permissions against new Prometheus CRD types.


193-277: Well-structured conditional CRD permissions.

The implementation follows a consistent pattern with appropriate read-only permissions gated by explicit configuration flags. This opt-in approach balances flexibility with security by allowing users to grant only the CRD access they need.


193-202: Verify whether Holmes actually requires these additional Argo and Istio API groups.

The current configuration grants wildcard access to argoproj.io (lines 193-202) and networking.istio.io, telemetry.istio.io (lines 247-257). The codebase does not show usage of the additional API groups mentioned:

  • Argo: No references to workflow.argoproj.io
  • Istio: No references to security.istio.io, install.istio.io, or extensions.istio.io

Before adding these API groups, confirm whether Holmes actually queries these resource types. If Holmes uses argocd CLI (as shown in the toolsets) rather than direct Kubernetes API calls to Argo CRDs, additional API groups may not be necessary. Similarly, verify if Istio integration queries security policies, IstioOperators, or WasmPlugins.

If these resources are required for comprehensive observability, add them to the respective blocks. Otherwise, the current wildcard grants for the included API groups are sufficient.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
helm/holmes/templates/holmesgpt-service-account.yaml (1)

184-209: Consider clarifying the comment to reflect both Prometheus and optional CRD permissions.

The comment # Prometheus CRDs on line 184 now precedes not just Prometheus CRDs but also conditionally-included CRDs for Argo, Flux, Kafka, KEDA, Crossplane, Istio, Gateway API, and Velero. Consider updating the comment to reflect the broader scope of CRD permissions in this section.

Suggested comment update
-  # Prometheus CRDs
+  # Prometheus CRDs and optional CRD permissions for common operators
docs/data-sources/permissions.md (1)

8-41: Consider adding more context about what each CRD permission enables.

The documentation shows the available CRD permission flags but doesn't explain what each one controls or when you might want to disable them. Adding brief descriptions would help users make informed decisions about which permissions to enable.

Suggested enhancement

Consider adding a table or list explaining each flag:

## Default CRD Permissions

HolmesGPT includes read-only permissions for common Kubernetes operators and tools by default. These can be individually enabled or disabled:

- **argo**: ArgoCD applications, Argo Workflows, and Argo Rollouts
- **flux**: Flux GitOps toolkit resources (sources, kustomizations, helm releases)
- **kafka**: Strimzi Kafka operator resources
- **keda**: KEDA autoscaling resources
- **crossplane**: Crossplane providers and compositions
- **istio**: Istio service mesh resources
- **gatewayApi**: Kubernetes Gateway API resources
- **velero**: Velero backup and restore resources
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3f1d3a3 and 40f8983.

📒 Files selected for processing (3)
  • docs/data-sources/permissions.md (2 hunks)
  • helm/holmes/templates/holmesgpt-service-account.yaml (2 hunks)
  • helm/holmes/values.yaml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: llm_evals
  • GitHub Check: build (3.11)
  • GitHub Check: build (3.12)
  • GitHub Check: build (3.10)
  • GitHub Check: build
🔇 Additional comments (2)
docs/data-sources/permissions.md (1)

51-90: LGTM! Cert-Manager is a good example choice.

The switch from ArgoCD to Cert-Manager for the custom permissions example is appropriate, since Cert-Manager is common but not included in the default CRD permissions. The examples correctly show the syntax for both Holmes and Robusta Helm charts.

helm/holmes/templates/holmesgpt-service-account.yaml (1)

210-383: Well-structured RBAC configuration across multiple operators.

The conditional permission blocks are consistently implemented and appropriate for read-only access. The listed CRD resources align with documented APIs for ArgoCD, Flux CD, Strimzi Kafka, KEDA, and Crossplane. However, verification against the latest Istio, Kubernetes Gateway API, and Velero operator documentation would be beneficial to ensure complete accuracy, particularly for operators that may have evolved their API structures in recent releases.

Comment thread helm/holmes/values.yaml
@RoiGlinik
RoiGlinik requested a review from moshemorad December 24, 2025 09:13
@github-actions

github-actions Bot commented Dec 24, 2025 •

Copy link
Copy Markdown
Contributor

Dev Docker images are ready for this commit:

Use this tag to pull the image for testing.

⚠️ Temporary images are deleted after 30 days. Copy to a permanent registry before using them:

gcloud auth configure-docker us-central1-docker.pkg.dev
docker pull us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:dbb42b1
docker tag us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:dbb42b1 me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:dbb42b1
docker push me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:dbb42b1

Patch Helm values in one line (choose the chart you use):

  • HolmesGPT chart:
helm upgrade --install holmesgpt ./helm/holmes \
  --set registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set image=holmes-dev:dbb42b1
  • Robusta wrapper chart:
helm upgrade --install robusta robusta/robusta \
  --reuse-values \
  --set holmes.registry=me-west1-docker.pkg.dev/robusta-development/development \
  --set holmes.image=holmes-dev:dbb42b1

Avi-Robusta
Avi-Robusta previously approved these changes Dec 24, 2025
arikalon1
arikalon1 previously approved these changes Dec 24, 2025

@arikalon1 arikalon1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good., left small comment

Comment thread helm/holmes/templates/holmesgpt-service-account.yaml Outdated
Comment thread docs/data-sources/permissions.md
Signed-off-by: Roi Glinik <groi.tech@gmail.com>
@RoiGlinik
RoiGlinik dismissed stale reviews from arikalon1 and Avi-Robusta via de9596f December 24, 2025 13:53
Signed-off-by: Roi Glinik <groi.tech@gmail.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
helm/holmes/templates/holmesgpt-service-account.yaml (1)

193-312: Well-structured conditional CRD permission blocks with proper defaults.

The implementation is clean and consistent across all CRD groups. The crdPermissions defaults are properly defined in values.yaml with all groups enabled. A few observations:

  1. Istio coverage: Consider adding security.istio.io for resources like AuthorizationPolicy and PeerAuthentication, which are commonly needed when troubleshooting Istio-related issues.

  2. Crossplane coverage: The crossplane.io core API group (for CompositeResourceDefinitions, Compositions) may also be useful, though the current coverage of pkg and apiextensions handles the most common troubleshooting scenarios.

🔎 Optional: Add Istio security API group
 {{- if .Values.crdPermissions.istio }}
   - apiGroups:
       - networking.istio.io
     resources:
       - "*"
     verbs:
       - get
       - list
       - watch
   - apiGroups:
       - telemetry.istio.io
     resources:
       - "*"
     verbs:
       - get
       - list
       - watch
+  - apiGroups:
+      - security.istio.io
+    resources:
+      - "*"
+    verbs:
+      - get
+      - list
+      - watch
 {{- end }}
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 40f8983 and de9596f.

📒 Files selected for processing (1)
  • helm/holmes/templates/holmesgpt-service-account.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: llm_evals
  • GitHub Check: build (3.12)
  • GitHub Check: build (3.11)
  • GitHub Check: build (3.10)
  • GitHub Check: build
🔇 Additional comments (1)
helm/holmes/templates/holmesgpt-service-account.yaml (1)

184-192: LGTM! Good implementation of the wildcard approach.

This addresses the previous review feedback by using "*" for resources under monitoring.coreos.com, ensuring any future Prometheus Operator CRDs are automatically covered without requiring chart updates.

@RoiGlinik
RoiGlinik enabled auto-merge (squash) December 24, 2025 14:08
@RoiGlinik
RoiGlinik requested a review from arikalon1 December 24, 2025 14:08
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

Results of HolmesGPT evals

  • ask_holmes: 27/37 test cases were successful, 1 regressions, 6 setup failures, 3 mock failures
Test suite Test case Status
ask 01_how_many_pods 🚧
ask 02_what_is_wrong_with_pod 🚧
ask 04_related_k8s_events ✅
ask 05_image_version ✅
ask 09_crashpod ✅
ask 10_image_pull_backoff ✅
ask 110_k8s_events_image_pull ✅
ask 11_init_containers ✅
ask 13a_pending_node_selector_basic ✅
ask 14_pending_resources ✅
ask 15_failed_readiness_probe ✅
ask 163_compaction_follow_up ✅
ask 17_oom_kill 🚧
ask 18_oom_kill_from_issues_history ✅
ask 19_detect_missing_app_details ✅
ask 20_long_log_file_search ❌
ask 24_misconfigured_pvc ✅
ask 24a_misconfigured_pvc_basic ✅
ask 28_permissions_error 🚧
ask 39_failed_toolset ✅
ask 41_setup_argo ✅
ask 42_dns_issues_steps_new_tools ✅
ask 43_current_datetime_from_prompt ✅
ask 45_fetch_deployment_logs_simple ✅
ask 51_logs_summarize_errors ✅
ask 53_logs_find_term ✅
ask 54_not_truncated_when_getting_pods ✅
ask 59_label_based_counting 🚧
ask 60_count_less_than ✅
ask 61_exact_match_counting 🚧
ask 63_fetch_error_logs_no_errors ✅
ask 79_configmap_mount_issue ✅
ask 83_secret_not_found ✅
ask 86_configmap_like_but_secret ✅
ask 93_calling_datadog[0] 🔧
ask 93_calling_datadog[1] 🔧
ask 93_calling_datadog[2] 🔧

Legend

  • ✅ the test was successful
  • :minus: the test was skipped
  • ⚠️ the test failed but is known to be flaky or known to fail
  • 🚧 the test had a setup failure (not a code regression)
  • 🔧 the test failed due to mock data issues (not a code regression)
  • 🚫 the test was throttled by API rate limits/overload
  • ❌ the test failed and should be fixed before merging the PR

@RoiGlinik
RoiGlinik merged commit daedf3f into master Dec 24, 2025
10 of 11 checks passed
@RoiGlinik
RoiGlinik deleted the ROB-2714-more-default-crd-permissions branch December 24, 2025 14:47
@coderabbitai coderabbitai Bot mentioned this pull request Dec 25, 2025
moshemorad pushed a commit that referenced this pull request Dec 25, 2025
add default read permission to some kubernetes tools
---------

Signed-off-by: Roi Glinik <groi.tech@gmail.com>
Signed-off-by: Mohse Morad <moshemorad12340@gmail.com>
FilipGrebowski pushed a commit to FilipGrebowski/holmesgpt that referenced this pull request Dec 27, 2025
add default read permission to some kubernetes tools
---------

Signed-off-by: Roi Glinik <groi.tech@gmail.com>
Signed-off-by: Filip Grebowski <grebowskifilip@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants