Skip to content

allow Holmes to read CRDs - #1088

Merged
arikalon1 merged 1 commit into
masterfrom
crds-roles
Oct 29, 2025
Merged

arikalon1 merged 1 commit into
masterfrom
crds-roles

Conversation

@arikalon1

Copy link
Copy Markdown
Collaborator

Will help to generate the CRDs feature required cluster roles fix aws mcp instructions

Will help to generate the CRDs feature required cluster roles
fix aws mcp instructions
@arikalon1
arikalon1 requested a review from RoiGlinik October 29, 2025 08:23
@coderabbitai

coderabbitai Bot commented Oct 29, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

The PR modifies Kubernetes RBAC configuration to grant CRD list/get permissions and significantly enhances AWS MCP server instructions by increasing CloudTrail result limits and adding comprehensive memory optimization guidelines, pagination best practices, and investigation strategies.

Changes

Cohort / File(s) Summary
RBAC CRD Permissions
helm/holmes/templates/holmesgpt-service-account.yaml
Adds ClusterRole rule granting list/get verbs for customresourcedefinitions in the apiextensions.k8s.io API group.
AWS MCP Instructions Enhancement
helm/holmes/templates/mcp-servers/aws/_helpers.tpl
Increases CloudTrail lookup-events max-items from 50 to 100; introduces MEMORY OPTIMIZATION GUIDELINES section with per-service item limits, critical rules, investigation principles, query examples, progressive investigation strategy, and pagination best practices; adds EKS pod/container CloudWatch logs filtering bash snippet; updates CloudTrail commands throughout to reflect higher max-items.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • Verify RBAC permissions are correctly scoped for CRD operations
  • Validate completeness and accuracy of memory optimization guidelines against AWS API documentation
  • Confirm all CloudTrail command examples reflect the increased max-items=100 limit correctly
  • Check pagination examples align with AWS SDK/CLI patterns and supported services

Possibly related PRs

  • Update permissions.md #751: Documents how to add custom cluster role rules via Helm, directly relevant to the RBAC configuration change introduced here.
  • aws mcp addon #1063: Modifies the same AWS MCP template file (helm/holmes/templates/mcp-servers/aws/_helpers.tpl), making changes to LLM instructions content in the same location as this PR's enhancements.

Suggested reviewers

  • pavangudiwada
  • nherment

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The pull request title "allow Holmes to read CRDs" directly describes the primary change in the changeset, which is the addition of RBAC permissions in the ClusterRole to grant list and get permissions for CustomResourceDefinitions. The title is concise, specific, and clearly communicates the main objective to a developer scanning commit history. While the PR includes secondary changes to AWS MCP instructions, the title appropriately focuses on the core modification.
Description Check ✅ Passed The pull request description references relevant components of the changeset, including generating the CRDs feature, fixing required cluster roles, and updating AWS MCP instructions. Although the description is poorly articulated and somewhat vague, it is not completely off-topic and does mention real aspects of the changes present in the files. Per the lenient evaluation criteria for this check, the description sufficiently relates to the changeset content.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch crds-roles

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown
Contributor

Results of HolmesGPT evals

  • ask_holmes: 34/36 test cases were successful, 1 regressions, 1 setup failures
Test suite Test case Status
ask 01_how_many_pods ✅
ask 02_what_is_wrong_with_pod ✅
ask 04_related_k8s_events ✅
ask 05_image_version ✅
ask 09_crashpod ✅
ask 10_image_pull_backoff ✅
ask 110_k8s_events_image_pull ✅
ask 11_init_containers ✅
ask 13a_pending_node_selector_basic ✅
ask 14_pending_resources ✅
ask 15_failed_readiness_probe ✅
ask 17_oom_kill ✅
ask 18_oom_kill_from_issues_history ✅
ask 19_detect_missing_app_details ✅
ask 20_long_log_file_search ✅
ask 24_misconfigured_pvc ✅
ask 24a_misconfigured_pvc_basic ✅
ask 28_permissions_error 🚧
ask 39_failed_toolset ✅
ask 41_setup_argo ✅
ask 42_dns_issues_steps_new_tools ✅
ask 43_current_datetime_from_prompt ✅
ask 45_fetch_deployment_logs_simple ✅
ask 51_logs_summarize_errors ✅
ask 53_logs_find_term ✅
ask 54_not_truncated_when_getting_pods ✅
ask 59_label_based_counting ✅
ask 60_count_less_than ✅
ask 61_exact_match_counting ✅
ask 63_fetch_error_logs_no_errors ✅
ask 79_configmap_mount_issue ✅
ask 83_secret_not_found ✅
ask 86_configmap_like_but_secret ❌
ask 93_calling_datadog[0] ✅
ask 93_calling_datadog[1] ✅
ask 93_calling_datadog[2] ✅

Legend

  • ✅ the test was successful
  • :minus: the test was skipped
  • ⚠️ the test failed but is known to be flaky or known to fail
  • 🚧 the test had a setup failure (not a code regression)
  • 🔧 the test failed due to mock data issues (not a code regression)
  • 🚫 the test was throttled by API rate limits/overload
  • ❌ the test failed and should be fixed before merging the PR

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
helm/holmes/templates/holmesgpt-service-account.yaml (1)

6-6: ClusterRole is cluster-scoped; drop metadata.namespace.

ClusterRoles must not be namespaced. Keeping namespace can confuse linters and operators.

Apply:

 metadata:
   name: {{ .Release.Name }}-holmes-cluster-role
-  namespace : {{ .Release.Namespace }}
🧹 Nitpick comments (8)
helm/holmes/templates/holmesgpt-service-account.yaml (2)

77-84: Duplicate HPA permissions; keep one.

HorizontalPodAutoscaler rules under apiGroup autoscaling appear twice. Remove the second to reduce noise and drift.

-  - apiGroups:
-      - autoscaling
-    resources:
-    - horizontalpodautoscalers
-    verbs:
-      - get
-      - list

Also applies to: 150-156


99-112: extensions API group is deprecated; consider removal or gating per cluster version.

You already cover apps/* and networking.k8s.io/ingresses. Keeping extensions/* is legacy. Gate behind a values flag or remove if you don’t support very old clusters.

helm/holmes/templates/mcp-servers/aws/_helpers.tpl (6)

36-39: Be consistent with pagination flags across examples.

You mix CLI paginator flags (--max-items/--starting-token) with service-level ones (--max-results/--next-token). Prefer one style per section and mention the alternative once to avoid confusion.

Also applies to: 178-184


46-50: Tighten EKS logs query to cut noise and payload.

Add a basic filter-pattern and show region placeholder for clarity.

 aws logs filter-log-events \
   --log-group-name /aws/containerinsights/CLUSTER_NAME/application \
   --start-time $(date -d '1 hour ago' +%s)000 \
-  --max-items 500
+  --filter-pattern "ERROR || Exception" \
+  --max-items 500 \
+  --region REGION

69-70: Use explicit UTC designator in ISO time.

Append Z to avoid locale ambiguity in some shells/environments.

-aws cloudtrail lookup-events --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) --max-items 100
+aws cloudtrail lookup-events --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) --max-items 100

210-218: Clarify paginator tokens per service.

For CloudWatch Logs, examples often use --next-token (service) while CloudTrail uses --starting-token (CLI paginator). Document both to match the flags used in preceding examples.

Example addition:

# CloudWatch Logs: either
aws logs filter-log-events ... --max-items 200 --starting-token <Token>
# or service-level
aws logs filter-log-events ... --limit 100 --next-token <Token>

Also applies to: 223-229


162-173: Optional: show JMESPath --query to shrink payloads.

Helps memory further when scanning logs.

aws logs filter-log-events ... --max-items 300 --query 'events[].{ts:timestamp,msg:message,stream:logStreamName}'

145-156: Optional: demonstrate --page-size with --max-items.

Smaller pages reduce peak memory while preserving total cap.

aws cloudtrail lookup-events --start-time ... --max-items 200 --page-size 50
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1855dfa and b54d185.

📒 Files selected for processing (2)
  • helm/holmes/templates/holmesgpt-service-account.yaml (1 hunks)
  • helm/holmes/templates/mcp-servers/aws/_helpers.tpl (5 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Pre-commit checks
  • GitHub Check: llm_evals
  • GitHub Check: build
🔇 Additional comments (2)
helm/holmes/templates/holmesgpt-service-account.yaml (1)

132-139: CRD read RBAC: looks correct; confirm if watch is needed.

Granting list/get on apiextensions.k8s.io/customresourcedefinitions aligns with “read CRDs.” If Holmes needs to react to CRD changes, add watch; otherwise current scope is least-privilege.

If watch is required, apply:

   verbs:
-    - "list"
-    - "get"
+    - "list"
+    - "get"
+    - "watch"
helm/holmes/templates/mcp-servers/aws/_helpers.tpl (1)

35-39: LGTM on raising CloudTrail cap to 100.

Matches the new memory guidance while staying safe for typical investigations.

Comment thread helm/holmes/templates/mcp-servers/aws/_helpers.tpl
@arikalon1
arikalon1 merged commit 93ddfe7 into master Oct 29, 2025
8 checks passed
@arikalon1
arikalon1 deleted the crds-roles branch October 29, 2025 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants