dd image fix

[arikalon1]

No description provided.

[coderabbitai]

Walkthrough

Expanded Datadog metrics instructions to require using kubectl_find_resource when Kubernetes resource type is ambiguous and to ask the user for clarification if the type cannot be determined. Removed one example query line. Updated a test case to consolidate prompts about robusta-runner graphs.

Changes

Cohort / File(s)	Change Summary
Datadog Metrics Instructions Update holmes/plugins/toolsets/datadog/datadog_metrics_instructions.jinja2	Clarified handling of ambiguous Kubernetes resource types (explicitly require kubectl_find_resource), added guidance to ask the user when type is undeterminable, and removed a sample query line.
Test Case Prompt Update tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml	Consolidated multiple node-exporter prompts into a single "generate me a graph of robusta-runner" prompt; other prompts and expected output unchanged.

Sequence Diagram(s)

Not applicable — changes are instructional text and test prompt edits only, with no control-flow or feature logic modifications.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~7 minutes

Possibly related PRs

• return errors on empty queries. Prompy update #797: Modifies the same Datadog instructions to require/clarify using kubectl_find_resource for ambiguous Kubernetes resources.
• DataDog metrics toolset improvement #758: Also updates the Datadog instructions, focusing on pod/resource name resolution and related guidance.

Suggested reviewers

• moshemorad
• nherment

---

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 43c8266 and c5256c6.

📒 Files selected for processing (1)

• tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

• tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: llm_evals
• GitHub Check: Pre-commit checks
• GitHub Check: Pre-commit checks

✨ Finishing Touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch dd-image-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7b5570e and 168b692.

📒 Files selected for processing (2)

• holmes/plugins/toolsets/datadog/datadog_metrics_instructions.jinja2 (1 hunks)
• tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml (1 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

tests/llm/fixtures/**

📄 CodeRabbit Inference Engine (CLAUDE.md)

Mock data must be located in tests/llm/fixtures/{test_name}/

Files:

• tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml

🧠 Learnings (1)

📚 Learning: in llm-as-judge test cases for holmesgpt, expected outputs should be descriptive rather than prescri...

Learnt from: Sheeproid
PR: robusta-dev/holmesgpt#586
File: tests/llm/fixtures/test_ask_holmes/03_what_is_the_command_to_port_forward/test_case.yaml:4-4
Timestamp: 2025-07-02T10:27:17.231Z
Learning: In LLM-as-judge test cases for HolmesGPT, expected outputs should be descriptive rather than prescriptive when testing for flexible responses like port numbers. Using specific values in expected outputs can cause unnecessary test failures when the AI generates different but equally valid responses.

Applied to files:

• tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: Pre-commit checks
• GitHub Check: llm_evals
• GitHub Check: Pre-commit checks

🔇 Additional comments (3)

holmes/plugins/toolsets/datadog/datadog_metrics_instructions.jinja2 (2)

4-4: LGTM - formatting improvement.

The empty line addition improves readability by providing visual separation.

---

8-8: Excellent guidance for handling ambiguous queries.

This instruction promotes clarity and prevents errors by explicitly directing the AI to ask for clarification rather than making potentially incorrect assumptions.

tests/llm/fixtures/test_ask_holmes/110_cpu_graph_robusta_runner/test_case.yaml (1)

2-2: Good test case simplification.

The new prompt is well-formed and consistent with the test case's focus on "robusta-runner". This simplification improves test maintainability and aligns well with the enhanced Kubernetes workload handling instructions.

[github-actions]

Results of HolmesGPT evals

• ask_holmes: 24/40 test cases were successful, 1 regressions, 2 skipped, 13 mock failures

Test suite	Test case	Status
ask	01_how_many_pods	✅
ask	02_what_is_wrong_with_pod	🔧
ask	03_what_is_the_command_to_port_forward	🔧
ask	04_related_k8s_events	↪️
ask	05_image_version	🔧
ask	09_crashpod	✅
ask	10_image_pull_backoff	🔧
ask	11_init_containers	✅
ask	14_pending_resources	✅
ask	15_failed_readiness_probe	🔧
ask	17_oom_kill	✅
ask	18_crash_looping_v2	✅
ask	19_detect_missing_app_details	🔧
ask	20_long_log_file_search	❌
ask	24_misconfigured_pvc	🔧
ask	28_permissions_error	✅
ask	29_events_from_alert_manager	↪️
ask	39_failed_toolset	🔧
ask	41_setup_argo	✅
ask	42_dns_issues_steps_new_tools	🔧
ask	43_current_datetime_from_prompt	✅
ask	45_fetch_deployment_logs_simple	✅
ask	51_logs_summarize_errors	🔧
ask	53_logs_find_term	✅
ask	54_not_truncated_when_getting_pods	🔧
ask	59_label_based_counting	✅
ask	60_count_less_than	🔧
ask	61_exact_match_counting	✅
ask	63_fetch_error_logs_no_errors	✅
ask	79_configmap_mount_issue	🔧
ask	83_secret_not_found	🔧
ask	86_configmap_like_but_secret	🔧
ask	89_runbook_missing_cloudwatch	✅
ask	93_calling_datadog	✅
ask	93_calling_datadog	✅
ask	93_calling_datadog	✅
ask	97_logs_clarification_needed	✅
ask	110_k8s_events_image_pull	🔧
ask	24a_misconfigured_pvc_basic	🔧
ask	13a_pending_node_selector_basic	🔧

Legend

• ✅ the test was successful
• ↪️ the test was skipped
• ⚠️ the test failed but is known to be flaky or known to fail
• 🔧 the test failed due to mock data issues (not a code regression)
• ❌ the test failed and should be fixed before merging the PR