Skip to content

Rename tempo and loki toolset in docs - #787

Merged
aantn merged 6 commits into
masterfrom
rename-tempo-toolset
Aug 10, 2025
Merged

aantn merged 6 commits into
masterfrom
rename-tempo-toolset

Conversation

@aantn

@aantn aantn commented Aug 5, 2025

Copy link
Copy Markdown
Collaborator

To reflect that they can be used without grafana

@aantn
aantn requested a review from moshemorad August 5, 2025 11:54
@coderabbitai

coderabbitai Bot commented Aug 5, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

This update standardizes the naming of Grafana-related toolsets across documentation, configuration, error messages, and tests. Instances of "Grafana Loki" are renamed to "Loki," and "Grafana Tempo" to "Tempo" or "Grafana" where appropriate. Minor documentation and comment updates are included, along with a namespace creation addition in a test fixture.

Changes

Cohort / File(s) Change Summary
Documentation: Toolset Naming Standardization
docs/data-sources/builtin-toolsets/datetime.md, docs/data-sources/builtin-toolsets/grafanaloki.md, docs/data-sources/builtin-toolsets/grafanatempo.md, docs/data-sources/builtin-toolsets/index.md, docs/snippets/toolsets_that_provide_logging.md
Updated references from "Grafana Loki" to "Loki" and "Grafana Tempo" to "Tempo" or "Grafana."
Navigation Configuration
mkdocs.yml
Renamed navigation entries "Grafana Loki" to "Loki" and "Grafana Tempo" to "Tempo."
Toolset Plugin Error Message
holmes/plugins/toolsets/grafana/toolset_grafana_loki.py
Updated error message to reference "Loki" instead of "Grafana Loki."
Test Fixture: Namespace Creation
tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml
Added explicit creation of Kubernetes namespace before applying manifests in test setup.
Test Fixture: Comment Update
tests/llm/fixtures/test_investigate/14_tempo/toolsets.yaml
Updated comment to remove "Grafana" from "Grafana tempo" for consistency.
README Update
README.md
Renamed "Grafana Loki" to "Loki" and "Grafana Tempo" to "Tempo" in the Data Sources section.

Sequence Diagram(s)

Not applicable: No new features or control flow changes introduced.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested reviewers

  • pavangudiwada

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between be92c1d and 0920a14.

📒 Files selected for processing (1)
  • mkdocs.yml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • mkdocs.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Pre-commit checks
  • GitHub Check: llm_evals
  • GitHub Check: Pre-commit checks
✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch rename-tempo-toolset

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@aantn
aantn marked this pull request as ready for review August 5, 2025 11:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (1)
tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml (1)

6-10: Delete the dedicated namespace in after_test to avoid residue

The test now creates app-23, but it is never removed.
Keeping stale namespaces can slow future CI runs and pollute shared clusters.

Suggested adjustment:

 after_test: |
   kubectl delete -f ./manifests/
+  kubectl delete namespace app-23 || true
🧹 Nitpick comments (2)
tests/llm/fixtures/test_investigate/14_tempo/toolsets.yaml (1)

4-6: YAML boolean style is inconsistent with the rest of the file.
enabled: True (Line 5) uses Python-style True, while every other boolean in the file is lowercase YAML (true/false). Mixing styles can surprise YAML parsers that enforce canonical booleans.

-    enabled: True
+    enabled: true
docs/data-sources/builtin-toolsets/grafanaloki.md (1)

1-1: Header rename LGTM, but consider renaming the file for full consistency.
The visible title is now “Loki,” yet the filename (grafanaloki.md) and internal toolset slug grafana/loki still carry the grafana prefix. If the long-term goal is to drop the prefix everywhere, renaming the file (and updating mkdocs navigation & inbound links) will avoid confusion down the road.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6d84eb0 and 797297c.

📒 Files selected for processing (9)
  • docs/data-sources/builtin-toolsets/datetime.md (1 hunks)
  • docs/data-sources/builtin-toolsets/grafanaloki.md (1 hunks)
  • docs/data-sources/builtin-toolsets/grafanatempo.md (1 hunks)
  • docs/data-sources/builtin-toolsets/index.md (1 hunks)
  • docs/snippets/toolsets_that_provide_logging.md (1 hunks)
  • holmes/plugins/toolsets/grafana/toolset_grafana_loki.py (1 hunks)
  • mkdocs.yml (1 hunks)
  • tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml (1 hunks)
  • tests/llm/fixtures/test_investigate/14_tempo/toolsets.yaml (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py

📄 CodeRabbit Inference Engine (CLAUDE.md)

**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Use Ruff for formatting and linting (configured in pyproject.toml)
Type hints required (mypy configuration in pyproject.toml)

Files:

  • holmes/plugins/toolsets/grafana/toolset_grafana_loki.py
🧠 Learnings (6)
📚 Learning: in robusta-dev/holmesgpt config.example.yaml, the azuremonitorlogs toolset configuration shows "enab...
Learnt from: vishiy
PR: robusta-dev/holmesgpt#782
File: config.example.yaml:31-49
Timestamp: 2025-08-05T00:42:23.792Z
Learning: In robusta-dev/holmesgpt config.example.yaml, the azuremonitorlogs toolset configuration shows "enabled: true" as an example of how to enable the toolset, not as a default setting. The toolset is disabled by default and requires explicit enablement in user configurations.

Applied to files:

  • tests/llm/fixtures/test_investigate/14_tempo/toolsets.yaml
  • docs/data-sources/builtin-toolsets/grafanatempo.md
📚 Learning: in the kubernetes logs toolset for holmes, both current and previous logs are intentionally fetched ...
Learnt from: nherment
PR: robusta-dev/holmesgpt#408
File: holmes/plugins/toolsets/kubernetes_logs.py:90-97
Timestamp: 2025-05-15T05:13:43.169Z
Learning: In the Kubernetes logs toolset for Holmes, both current and previous logs are intentionally fetched and combined for each pod, even though this requires more API calls. This design ensures all logs are captured even when pods restart but retain their name, providing complete diagnostic information.

Applied to files:

  • docs/data-sources/builtin-toolsets/grafanaloki.md
  • docs/data-sources/builtin-toolsets/grafanatempo.md
  • docs/snippets/toolsets_that_provide_logging.md
📚 Learning: each llm test must use a dedicated namespace app- (e.g., app-01, app-02) to prevent conflict...
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T07:25:36.018Z
Learning: Each LLM test must use a dedicated namespace app-<testid> (e.g., app-01, app-02) to prevent conflicts when tests run simultaneously

Applied to files:

  • tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml
📚 Learning: in llm-as-judge test cases for holmesgpt, expected outputs should be descriptive rather than prescri...
Learnt from: Sheeproid
PR: robusta-dev/holmesgpt#586
File: tests/llm/fixtures/test_ask_holmes/03_what_is_the_command_to_port_forward/test_case.yaml:4-4
Timestamp: 2025-07-02T10:27:17.231Z
Learning: In LLM-as-judge test cases for HolmesGPT, expected outputs should be descriptive rather than prescriptive when testing for flexible responses like port numbers. Using specific values in expected outputs can cause unnecessary test failures when the AI generates different but equally valid responses.

Applied to files:

  • tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml
📚 Learning: never use resource names that hint at the problem or expected behavior (e.g., avoid broken-pod, test...
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T07:25:36.018Z
Learning: Never use resource names that hint at the problem or expected behavior (e.g., avoid broken-pod, test-project-that-does-not-exist, crashloop-app). Use neutral names that don't give away what the LLM should discover

Applied to files:

  • tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml
📚 Learning: all pod names must be unique across tests (e.g., giant-narwhal, blue-whale, sea-turtle) - never reus...
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T07:25:36.018Z
Learning: All pod names must be unique across tests (e.g., giant-narwhal, blue-whale, sea-turtle) - never reuse pod names between tests

Applied to files:

  • tests/llm/fixtures/test_ask_holmes/23_app_error_in_current_logs/test_case.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Pre-commit checks
  • GitHub Check: llm_evals
  • GitHub Check: Pre-commit checks
🔇 Additional comments (6)
tests/llm/fixtures/test_investigate/14_tempo/toolsets.yaml (1)

22-22: Comment wording now consistent – looks good.
The updated comment removes the “Grafana” prefix, matching the new naming convention.

docs/snippets/toolsets_that_provide_logging.md (1)

8-8: Bullet text updated correctly.
The list now shows “Loki,” matching the broader rename.

docs/data-sources/builtin-toolsets/datetime.md (1)

9-9: Dependency list updated – looks good.
No further action needed.

mkdocs.yml (1)

168-169: Confirm page titles inside linked docs also reflect the new wording

Nav labels were shortened to “Loki” / “Tempo”, but the underlying markdown files are still named grafanaloki.md and grafanatempo.md.
Please make sure the H1 headings inside those two files were also updated, otherwise the page titles shown in the browser tab / search results will still read Grafana Loki and Grafana Tempo, undermining the consistency this PR aims for.
If not yet done, a one-liner fix in each markdown file is enough.

docs/data-sources/builtin-toolsets/index.md (1)

20-21: LGTM – label change only

Display names updated to match the new terminology; links remain intact. No further action needed.

holmes/plugins/toolsets/grafana/toolset_grafana_loki.py (1)

50-53: LGTM – message text aligns with new naming

Only the error message changed; logic remains untouched and consistent with the overall rebranding.

Comment thread docs/data-sources/builtin-toolsets/grafanatempo.md Outdated
Comment thread docs/data-sources/builtin-toolsets/grafanatempo.md Outdated
@aantn
aantn requested a review from moshemorad August 7, 2025 19:44
@aantn
aantn enabled auto-merge (squash) August 7, 2025 19:44
@github-actions

github-actions Bot commented Aug 7, 2025

Copy link
Copy Markdown
Contributor

Results of HolmesGPT evals

  • ask_holmes: 22/43 test cases were successful, 0 regressions, 1 skipped, 20 mock failures
Test suite Test case Status
ask 01_how_many_pods ✅
ask 02_what_is_wrong_with_pod 🔧
ask 03_what_is_the_command_to_port_forward 🔧
ask 04_related_k8s_events ↪️
ask 05_image_version 🔧
ask 09_crashpod ✅
ask 10_image_pull_backoff 🔧
ask 11_init_containers ✅
ask 14_pending_resources 🔧
ask 15_failed_readiness_probe 🔧
ask 17_oom_kill ✅
ask 18_crash_looping_v2 ✅
ask 19_detect_missing_app_details 🔧
ask 24_misconfigured_pvc 🔧
ask 28_permissions_error ✅
ask 29_events_from_alert_manager 🔧
ask 39_failed_toolset 🔧
ask 41_setup_argo ✅
ask 42_dns_issues_steps_new_tools 🔧
ask 43_current_datetime_from_prompt ✅
ask 45_fetch_deployment_logs_simple ✅
ask 51_logs_summarize_errors 🔧
ask 53_logs_find_term ✅
ask 54_not_truncated_when_getting_pods 🔧
ask 59_label_based_counting ✅
ask 60_count_less_than 🔧
ask 61_exact_match_counting ✅
ask 63_fetch_error_logs_no_errors ✅
ask 77_liveness_probe_misconfiguration 🔧
ask 79_configmap_mount_issue 🔧
ask 83_secret_not_found 🔧
ask 86_configmap_like_but_secret 🔧
ask 88_affinity_like_but_taints 🔧
ask 89_runbook_missing_cloudwatch 🔧
ask 90_runbook_basic_selection 🔧
ask 93_calling_datadog ✅
ask 93_calling_datadog ✅
ask 93_calling_datadog ✅
ask 97_logs_clarification_needed ✅
ask 100_historical_logs 🔧
ask 110_k8s_events_image_pull 🔧
ask 24a_misconfigured_pvc_basic 🔧
ask 13a_pending_node_selector_basic 🔧

Legend

  • ✅ the test was successful
  • ↪️ the test was skipped
  • ⚠️ the test failed but is known to be flaky or known to fail
  • 🔧 the test failed due to mock data issues (not a code regression)
  • ❌ the test failed and should be fixed before merging the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants