Repository navigation
Conversation
Add dedicated Grafana Cloud section to Loki docs with step-by-step instructions for service account setup, datasource UID discovery, and common mistake warnings. Fix trailing slash bug in get_base_url() that produced double-slash URLs (e.g. https://x.grafana.net//api/...) causing 404 errors. Also add direct Grafana Cloud Loki access method and correct the capabilities table tool name. https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK Signed-off-by: Claude <noreply@anthropic.com>
https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK Signed-off-by: Claude <noreply@anthropic.com>
✅ Deploy Preview for holmes-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
✅ Docker image ready for
Use this tag to pull the image for testing. 📋 Copy commandsgcloud auth configure-docker us-central1-docker.pkg.dev
docker pull us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:4b90f0b
docker tag us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:4b90f0b me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:4b90f0b
docker push me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:4b90f0bPatch Helm values in one line (choose the chart you use): HolmesGPT chart: helm upgrade --install holmesgpt ./helm/holmes \
--set registry=me-west1-docker.pkg.dev/robusta-development/development \
--set image=holmes-dev:4b90f0bRobusta wrapper chart: helm upgrade --install robusta robusta/robusta \
--reuse-values \
--set holmes.registry=me-west1-docker.pkg.dev/robusta-development/development \
--set holmes.image=holmes-dev:4b90f0b |
📂 Previous Runs📜 Run @ 17fa0ee (#22177689261)✅ Results of HolmesGPT evalsAutomatically triggered by commit 17fa0ee on branch Results of HolmesGPT evals
📜 Run @ 7ddeeab (#22176994733)✅ Results of HolmesGPT evalsAutomatically triggered by commit 7ddeeab on branch Results of HolmesGPT evals
📜 Run @ 7ce3762 (#22176822295)✅ Results of HolmesGPT evalsAutomatically triggered by commit 7ce3762 on branch Results of HolmesGPT evals
✅ Results of HolmesGPT evalsAutomatically triggered by commit 96716a7 on branch Results of HolmesGPT evals
📖 Legend
🔄 Re-run evals manually
Option 1: Comment on this PR with Or with more options (one per line): Run evals on a different branch (e.g., master) for comparison:
Quick re-run: Use Option 2: Trigger via GitHub Actions UI → "Run workflow" 🏷️ Valid markers
Commands: CLI: |
WalkthroughDocs reorganized to present Grafana proxy, Grafana Cloud, Self‑Hosted, and direct‑to‑Loki connection flows; code stabilizes Grafana API URL by stripping trailing slashes in get_base_url; a test expectation updated to match the corrected URL formatting. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔬 CLI Performance Benchmark🟡 Startup Time (no LLM)Measures
🟡 Full CLI with LLMMeasures
PR: |
The old Options 1/2/3 were confusing because "Grafana Cloud" is just Grafana. Restructured into two real options: - Option 1: Through Grafana (self-hosted or cloud, via tabs) - Option 2: Direct to Loki (self-hosted or cloud, via tabs) https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK Signed-off-by: Claude <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
docs/data-sources/builtin-toolsets/grafanaloki.md (1)
105-118: Consider adding abase64generation command for clarity.
Authorization: "Basic <base64 of USER_ID:API_KEY>"leaves the credential type ofAPI_KEYambiguous (Grafana Cloud API token vs account password) and doesn't show how to produce the encoded value. Consistent with the rest of the doc's hands-on approach, a one-liner would help:✏️ Suggested addition after the YAML block
+To generate the Base64 value (use your numeric user ID and a Grafana Cloud API token): +```bash +echo -n "YOUR_USER_ID:YOUR_API_KEY" | base64 +```🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/data-sources/builtin-toolsets/grafanaloki.md` around lines 105 - 118, The docs snippet for Grafana Cloud Loki direct access leaves the Authorization header unclear and lacks a command to generate the base64 string; update the description after the YAML example that contains the Authorization: "Basic <base64 of USER_ID:API_KEY>" line to explicitly state that API_KEY refers to the Grafana Cloud API key/token (not account password) and add a one-line example showing how to produce the base64 value (echo -n "USER_ID:API_KEY" | base64) so users know how to generate the encoded credential for the Authorization header.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@docs/data-sources/builtin-toolsets/grafanaloki.md`:
- Around line 105-118: The docs snippet for Grafana Cloud Loki direct access
leaves the Authorization header unclear and lacks a command to generate the
base64 string; update the description after the YAML example that contains the
Authorization: "Basic <base64 of USER_ID:API_KEY>" line to explicitly state that
API_KEY refers to the Grafana Cloud API key/token (not account password) and add
a one-line example showing how to produce the base64 value (echo -n
"USER_ID:API_KEY" | base64) so users know how to generate the encoded credential
for the Authorization header.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/data-sources/builtin-toolsets/grafanaloki.md (1)
151-155: Capabilities section violates the documentation guideline.The table lists the tool name and a description of what it does. As per coding guidelines: "In 'Capabilities' sections, don't list what a toolset/integration can do — users discover capabilities by using Holmes. Feature lists become stale quickly."
Consider removing the
## Capabilitiessection entirely.✏️ Suggested removal
-## Capabilities - -| Tool Name | Description | -|-----------|-------------| -| grafana_loki_query | Run LogQL queries against Loki |🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/data-sources/builtin-toolsets/grafanaloki.md` around lines 151 - 155, Remove the entire "## Capabilities" section (including the table and the "grafana_loki_query" row) from docs/data-sources/builtin-toolsets/grafanaloki.md so the documentation no longer enumerates feature capabilities; ensure no leftover header or table markup remains (delete the "## Capabilities" heading and the pipe-table block that lists grafana_loki_query).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/data-sources/builtin-toolsets/grafanaloki.md`:
- Around line 43-49: The documentation shows the curl command that requires
YOUR_GLSA_TOKEN before the instructions that explain how to obtain that token,
which can confuse readers; move the sentence starting "To create a service
account token: navigate to \"Administration → Service accounts\", create a new
service account with \"Viewer\" role, and generate a token (starts with
`glsa_`)." so it appears immediately before the curl block (i.e., place the
service account creation instructions above the curl command that calls "curl -H
\"Authorization: Bearer YOUR_GLSA_TOKEN\" ..."), preserving wording and
formatting (bolding if present) and keeping the curl code block unchanged.
---
Nitpick comments:
In `@docs/data-sources/builtin-toolsets/grafanaloki.md`:
- Around line 151-155: Remove the entire "## Capabilities" section (including
the table and the "grafana_loki_query" row) from
docs/data-sources/builtin-toolsets/grafanaloki.md so the documentation no longer
enumerates feature capabilities; ensure no leftover header or table markup
remains (delete the "## Capabilities" heading and the pipe-table block that
lists grafana_loki_query).
| ```bash | ||
| curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" \ | ||
| "https://YOUR-INSTANCE.grafana.net/api/datasources" | \ | ||
| jq '.[] | select(.type=="loki") | {name, uid}' | ||
| ``` | ||
|
|
||
| The toolset can directly connect to a Loki instance without proxying through a Grafana instance. This is done by not setting the `grafana_datasource_uid` field. Not setting this field makes HolmesGPT assume that it is directly connecting to Loki. | ||
| To create a service account token: navigate to "Administration → Service accounts", create a new service account with "Viewer" role, and generate a token (starts with `glsa_`). |
There was a problem hiding this comment.
Token creation instructions appear after the curl that requires the token.
A reader following the steps sequentially would encounter the curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" command before being told how to obtain YOUR_GLSA_TOKEN. Swapping the order — service account creation instructions first, curl command second — removes this catch-22.
✏️ Suggested reorder
=== "Grafana Cloud"
- ```bash
- curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" \
- "https://YOUR-INSTANCE.grafana.net/api/datasources" | \
- jq '.[] | select(.type=="loki") | {name, uid}'
- ```
-
- To create a service account token: navigate to "Administration → Service accounts", create a new service account with "Viewer" role, and generate a token (starts with `glsa_`).
+ To create a service account token: navigate to **Administration → Service accounts**, create a new service account with **Viewer** role, and generate a token (starts with `glsa_`).
+
+ ```bash
+ curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" \
+ "https://YOUR-INSTANCE.grafana.net/api/datasources" | \
+ jq '.[] | select(.type=="loki") | {name, uid}'
+ ```Note on static analysis: Both Gitleaks findings for this block are false positives —
admin:adminis the standard local Grafana default againstlocalhost:3000, andYOUR_GLSA_TOKENis a documentation placeholder, not a real credential.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ```bash | |
| curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" \ | |
| "https://YOUR-INSTANCE.grafana.net/api/datasources" | \ | |
| jq '.[] | select(.type=="loki") | {name, uid}' | |
| ``` | |
| The toolset can directly connect to a Loki instance without proxying through a Grafana instance. This is done by not setting the `grafana_datasource_uid` field. Not setting this field makes HolmesGPT assume that it is directly connecting to Loki. | |
| To create a service account token: navigate to "Administration → Service accounts", create a new service account with "Viewer" role, and generate a token (starts with `glsa_`). | |
| To create a service account token: navigate to **Administration → Service accounts**, create a new service account with **Viewer** role, and generate a token (starts with `glsa_`). |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/data-sources/builtin-toolsets/grafanaloki.md` around lines 43 - 49, The
documentation shows the curl command that requires YOUR_GLSA_TOKEN before the
instructions that explain how to obtain that token, which can confuse readers;
move the sentence starting "To create a service account token: navigate to
\"Administration → Service accounts\", create a new service account with
\"Viewer\" role, and generate a token (starts with `glsa_`)." so it appears
immediately before the curl block (i.e., place the service account creation
instructions above the curl command that calls "curl -H \"Authorization: Bearer
YOUR_GLSA_TOKEN\" ..."), preserving wording and formatting (bolding if present)
and keeping the curl code block unchanged.
The code automatically adds Bearer auth from api_key, so no need for manual additional_headers with Basic auth. https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK Signed-off-by: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/data-sources/builtin-toolsets/grafanaloki.md`:
- Around line 106-120: Update the Grafana Cloud Loki docs for the grafana/loki
toolset to require and document the tenant ID and Access Policy token (with
logs:read scope) instead of only api_key: explain that Grafana Cloud requires
either HTTP Basic Auth (username=tenant_id, password=access_policy_token) or an
Authorization: Bearer header in the form TENANT_ID:TOKEN, and add a new config
field suggestion tenant_id (or show using additional_headers with Authorization:
Basic <base64 tenant:token>) alongside api_key/access_policy_token; clarify that
glsa_* service account tokens are not acceptable and update the example config
under toolsets.grafana/loki to demonstrate the correct fields and authentication
method for direct Loki endpoints.
---
Duplicate comments:
In `@docs/data-sources/builtin-toolsets/grafanaloki.md`:
- Around line 43-49: Move the service account token creation sentence so it
appears before the curl command that references YOUR_GLSA_TOKEN; specifically,
place the sentence "To create a service account token: navigate to
'Administration → Service accounts', create a new service account with 'Viewer'
role, and generate a token (starts with `glsa_`)." immediately above the ```bash
curl -H "Authorization: Bearer YOUR_GLSA_TOKEN" ...``` block so readers know how
to obtain the token before the curl example uses it.
| === "Grafana Cloud Loki" | ||
|
|
||
| Find your Loki URL in Grafana Cloud under "My Account → Loki": | ||
|
|
||
| ```yaml-toolset-config | ||
| toolsets: | ||
| grafana/loki: | ||
| enabled: true | ||
| config: | ||
| api_url: https://logs-prod-XXX.grafana.net | ||
| api_key: <your Grafana Cloud API key> | ||
|
|
||
| kubernetes/logs: | ||
| enabled: false # HolmesGPT's default logging mechanism MUST be disabled | ||
| ``` |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "common.py" -path "*/grafana/*" | head -20Repository: HolmesGPT/holmesgpt
Length of output: 106
🏁 Script executed:
rg -n "api_key|Bearer|basic_auth|Authorization" --type py -A3 -B3 holmes/plugins/toolsets/grafana/common.pyRepository: HolmesGPT/holmesgpt
Length of output: 1044
🏁 Script executed:
find . -name "*.py" -path "*/loki*" -o -name "*loki*" -type f | head -20Repository: HolmesGPT/holmesgpt
Length of output: 392
🏁 Script executed:
rg -i "grafana.*loki|loki.*cloud|basic.*auth|instance.*id" --type py -A2 -B2 | head -100Repository: HolmesGPT/holmesgpt
Length of output: 6629
🏁 Script executed:
cat -n holmes/plugins/toolsets/grafana/loki_api.pyRepository: HolmesGPT/holmesgpt
Length of output: 3362
🌐 Web query:
Grafana Cloud Loki authentication API key bearer token
💡 Result:
For Grafana Cloud Loki (logs), the “API key” you create is typically a Grafana Cloud Access Policy token. How you pass it depends on which endpoint you’re calling:
1) Calling Loki (read/write logs): use tenant/user + token
Grafana Cloud Loki expects your Logs tenant ID (a.k.a. logs user / tenant) plus the token.
Option A — HTTP Basic Auth (most common)
- Username: your logs tenant ID
- Password: your Access Policy token (Grafana Cloud token) (grafana.com)
Option B — Authorization: Bearer <user>:<token> (common for push examples)
Grafana’s own curl example for pushing logs uses:
Authorization: Bearer <LOGS_USERID>:<API_KEY> (grafana.com)
Example (push):
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <LOGS_TENANT_ID>:<ACCESS_POLICY_TOKEN>" \
-d '{"streams":[{"stream":{"source":"curl"},"values":[["'"$(($(date +%s)*1000000000))"'","hello"]]}]}' \
https://<your-loki-endpoint>/loki/api/v1/push2) Calling the Grafana Cloud API (not Loki): use Bearer <token>
For Grafana Cloud API requests, you pass the Access Policy token as:
Authorization: Bearer <CLOUD ACCESS POLICY TOKEN> (grafana.com)
If you tell me whether you’re pushing logs (/loki/api/v1/push) or querying (/loki/api/v1/query / query_range), and paste your Loki endpoint host pattern (redact IDs), I can give you the exact curl for your case.
Citations:
- 1: https://grafana.com/docs/grafana-cloud/account-management/authentication-and-permissions/access-policies/using-an-access-policy-token/?pg=intro-to-prometheus-and-grafana&utm_source=openai
- 2: https://grafana.com/blog/how-to-use-http-apis-to-send-metrics-and-logs-to-grafana-cloud/?utm_source=openai
- 3: https://grafana.com/blog/how-to-use-http-apis-to-send-metrics-and-logs-to-grafana-cloud/?utm_source=openai
- 4: https://grafana.com/docs/grafana-cloud/account-management/authentication-and-permissions/access-policies/using-an-access-policy-token/?pg=intro-to-prometheus-and-grafana&utm_source=openai
🌐 Web query:
Grafana Cloud Loki query_range authentication tenant ID logs user
💡 Result:
To call Loki’s GET /loki/api/v1/query_range against Grafana Cloud Logs (Loki) you typically need (a) the logs tenant ID / logs userID and (b) a token (Cloud Access Policy token) with logs:read.
1) Endpoint
Use your stack’s Loki “Logs endpoint” (shown in the Grafana Cloud Portal). The range-query path is: .../loki/api/v1/query_range. [1]
2) Authentication (Grafana Cloud)
Grafana Cloud’s recommended setup is HTTP Basic Auth:
- Username = log tenant ID (often shown as “Logs user” / “UserID” in the Cloud Portal)
- Password = Grafana Cloud token (Cloud Access Policy token) [2]
3) Tenant / Org header (X-Scope-OrgID)
Loki is multi-tenant and, in general, identifies the tenant with the X-Scope-OrgID header. [3]
In Grafana Cloud, the fronting gateway commonly derives tenant from your credentials (so you often don’t need to manually add X-Scope-OrgID if you’re using the Cloud endpoint + basic auth), but if you’re hitting Loki behind your own proxy or a custom deployment, you may need to set it explicitly. [3]
4) Example curl (range query)
curl -u "$LOKI_TENANT_ID:$GRAFANA_CLOUD_TOKEN" \
"https://<your-logs-endpoint>/loki/api/v1/query_range?query=%7Bjob%3D%22api%22%7D&start=1700000000000000000&end=1700003600000000000&limit=1000"Common failure modes
- Using a token without
logs:readscope. [2] - Using the wrong “user/tenant ID” (must be the logs tenant ID, not your email/login). [2]
- Missing/incorrect
X-Scope-OrgIDwhen talking to Loki without a gateway that injects it. [3]
Sources: [1] [2] [3]
🏁 Script executed:
cat -n holmes/plugins/toolsets/grafana/common.py | head -60Repository: HolmesGPT/holmesgpt
Length of output: 2498
🏁 Script executed:
cat -n holmes/plugins/toolsets/grafana/loki/toolset_grafana_loki.py | head -100Repository: HolmesGPT/holmesgpt
Length of output: 4074
The Grafana Cloud Loki configuration example is incomplete for direct-access authentication.
Grafana Cloud Loki requires both the logs tenant ID (numeric user ID) and an Access Policy token with logs:read scope. Official Grafana documentation specifies HTTP Basic Auth (username=tenant ID, password=token) or Authorization: Bearer <TENANT_ID>:<TOKEN>.
The current code implements only Authorization: Bearer <api_key>, omitting the tenant ID. This will fail for Grafana Cloud Loki direct access (logs-prod-XXX.grafana.net/loki/api/v1/...).
The configuration should:
- Clarify that a Grafana Cloud Access Policy token (not a service account token like
glsa_*) is required withlogs:readscope. - Document that the logs tenant/user ID must be provided—either via a new
tenant_idconfig field or throughadditional_headersusing HTTP Basic Auth (Authorization: Basic <base64-encoded-tenant:token>). - Update the example to show how users can configure direct Loki access with proper authentication.
Without this, users following the current example with a direct Loki endpoint will encounter authentication errors.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/data-sources/builtin-toolsets/grafanaloki.md` around lines 106 - 120,
Update the Grafana Cloud Loki docs for the grafana/loki toolset to require and
document the tenant ID and Access Policy token (with logs:read scope) instead of
only api_key: explain that Grafana Cloud requires either HTTP Basic Auth
(username=tenant_id, password=access_policy_token) or an Authorization: Bearer
header in the form TENANT_ID:TOKEN, and add a new config field suggestion
tenant_id (or show using additional_headers with Authorization: Basic <base64
tenant:token>) alongside api_key/access_policy_token; clarify that glsa_*
service account tokens are not acceptable and update the example config under
toolsets.grafana/loki to demonstrate the correct fields and authentication
method for direct Loki endpoints.
- Move Grafana Cloud service account creation instructions before the curl command that references the token - Remove Capabilities table per docs guidelines https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK Signed-off-by: Claude <noreply@anthropic.com>
This branch has not been deployed
2 participants
Add dedicated Grafana Cloud section to Loki docs with step-by-step
instructions for service account setup, datasource UID discovery, and
common mistake warnings. Fix trailing slash bug in get_base_url() that
produced double-slash URLs (e.g. https://x.grafana.net//api/...) causing
404 errors. Also add direct Grafana Cloud Loki access method and correct
the capabilities table tool name.
https://claude.ai/code/session_01CcT3RF7eByFp1W1mtdbXyK
Signed-off-by: Claude noreply@anthropic.com
Summary by CodeRabbit
Documentation
Bug Fixes