Skip to content

helm: make GitHub MCP health_check_tool configurable per auth type - #2206

Closed
mdecalf wants to merge 3 commits into
HolmesGPT:masterfrom
mdecalf:feat/github-mcp-configurable-health-check-tool
Closed

mdecalf wants to merge 3 commits into
HolmesGPT:masterfrom
mdecalf:feat/github-mcp-configurable-health-check-tool

Conversation

@mdecalf

@mdecalf mdecalf commented Jun 19, 2026 •

Copy link
Copy Markdown

Fixes #2205

Changes

  • GitHub App auth: health_check_tool defaults to "" (health check skipped — App tokens cannot call GET /user)
  • PAT auth: health_check_tool defaults to "get_me" (no change in behaviour)
  • Both: new mcpAddons.github.healthCheckTool values field to override the default

Why

get_me calls GET /user, which requires a user-level OAuth token. GitHub App installation tokens always return 403 Resource not accessible by integration for this endpoint. This causes the github toolset to be permanently marked as failed and all GitHub tools become unavailable to users.

Summary by CodeRabbit

  • New Features
    • Added an override for the GitHub MCP health check tool, with authentication-aware defaults when using a GitHub App versus other authentication methods.
  • Bug Fixes
    • Improved health check selection logic: null now triggers auto-detection, while an empty value cleanly disables the health/auth check; non-empty values directly select the tool.
  • Documentation
    • Added inline explanations in Helm values and templates describing the default behavior and how to override the health check tool.

GitHub App installation tokens cannot call GET /user (returns 403
"Resource not accessible by integration"). The health_check_tool was
hardcoded to "get_me" for both PAT and GitHub App auth, causing the
github toolset to be permanently marked as failed when using App auth.

New behaviour:
- GitHub App auth:   health_check_tool defaults to "" (check skipped)
- PAT auth:          health_check_tool defaults to "get_me" (unchanged)
- Both:              mcpAddons.github.healthCheckTool overrides the default

Closes #<TBD>
@coderabbitai

coderabbitai Bot commented Jun 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ced536a0-cf48-4e69-85fb-bf0bc7e26dd3

📥 Commits

Reviewing files that changed from the base of the PR and between 71ee2ff and d91f5a7.

📒 Files selected for processing (1)
  • holmes/plugins/toolsets/mcp/toolset_mcp.py

Walkthrough

The Helm chart template for the GitHub MCP server now computes health_check_tool conditionally: empty string (disabled) when GitHub App auth is configured, "get_me" otherwise. Both branches allow override via .Values.mcpAddons.github.healthCheckTool, and inline documentation explains the per-auth defaults. The Python backend updates prerequisites_callable to distinguish None (auto-detect) from "" (disable) instead of using truthy-or fallback.

Changes

GitHub MCP health_check_tool per-auth-type configuration

Layer / File(s) Summary
Helm template and values documentation for health_check_tool defaults
helm/holmes/templates/toolset-config.yaml, helm/holmes/values.yaml
Both auth branches now compute $hct with auth-appropriate defaults (empty for GitHub App, "get_me" for PAT) and accept a healthCheckTool override. values.yaml adds comments documenting the per-auth defaults and override behavior.
Python implementation: None vs empty string semantics
holmes/plugins/toolsets/mcp/toolset_mcp.py
prerequisites_callable now explicitly checks for None to trigger auto-detection, treats empty string as disabling the health check, and accepts any other string as a direct tool name—replacing the previous truthy-or fallback pattern.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • HolmesGPT/holmesgpt#434: Introduces RemoteMCPToolset, which is the class whose prerequisites_callable method this PR updates.
  • HolmesGPT/holmesgpt#2118: Originally introduced the hardcoded health_check_tool: "get_me" for both auth variants that this PR replaces with per-auth defaults.

Suggested reviewers

  • Avi-Robusta
  • mainred
  • RoiGlinik
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically summarizes the main objective: making the GitHub MCP health_check_tool configurable per authentication type.
Linked Issues check ✅ Passed The PR fully addresses issue #2205 requirements: implements auth-type-specific defaults (empty string for GitHub App, 'get_me' for PAT), adds configurability via mcpAddons.github.healthCheckTool, and fixes the logic to distinguish None from empty string.
Out of Scope Changes check ✅ Passed All changes are directly related to the linked issue: Helm template updates for health_check_tool configuration, values.yaml documentation, and MCP toolset logic fix for proper None vs empty string handling.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@netlify

netlify Bot commented Jun 19, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for holmes-docs ready!

Name Link
🔨 Latest commit 5da2604
🔍 Latest deploy log https://app.netlify.com/projects/holmes-docs/deploys/6a391bdf1dd2a90008fb1956
😎 Deploy Preview https://deploy-preview-2206--holmes-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@helm/holmes/templates/toolset-config.yaml`:
- Around line 110-119: The `health_check_tool` field in the githubConfig dict is
being set to an empty string as a default to disable health checks, but
downstream logic treats empty strings as falsey and falls back to
auto-detection, defeating the purpose and allowing the system to pick `get_me`
which causes 403 errors. Instead of using an empty string as a sentinel value,
implement an explicit disable mechanism such as a dedicated config flag (for
example `"health_check_enabled" false` or `"health_check_tool" "disabled"`) that
the downstream MCP prerequisite logic can check for to truly disable health
checks instead of auto-detecting when the value is absent or empty.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7c3d8a02-83fd-4b6f-844b-edca75976b64

📥 Commits

Reviewing files that changed from the base of the PR and between 5b9f943 and 71ee2ff.

📒 Files selected for processing (2)
  • helm/holmes/templates/toolset-config.yaml
  • helm/holmes/values.yaml

Comment thread helm/holmes/templates/toolset-config.yaml
mdecalf and others added 2 commits June 22, 2026 09:03
…o-detect

Previously, `health_check_tool or auto_detect()` meant that both None
and "" triggered auto-detection.  An empty string set in the Helm chart
(the intended "skip" sentinel for GitHub App auth) would still cause
auto-detection to pick up get_me — and the resulting GET /user call
returns 403 for App installation tokens, keeping the toolset unhealthy.

Now the logic is:
  - None  → not configured → auto-detect (unchanged default behaviour)
  - ""    → explicitly disabled → skip the health check entirely
  - name  → call that specific tool (unchanged)

This makes the empty-string sentinel in the Helm chart's GitHub App
branch actually work as intended.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: GitHub MCP health_check_tool hardcoded to get_me breaks GitHub App auth (403 Resource not accessible by integration)

1 participant