fix(cli): accept configured MCP prefixed toolsets before discovery

[thedavidweng]

What does this PR do?

Fix the same class of false-positive MCP toolset validation across CLI, TUI, and oneshot (-z) startup paths.

Current main already suppresses the warning for bare MCP server names (agentmail, notion, etc.), but it still warns for the documented mcp-<server> spelling (mcp-agentmail, mcp-notion, etc.) before MCP discovery registers runtime toolsets. That causes startup warnings even when the user has configured MCP servers correctly in config.yaml.

This PR adds shared pre-discovery MCP toolset helpers and applies them consistently to the three startup validators so mcp-<server> is accepted when the underlying MCP server is configured (enabled or disabled), while genuinely unknown toolsets still warn.

Fixes #78102

Related Issue

Related #41625
Related #23997
Related #29532
Related #5279

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• Added shared MCP toolset helpers:
  • hermes_cli/mcp_toolsets.py
• Updated CLI startup validator to accept configured MCP prefixed toolsets before discovery:
  • cli.py
• Updated -z / oneshot validator to accept configured MCP prefixed toolsets before discovery:
  • hermes_cli/oneshot.py
• Updated TUI toolset loader to accept configured MCP prefixed toolsets before discovery:
  • tui_gateway/server.py
• Added regression tests:
  • tests/cli/test_mcp_toolset_validation.py
  • tests/hermes_cli/test_tui_resume_flow.py
  • tests/test_tui_gateway_server.py

How to Test

1. Configure mcp_servers in ~/.hermes/config.yaml with one or more servers.
2. Use platform_toolsets entries such as:
   • mcp-agentmail
   • mcp-notion
   • mcp-notion-findit
   • mcp-tinyfish
3. Start Hermes CLI / TUI / oneshot (hermes -z) and confirm no false:
   Warning: Unknown toolsets: mcp-agentmail, mcp-notion, ...
4. Run tests:
   • venv/bin/python -m pytest tests/cli/test_mcp_toolset_validation.py -q
   • venv/bin/python -m pytest tests/hermes_cli/test_tui_resume_flow.py -q
   • venv/bin/python -m pytest tests/test_tui_gateway_server.py -q
5. Run broader MCP-related tests:
   • venv/bin/python -m pytest tests/tools/test_mcp*.py -q

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run targeted pytest passes for the affected areas
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 26.5.1

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

venv/bin/python -m pytest tests/cli/test_mcp_toolset_validation.py -q
4 passed in 1.52s

venv/bin/python -m pytest tests/hermes_cli/test_tui_resume_flow.py::test_oneshot_accepts_configured_mcp_prefixed_toolset_before_discovery tests/hermes_cli/test_tui_resume_flow.py::test_oneshot_rejects_disabled_mcp_toolset tests/hermes_cli/test_tui_resume_flow.py::test_oneshot_distinguishes_disabled_mcp_from_unknown -q
3 passed in 0.24s

venv/bin/python -m pytest tests/test_tui_gateway_server.py::test_load_enabled_toolsets_accepts_configured_mcp_prefixed_env_before_discovery tests/test_tui_gateway_server.py::test_load_enabled_toolsets_rejects_disabled_mcp_env tests/test_tui_gateway_server.py::test_load_enabled_toolsets_reports_disabled_mcp_separately tests/test_tui_gateway_server.py::test_load_enabled_toolsets_folds_project_into_focus_posture -q
4 passed in 0.90s

[ahmadashfq]

Confirming this reproduces in a real-world multi-profile deployment on main (Hermes v0.16.0).

We run a fleet of Hermes profiles where CodeGraph's installer added mcp-codegraph under platform_toolsets.cli (its documented spelling). Every CLI/worker session for those profiles prints:

Warning: Unknown toolsets: mcp-codegraph

even though the server is configured correctly and fully functional — hermes mcp test codegraph connects and discovers all 8 tools. So it's purely the pre-discovery static validator flagging the mcp-<server> spelling, exactly as described here. It's especially noisy because it lands at the top of every kanban worker run log across many boards.

This PR's approach — accepting both <server> and mcp-<server> spellings for configured servers before discovery, across the CLI/oneshot/TUI paths — matches the actual alias registration in tools/mcp_tool.py (toolset_name = f"mcp-{name}" + register_toolset_alias) and fixes it at the right layer. +1 to getting this reviewed.

[teknium1]

Thanks for covering the three pre-discovery validators. The premise is confirmed on current main: cli.py:3887-3890 excludes only the bare configured server key, while MCP registration creates mcp-<server> at tools/mcp_tool.py:4750 and the docs endorse that spelling at website/docs/reference/toolsets-reference.md:124-134.

Problems

• hermes_cli/mcp_toolsets.py:53 aliases non-mapping server values. Existing oneshot/TUI validation skips them, and runtime loading only retains dict entries in tools/mcp_tool.py:3773-3776; this can validate a toolset that discovery cannot register.

Suggested changes

• Skip non-mapping server entries in the shared helper and cover that malformed-config case with a focused test.

The PR is currently conflicting, but the current CLI, oneshot, and TUI code still contains the verified bug class, so this otherwise looks mechanically salvageable.

Automated hermes-sweeper review.

[thedavidweng]

Update after rebase + teknium1 review

Rebased onto latest main (resolved 2 conflict in test files, kept both sides).

Addressed teknium1's review:

• hermes_cli/mcp_toolsets.py:53: skip non-mapping server entries with if not isinstance(server_cfg, Mapping): continue — prevents validating toolsets that MCP discovery can't register
• Added 2 new tests in tests/cli/test_mcp_toolset_validation.py:
  • test_mcp_toolset_aliases_skips_non_mapping_entries — covers string/int server values
  • test_split_configured_mcp_toolset_aliases_skips_non_mapping — covers both enabled/disabled paths

All targeted tests pass:

• tests/cli/test_mcp_toolset_validation.py: 4 passed
• tests/hermes_cli/test_tui_resume_flow.py: 3 passed
• tests/test_tui_gateway_server.py: 4 passed (incl. conflict-reserved folds_project test)

Pre-existing test failures in tests/tools/test_mcp_reconnect_signal.py and test_mcp_server_log_notifications.py exist on unmodified main — unrelated to this PR.

[thedavidweng]

Rebased onto current main (was 9295 commits behind)

Force-pushed as d3d769f01e. The bug is still present on latest main (same bare-names-only gap), but upstream independently added bare-names MCP handling in all three paths since June, so this rebase adapts rather than duplicates:

• cli.py: same hunk as before (upstream still bare-names-only here) — take the helper call.
• oneshot.py: upstream added _configured_mcp_servers() (bare names). Kept this PR's shared-helper design as the single source of truth and removed the now-unused upstream function instead of carrying two overlapping implementations.
• tui_gateway/server.py: upstream extracted _resolve_explicit_toolsets() with the same bare-names gap. Dropped my stale inline block; the PR now extends that function's MCP section to both spellings via split_configured_mcp_toolset_aliases (also fixes disabled-server prefixed forms reporting as unknown instead of disabled).
• mcp_toolsets.py: dropped the unused canonical_mcp_toolset_name (dead on arrival).
• tests: 2 of my oneshot tests updated for upstream's new _run_agent tuple contract (tuple[str, dict]).

Verification on the rebased commit: test_mcp_toolset_validation + test_tui_resume_flow + test_tui_gateway_server → 661 passed, 1 failed (test_model_options_preserves_canonical_custom_row_after_agent_init, confirmed failing on clean origin/main too — pre-existing, unrelated).