Skip to content

fix(mcp): classify the /reload-mcp diff against config, not live sockets - #80794

Closed
0xGr1mm wants to merge 1 commit into
NousResearch:mainfrom
0xGr1mm:fix/reload-mcp-diff-reads-config
Closed

0xGr1mm wants to merge 1 commit into
NousResearch:mainfrom
0xGr1mm:fix/reload-mcp-diff-reads-config

Conversation

@0xGr1mm

@0xGr1mm 0xGr1mm commented Aug 7, 2026 •

Copy link
Copy Markdown

What does this PR do?

/reload-mcp built its diff purely from the live connection registry on either side of the reconnect:

added = connected_servers - old_servers
removed = old_servers - connected_servers
reconnected = connected_servers & old_servers

So removed answers "was connected, isn't now" — which is not what the label claims. Any server still in config.yaml but not connected at the instant the diff runs is reported as ➖ Removed: a connect still in flight past mcp_discovery_timeout (1.5s by default), a server set enabled: false, or one that failed this round.

On the reported shape:

♻️ Reconnected: docs, search
➖ Removed: calendar, maps      ← both still configured, both connect
                                 seconds later via the late-binding refresh

The label is not the only casualty. The same set is injected into conversation history as Removed servers: calendar, maps, so the model is told its tools were deleted while they are on their way in.

The fix

classify_reload_diff() in tools/mcp_tool.py owns the classification for both surfaces:

  • removed now means no longer in the config.
  • A configured server that simply is not connected yet comes back separately as not_connected.
  • _load_mcp_config deliberately does not apply the enabled filter, so a server switched off in config stays out of removed too — matching what the label promises.
  • A config read that fails degrades to the old connection-only answer rather than raising through a reload path.

Old vs new on the reported inputs (before={docs,search,calendar,maps}, after={docs,search}, all four configured):

OLD  removed : ['calendar', 'maps']
NEW  removed : []
NEW  not_connected: ['calendar', 'maps']

injected into history:
  OLD -> "Removed servers: calendar, maps"
  NEW -> (removed is empty — the sentence is not produced)

Related Issue

Refs #80771 (P2, comp/cli, tool/mcp, area/config). Searched open and merged PRs first, per CONTRIBUTING's search-first section:

gh search prs --repo NousResearch/hermes-agent "80771"                  --limit 10
gh search prs --repo NousResearch/hermes-agent "reload-mcp in:title"    --limit 20

Update: #80855 (open) also fixes #80771 and is the broader of the two. It adds summarize_mcp_reload() in tools/mcp_tool.py, wires it into cli.py and gateway/run.py, and adds the gateway.reload_mcp.pending key across all 17 locale catalogs, which is exactly the gateway i18n this PR's reviewer note leaves out by design. A reviewer should see both before picking one; if #80855 is preferred, this PR can be closed in its favour.

Nothing else addresses the diff's source of truth. The other open /reload-mcp PRs are about different things: #77074 / #77643 point mcp add success messages at the command, #72864 reloads args on config changes, #46520 restarts the MCP loop. The issue also distinguishes itself from #78426, where the servers genuinely failed and Removed was the correct label.

Type of Change

Bug fix (non-breaking).

Changes Made

  • tools/mcp_tool.py — new classify_reload_diff(before, after) returning added / removed / reconnected / not_connected, with the config read as the source of truth for "removed" and a documented fallback.
  • cli.py — _reload_mcp classifies through the helper; gains an explicit ⏳ Still configured, not connected: line. The history injection a few lines below reads the same corrected removed, so it is fixed by the same change.
  • gateway/run.py — the /reload-mcp handler had the identical expression; moved onto the helper.
  • tests/tools/test_mcp_reload_diff.py — new, 6 tests.

How to Test

pytest tests/tools/test_mcp_reload_diff.py -q

6 passed: the reported shape, a genuinely deleted server, a disabled-but-configured server, a fresh add, bucket disjointness/coverage, and the config-unreadable fallback.

pytest tests/tools/ -q -k mcp

450 passed — the existing MCP suite is unaffected.

Because the classifier is new code rather than an edited branch, the before/after proof is the direct comparison above: the pre-fix expression (copied verbatim from cli.py and gateway/run.py) against the shipped helper on the issue's own inputs.

Regression checks across all three touched surfaces:

tests/gateway/ + tests/hermes_cli/  -k "reload or mcp"   149 passed, 1 skipped

Full tests/tools/ suite, this branch vs main, same environment, run serially:

main:   71 failed, 5473 passed, 25 skipped, 7 deselected
branch: 71 failed, 5479 passed, 25 skipped, 7 deselected

Comparing the failing sets rather than the counts: zero regressions — the two sets are identical, nothing added and nothing removed. The +6 is this PR's new tests. Those 71 are pre-existing in a non-hermetic single-process run (that suite carries browser/docker/network-dependent cases); CI's per-file isolation via run_tests_parallel.py is the supported path.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(mcp): …)
  • I searched for existing PRs to make sure this isn't a duplicate (queries above)
  • My PR contains only changes related to this fix
  • I've run the affected suites and all tests pass
  • I've added tests for my changes
  • I've tested on my platform: macOS 15 (Darwin 25.5), Python 3.11

Documentation & Housekeeping

  • I've updated relevant documentation — the helper's docstring records what each bucket means and why the config, not the socket, decides removed
  • cli-config.yaml.example — N/A, no config keys
  • CONTRIBUTING.md / AGENTS.md — N/A
  • Cross-platform impact — N/A, set arithmetic
  • Tool descriptions/schemas — N/A

Notes for the reviewer

One deliberate asymmetry: the CLI prints raw strings, so it gained the Still configured, not connected line directly. The gateway renders through t(), so surfacing that set there needs a new i18n key across locales — I left it out rather than add translations as a side effect of a bug fix. The gateway's false Removed claim is fixed either way, in both its output and its injected history; if you want the extra line there too, say so and I will add the key.

@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery tool/mcp MCP client and OAuth area/config Config system, migrations, profiles P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 7, 2026
@spfcraze

spfcraze commented Aug 8, 2026

Copy link
Copy Markdown

This was generated by AI during triage.

Summary:
The description's search-first note is now outdated: open PR #80855 addresses the same #80771 /reload-mcp mislabel this PR fixes.

Problems:

Solution:
Add #80855 to the description's related-PR note so a reviewer sees both open fixes for #80771.


Checked against c452387 — the tip of fix/reload-mcp-diff-reads-config when this was written.

`/reload-mcp` built its added/removed/reconnected sets purely from the live
connection registry on either side of the reconnect:

    added = connected_servers - old_servers
    removed = old_servers - connected_servers
    reconnected = connected_servers & old_servers

So "removed" answered "was connected, isn't now" — not what the label
claims. Any server still in `config.yaml` but not connected at the instant
the diff runs is reported as `➖ Removed`: a connect still in flight past
`mcp_discovery_timeout` (1.5s by default), a server set `enabled: false`, or
one that failed this round. On the reported shape:

    ♻️ Reconnected: docs, search
    ➖ Removed: calendar, maps      <- both still configured, both connect
                                      seconds later via late binding

The label is not the only casualty. The same set is injected into
conversation history ("Removed servers: calendar, maps"), so the model is
told its tools were deleted while they are on their way in.

`classify_reload_diff()` in tools/mcp_tool.py owns the classification for
both surfaces: `removed` is now "no longer in the config", and a configured
server that simply is not connected yet comes back separately as
`not_connected`. `_load_mcp_config` deliberately does not apply the
`enabled` filter, so a server switched off in config stays out of `removed`
too — matching what the label promises. A config read that fails degrades to
the old connection-only answer rather than raising through a reload path.

Both call sites move onto it — the CLI (`_reload_mcp`) and the gateway
(`/reload-mcp` handler), which had the identical expression. The CLI gains an
explicit "Still configured, not connected" line; the gateway renders through
i18n, so reporting that set there needs a new translation key and is left
out rather than smuggled in. Its false `Removed` claim — the actual bug, in
output and in injected history alike — is fixed either way.

Tests cover the reported shape, a genuinely deleted server, a disabled one,
a fresh add, bucket disjointness/coverage, and the config-unreadable
fallback.

Refs NousResearch#80771

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: /reload-mcp reports still-configured servers as "➖ Removed" — the diff never reads config

3 participants