Skip to content

hermes config check reports stale toolset names and disabled platforms the way the runtime sees them (salvage #127392) - #127782

Merged
kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/config-check-stale-selections-127392
Sep 29, 2026
Merged

kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/config-check-stale-selections-127392

Conversation

@kshitijk4poor

@kshitijk4poor kshitijk4poor commented Sep 29, 2026 •

Copy link
Copy Markdown

hermes config check now reports saved platform_toolsets names that no longer resolve, and bundled platform plugins that are disabled while their required credentials are configured. It judges both the way the runtime does.

Salvage of #127392 by @helix4u. Their commit is cherry-picked, so authorship is preserved. Two follow-up commits fix what review found before landing.

Changes

  • Stale toolset names: config check and hermes config migrate now share one resolver, toolset_validation.saved_toolset_resolver. Before this, migrate warned about MCP server names and plugin-platform bundles (it even suggested 'hermes-teams' for 'hermes-teams'). The new check accepts them.
    • A configured MCP server counts as known even when it has enabled: false. It is inactive, not a typo.
    • known_plugin_toolsets no longer counts as proof that a toolset exists. It records what a platform was offered at save time, so it hid toolsets from uninstalled plugins, which is exactly the stale case this check exists for.
    • Plugin-provided names and the platform-manifest scan are only consulted for names the cheaper checks cannot place. A clean config never runs plugin discovery.
  • Disabled platforms: plugins.disabled is read through the runtime's own reader. A non-list value, which the runtime ignores, is no longer reported as disabling a platform. Required credentials come from _platform_manifest_env_entries(optional=False), so optional vars never count.
  • Crash fix: a malformed mcp_servers (for example a list) no longer crashes the command.
  • Dropped check: the original PR also checked the top-level toolsets key. The CLI resolves toolsets from --toolsets or platform_toolsets and reads the top-level key only for the legacy kanban opt-in, so that check is gone.
  • Not included: the original PR's Desktop fixture-shebang commit. It is byte-identical to Python lints lane green again: portable bash shebang in the remote-lifecycle test fixture #127265, which is still open.

Validation

Case (real hermes config check, temp HERMES_HOME) Original #127392 This PR
platform_toolsets.cli lists a removed name reported reported
toolset of an uninstalled plugin, remembered in known_plugin_toolsets silent reported
MCP server with enabled: false listed in platform_toolsets "unknown toolset … did you mean 'hermes-cli'?" silent
mcp_servers: [a] AttributeError traceback runs
plugins.disabled: platforms/telegram (string) with token "disabled" (runtime still loads it) silent
plugins.disabled: [platforms/telegram] with token reported reported
fresh config silent silent, no plugin discovery
  • Tests: 2 invariant tests in tests/hermes_cli/test_config_check_diagnostics.py, red on the original commit and green here.
    • Test 1 asserts that config check and migrate report the same names across A→B→A homes. Mutating migrate back to bare validate_toolset turns it red.
    • Test 2 uses a synthetic platform manifest rather than Telegram's.
  • The 34 test files that reference platform_toolsets, _platform_manifest_env_entries or migrate_config give identical results on this branch and on origin/main: 852 passed, 17 failed and 8 errors on both, and the failures are the same set.
  • ruff check on the touched files, and the Windows footgun checker, are clean.

Known limits

  • platform_toolsets.cli: [no_mcp] on its own is still not reported. validate_platform_toolsets counts no_mcp as a valid toolset.
  • When the config names something the cheaper checks cannot place, migrate now runs plugin discovery. It stays inside the existing best-effort try/except and under the profile's own plugin scope.

Credit (post-merge sweep)

Earlier PRs fixed parts of the migrate-side false positives that this resolver now covers. They were closed as resolved, with credit:

helix4u and others added 3 commits September 29, 2026 17:04
…heck

Follow-up to the config check diagnostics from NousResearch#127392.

- Saved platform_toolsets are validated by one resolver,
  toolset_validation.saved_toolset_resolver, shared with
  `hermes config migrate`. The two commands previously disagreed: migrate
  flagged MCP server names and plugin-platform bundles (even suggesting
  'hermes-teams' for 'hermes-teams'), while config check accepted them.
- A configured MCP server is known even when disabled (enabled: false);
  it is inactive, not a typo.
- known_plugin_toolsets is no longer treated as proof that a toolset
  exists. It records what a platform was offered at save time, so it hid
  toolsets of uninstalled plugins, the stale case this check is for.
- Plugin-provided names are consulted only for names nothing else
  explains, so a clean config no longer runs plugin discovery.
- A malformed mcp_servers (a list) no longer crashes the command.
- The top-level `toolsets` check is dropped: the CLI resolves its
  toolsets from --toolsets or platform_toolsets, and reads the top-level
  key only for the legacy kanban opt-in.
- Disabled platforms are read with the runtime's plugins.disabled reader,
  so a non-list value (which the runtime ignores) is no longer reported
  as disabling a platform. Required credentials come from
  _platform_manifest_env_entries, which now takes optional=False.
- Tests use a synthetic platform manifest instead of the real Telegram
  one, and assert config check and migrate report the same names.
…nnot place

saved_toolset_resolver built the hermes-<platform> bundle set eagerly, so
every migrate_config run (hermes update, config migrate, doctor) and every
config check paid a full bundled+user manifest scan even with no saved
platform_toolsets. The scan is now cached and lazy, like the plugin lookup.
The stale-toolset test stubs the plugin lookups so host-installed plugins
cannot change its verdict.
@whyyagswhy

Copy link
Copy Markdown

Independent verification on the PR head (f64add8): config-check-diagnostics suite 2/2 green on Linux. No findings.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets area/config Config system, migrations, profiles sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Sep 29, 2026
@kshitijk4poor
kshitijk4poor merged commit 16c59d0 into NousResearch:main Sep 29, 2026
62 checks passed
@francip

francip commented Sep 29, 2026

Copy link
Copy Markdown

@kshitijk4poor does this supersede #86233 or was that left open for other reasons?

kshitijk4poor pushed a commit to kshitijk4poor/hermes-agent that referenced this pull request Oct 3, 2026
MCP tools register under the toolset name mcp-<server>, an alias that only
exists after MCP discovery. `hermes config migrate`, `hermes update` and
`hermes config check` run without discovery, so a platform_toolsets entry
such as mcp-codegraph for a configured server was reported as an unknown
toolset with a 'hermes-cli' hint (NousResearch#78102).

saved_toolset_resolver now accepts the prefixed form of every configured
and plugin-provided (portable) MCP server. An mcp-<name> with no such
server is still reported.

Rule from NousResearch#106715 by @a692570, adapted onto the shared resolver that
landed in NousResearch#127782.
kshitijk4poor pushed a commit that referenced this pull request Oct 3, 2026
MCP tools register under the toolset name mcp-<server>, an alias that only
exists after MCP discovery. `hermes config migrate`, `hermes update` and
`hermes config check` run without discovery, so a platform_toolsets entry
such as mcp-codegraph for a configured server was reported as an unknown
toolset with a 'hermes-cli' hint (#78102).

saved_toolset_resolver now accepts the prefixed form of every configured
and plugin-provided (portable) MCP server. An mcp-<name> with no such
server is still reported.

Rule from #106715 by @a692570, adapted onto the shared resolver that
landed in #127782.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants