Skip to content

fix(cli): accept plugin platform bundle entries in platform_toolsets validation - #107104

Closed
kokhlo wants to merge 1 commit into
NousResearch:mainfrom
kokhlo:fix-107098-plugin-bundle-toolsets
Closed

kokhlo wants to merge 1 commit into
NousResearch:mainfrom
kokhlo:fix-107098-plugin-bundle-toolsets

Conversation

@kokhlo

@kokhlo kokhlo commented Sep 10, 2026

Copy link
Copy Markdown

Summary

platform_toolsets entries naming a plugin platform's synthesized bundle (teams: [hermes-teams] — verbatim from the shipped cli-config.yaml.example) were reported as unknown toolsets, followed by a claim that the platform will have no tools. Both statements are false: the bundle resolves to the full platform toolset via toolsets.resolve_toolset() → _plugin_platform_bundle().

Root cause: the two validation paths in hermes_cli/toolset_validation.py disagree about plugin platforms. _platform_default_is_valid() (used by the null/scalar fallback branch) consults gateway.platform_registry.is_registered(platform), but the per-entry list loop only called the injected is_valid_toolset — which resolves to toolsets.validate_toolset(), a predicate that never knows about synthesized hermes-<platform> bundles. The same string was "valid" as a platform default and "unknown" as an explicit entry, and the "did you mean" hint could echo the exact string it just declared unknown.

Fix: in the list branch, treat an entry equal to the platform's own default as valid when default_valid already holds — the same source of truth the null/scalar branch trusts (the platform registry for plugin platforms). Genuinely unknown names on the same list still warn, with the bundle as a now-correct suggestion; the false "no tools" claim disappears for bundle entries. Registry-unaware platforms keep the old behavior entirely (typo case unchanged).

Testing

  • env -u HERMES_YOLO_MODE .venv/bin/python -m pytest tests/hermes_cli/test_toolset_validation.py -q → 15 passed (12 pre-existing + 3 new).
  • New regression tests: explicit bundle entry is valid and produces zero warnings; bundle alongside a genuinely unknown name still warns for the unknown one without the "no tools" claim; unregistered platform keeps the old unknown-toolset warning and the zero-valid safety net.
  • Mutation check: disabling the new branch (if False and ...) turns the first two new tests red.
  • ruff check clean; ruff format --diff complaint count equals the pristine-main baseline (77 = 77).

Fixes #107098

@kokhlo

kokhlo commented Sep 10, 2026

Copy link
Copy Markdown
Author

Note for maintainers on CI status: all three workflows (CI, Docker, Nix) on this PR completed with action_required and zero jobs — the fork-PR approval gate that hit every external contributor in this window (27 runs across 7 forks between 04:49Z and 04:55Z, while internal branches ran normally). It is not a diff failure and cannot be re-triggered from my side.

Local gates, run against fresh upstream main cfdbbb6:

  • python -m pytest tests/hermes_cli/test_toolset_validation.py -q → 15 passed (12 pre-existing + 3 new; mutation check confirms the new tests go red when the fix branch is disabled)
  • ruff check clean on both touched files; ruff format --diff complaint count identical to pristine main baseline (77 = 77)

Happy to rebase or adjust once someone with permissions re-runs the workflows.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/plugins Plugin system and bundled plugins area/config Config system, migrations, profiles labels Sep 10, 2026
@flysky8072-hue

Copy link
Copy Markdown

Independent confirmation on Windows, and a before/after on the patch itself.

Environment: Hermes Agent v0.21.1 (2026.9.7), upstream 564aef29, git install on Windows 10 (git-bash), repo mirror cnb.cool/hermesagent-cn/hermes-agent-cn-mirror.

Reproduced with the shipped cli-config.yaml.example block, via hermes update (the migration path, hermes_cli/config.py:1391). Two extra data points beyond the report:

  1. validate_toolset("hermes-teams") stays False after discover_plugins(), while resolve_toolset("hermes-teams") returns 59 tools (core = 59). toolset_allowed_for_platform("hermes-teams", "teams") is also True, so validate_toolset() is the only predicate that disagrees — the patch's shortcut (counting the bundle entry valid without re-running the scope check) is equivalent to calling it for every plugin platform reachable this way (teams, google_chat, irc, line, ntfy, photon, raft, simplex, a2a, buzz).

  2. Applied the PR's name == default and default_valid guard to a copy of hermes_cli/toolset_validation.py and ran it against the real toolsets + gateway.platform_registry (patched module loaded from a temp dir; no checkout mutated). validate_platform_toolsets(cfg, validate_toolset, toolset_allowed_for_platform):

platform_toolsets input before after
{teams: [hermes-teams], google_chat: [hermes-google_chat]} 5 warnings (incl. the global "zero valid toolsets" line) 0
{teams: [hermes-teams, bogus]} 4 1 — unknown toolset 'bogus', hint now points at a valid name
{notaplugin: [hermes-notaplugin]} 3 3 (unchanged — safety net intact)
real config, teams + google_chat restored 4 0

hermes-feishu still resolves 64 tools and the live gateway stays connected, so no regression in the toolset paths the fix does not touch.

@kshitijk4poor

Copy link
Copy Markdown

Thanks @kokhlo. You were the first to fix this: Merged in #127782 (16c59d0 on main). hermes config migrate / hermes update and hermes config check now judge saved platform_toolsets with one shared resolver, hermes_cli.toolset_validation.saved_toolset_resolver. It accepts a plugin platform's hermes-<platform> bundle.

Verified on 16c59d0 through the real migrate path, platform_toolsets: {teams: [hermes-teams], google_chat: [hermes-google_chat]}:

  • before: 5 warnings, including the self-referential hint and the "no tools" safety nets
  • after: none

A name that isn't a platform (hermes-notaplugin) is still reported. Closing as resolved. Your PR predates the landed fix, and I've credited you in #127782's description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: platform_toolsets warns "unknown toolset hermes-<platform>" for valid plugin platforms (list-entry check ignores the platform registry)

4 participants