fix(toolsets): platform plugins without a static bundle silently get zero tools; config set stores list literals as strings - #57063
Conversation
Every bundled platform plugin without a matching hermes-<platform> static bundle silently resolved to ZERO tools outside the gateway process: _get_platform_tools falls back to hermes-<platform> for platforms missing from platform_toolsets, resolve_toolset returns [] for unknown names (NousResearch#38798 shape), and the runtime auto-generate only fires once gateway.platform_registry has the platform registered. The explicit-config composite expansion also skips names absent from TOOLSETS. LINE shipped without a bundle in NousResearch#23197 (zero core edits by design) and ran with zero tools for any deployment that never hand-listed it in platform_toolsets. Same for google_chat, teams, irc, ntfy, photon, simplex, raft. Adds the 8 missing static bundles (core tools, mirroring hermes-telegram/hermes-signal), includes them in the hermes-gateway composite, and adds a regression test that walks plugins/platforms/ so the next platform plugin cannot ship without a bundle.
hermes config set only coerced bool/int/float; a list or mapping literal
(e.g. platform_toolsets.line set to a JSON-style list) was stored as a
raw STRING with no warning. Every reader gated on isinstance(..., list)
— _get_platform_tools, _get_enabled_set, _get_disabled_set — then
silently ignored the value and fell back to its default, so the setting
looked saved but never took effect (observed in the wild: a platform
running on the wrong toolset bundle for weeks, and a plugins.enabled
entry that never enabled anything).
Values starting with '[' or '{' are now parsed with yaml.safe_load;
non-list/dict results and YAML errors warn on stderr and keep the
legacy string behavior.
alt-glitch
added
type/bug
…o the real switch
`hermes plugins enable gemini` printed a green success message and wrote
model-providers/gemini into plugins.enabled — but nothing ever reads that
flag for model providers: the general loader explicitly skips
kind: model-provider (handled by providers/__init__.py's own discovery,
selected via `hermes model` / model.provider) and kind: exclusive
(activated via `<category>.provider`). Same for disable: the entry lands
in plugins.disabled, the loader records it for introspection, and the
provider registers anyway.
The success message misleads users into believing they switched a
provider on or off. Found in the wild: a config with
model-providers/gemini in BOTH plugins.enabled (as a stray string) and
plugins.disabled, while the gemini provider had been registered and
usable the whole time.
enable/disable now detect the manifest kind and print what actually
controls the plugin, changing nothing:
! model-providers/gemini is a model provider — it is not controlled by
plugins.enabled/disabled (providers register automatically at startup).
To use it: run `hermes model` and pick it, or set model.provider.
To stop using it: select a different provider; remove its API key.
Nothing was changed.
…silently dropping tools
A YAML indentation slip nests toolset names under a mapping, e.g.
discord:
- hermes-discord:
- browser
- terminal
which parses to `[{'hermes-discord': ['browser', 'terminal', ...]}]`.
`_get_platform_tools` normalised with `[str(ts) for ts in toolset_names]`,
turning the mapping into the literal string `"{'hermes-discord': [...]}"` —
a name that matches no toolset. `has_explicit_config` stayed False for the
real toolsets and every nested toolset was silently dropped, so the platform
loaded almost no tools (the model could reach cronjob/tts but not
terminal/file/web/browser) while config.yaml looked correct, with no warning
anywhere.
Sibling fixes (NousResearch#38798/NousResearch#52920 invalid names -> zero tools, NousResearch#57063 missing
plugin bundles / list-stored-as-string) don't cover this shape: the result
is non-empty-but-wrong, so their zero-tools guards never fire. Add
`_flatten_toolset_names`, which recovers the intended names (mapping keys +
nested values) so the platform still works and logs a loud warning so the
malformed config is visible and fixable. Well-formed flat lists pass through
unchanged with no warning.
This resolves through the shared resolver, so it fixes CLI, messaging
gateway, and TUI alike.
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing two real configuration failure modes. The list/mapping coercion gap remains on current main (hermes_cli/config.py:8264-8279), but the platform-bundle part needs a narrower current-main repro before salvage.
Problems
- Current platform plugins deliberately resolve dynamically: commit
52d9e578introduced support withouttoolsets.pyentries, andtoolsets.py:731-750supplies core tools for registered platforms. Deferred loaders count as registered (gateway/platform_registry.py:271-276). The new static-bundle invariant (tests/test_toolsets.py:321-328) reverses that design without demonstrating a live path where dynamic registration is unavailable. _plugin_kind()reads rawkindtext (hermes_cli/plugins_cmd.py:835-836), but the loader normalizes and heuristically detects kind-less providers (hermes_cli/plugins.py:1583-1627). Those providers can still bypass the proposed guard.- The new provider hint hardcodes
~/.hermes/.env(hermes_cli/plugins_cmd.py:852), which is not profile-safe.
Suggested changes
- Reproduce and target the exact resolver boundary that loses tools, preserving dynamic plugin-platform support.
- Share the loader's kind classification, and use
display_hermes_home()in the hint.
Automated hermes-sweeper review.
| if not child.is_dir() or not (child / "plugin.yaml").exists(): | ||
| continue | ||
| bundle = f"hermes-{child.name}" | ||
| if bundle not in TOOLSETS or not resolve_toolset( |
There was a problem hiding this comment.
This permanently requires static core bundles for every platform plugin, but current dynamic-platform support was introduced specifically to avoid toolsets.py entries (commit 52d9e57; current toolsets.py:731-750). Please first demonstrate the current-main path where the deferred registry cannot resolve the platform, then constrain the regression to that path rather than reversing the plugin contract.
| import yaml | ||
|
|
||
| data = yaml.safe_load(mf.read_text(encoding="utf-8")) or {} | ||
| return str(data.get("kind", "standalone")) |
There was a problem hiding this comment.
This raw manifest read disagrees with the loader: PluginManager normalizes kind and heuristically classifies kind-less memory/model providers (hermes_cli/plugins.py:1583-1627). Such a provider will still bypass this guard and receive the misleading success path. Reuse the canonical classification or duplicate its normalization and heuristic with coverage.
| " To use it: run [bold]hermes model[/bold] and pick it, or set " | ||
| "[dim]model.provider[/dim] in config.yaml.\n" | ||
| " To stop using it: select a different provider; remove its API key " | ||
| "from ~/.hermes/.env to make it unselectable.\n" |
There was a problem hiding this comment.
Use display_hermes_home() here. A literal ~/.hermes/.env is incorrect for named profiles and custom HERMES_HOME directories.
teknium1
added
sweeper:risk-security-boundary
|
Empirical status check on current main (Aug 5, 2026) — one of the two failure modes no longer reproduces. I ran the exact resolver boundary the hermes-sweeper review asked to narrow down: After plugin discovery (which any CLI/cron/doctor process runs), all eight platforms from the PR's list resolve to the full core toolset via the dynamic auto-generation in What still reproduces, confirmed live:
I'm working a PR that fixes both remaining halves (a2a client tools at discovery + config-set list/dict coercion) with tests. The 8 static bundles part of this PR looks dead on current main — the dynamic generation made it obsolete — so I'd suggest scoping the salvage to the config-set coercion half. |
|
I rebased this PR onto current main and narrowed it to the two halves that are still live. The rebased branch is at What I did:
Verification: 191 tests pass on the rebased branch (config coercion, passive kinds, plugins, toolsets suites). If you'd rather I open this as a fresh PR (with you credited as original author), say the word — or take the branch and force-push it to your own. Goal is the two live fixes landing on current main. |
|
Partial overlap note: PR #86660 (#86660) fixed the deferred-platform client-tool registration class via a |
fix(toolsets): platform plugins without a static bundle silently get zero tools; config set stores list literals as strings Two solid fixes with good tests. Observations:
|
alt-glitch
added
P3
|
Note upfront: this comment was written and posted by an AI agent on behalf of Closing this PR. First, thanks are owed. @cadamec — the empirical status check on Aug 5 was The reason for closing is that this PR bundled several separate problems, and
The takeaway on this end is to split work up front rather than ship a |
5 participants
What does this PR do?
Fixes two silent-failure paths that leave a gateway platform running with zero tools while everything looks configured.
1. Eight bundled platform plugins ship without a
hermes-<platform>static bundle_get_platform_toolsfalls back tohermes-<platform>for any platform missing from config'splatform_toolsets, andresolve_toolsetreturns[]for an unknown name (the #38798 shape). The runtime auto-generate inresolve_toolsetonly fires oncegateway.platform_registryhas the platform registered, so outside the gateway process (cron delivery, kanban dispatch,hermes tools, doctor) — and in the explicit-config composite expansion, which skips names absent fromTOOLSETS— these platforms silently degrade to zero tools:line,google_chat,teams,irc,ntfy,photon,simplex,raftLINE shipped this way in #23197 ("zero core edits" by design). Real-world impact we hit: a LINE deployment ran for weeks with a zero-tool agent — no
read_file, nomemory, no MCP servers — so the model could only ask the user for information it was supposed to look up; every turn wasapi_calls=1with no diagnostic anywhere.Fix: add the 8 missing static bundles (core tools, mirroring
hermes-telegram/hermes-signal), include them in thehermes-gatewaycomposite, and add a regression test that walksplugins/platforms/so the next platform plugin cannot ship without a bundle.2.
hermes config setstores list/mapping literals as stringsset_config_valueonly coerces bool/int/float. A list literal (e.g.hermes config set platform_toolsets.discord '["clarify","file",...]') is stored as a raw string with no warning — and every reader gated onisinstance(..., list)(_get_platform_tools,_get_enabled_set,_get_disabled_set) silently ignores it and falls back to its default. The setting looks saved but never takes effect. We found both aplatform_toolsetsentry and aplugins.enabledentry in this state in the wild.Fix: values starting with
[or{are parsed withyaml.safe_load; non-list/dict results and YAML errors warn on stderr and keep the legacy string behavior.Tests
tests/test_toolsets.py::TestBundledPlatformBundles— everyplugins/platforms/<name>/with aplugin.yamlmust have a non-empty static bundle (include_registry=False); the 8 new bundles resolve to core tools; gateway composite includes them.tests/hermes_cli/test_config_set_list_values.py— list/mapping literals parse to real lists/dicts, YAML flow lists work, invalid literals warn and keep string behavior, scalars unaffected.tests/test_toolsets.py,tests/hermes_cli/test_tools_config.py,tests/hermes_cli/test_managed_scope_writeguard.py,tests/test_toolset_distributions.py: 170 passed.Related