fix(config): surface invalid platform_toolsets instead of silently dropping tools (#38798) - #52920
Merged
Merged
Conversation
Contributor
🔎 Lint report:
|
| Rule | Count |
|---|---|
invalid-assignment |
1 |
unresolved-import |
1 |
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`
tests/hermes_cli/test_toolset_validation.py:7: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
✅ Fixed issues (2):
| Rule | Count |
|---|---|
unresolved-attribute |
2 |
First entries
run_agent.py:3002: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
Unchanged: 6032 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
…opping tools (#38798) A config migration (or hand-edit) that leaves an invalid toolset name in `platform_toolsets` — e.g. the #38798 corruption that rewrote `hermes-cli` to the non-existent `hermes` — silently disabled all affected tools: resolve_toolset() returns [] for an unknown name, so the agent quietly lost its tools with no error, warning, or log entry and degraded to text-only replies. Surface it loudly at two points: - After migration (migrate_config): validate platform_toolsets and record/print a warning per unknown name, with a `hermes-<platform>` suggestion when that would have been valid (the exact #38798 shape). - At runtime (_get_platform_tools): if a platform was explicitly configured but every toolset name is invalid, log a warning when tools are resolved for a session — so an ALREADY-corrupted config is caught at startup, not only on the next `hermes update`. Logic lives in a new pure, side-effect-free helper (toolset_validation.py) with validate_toolset injected, so it is unit-testable without the tool registry. Note: the original v25→v26 migration that caused the corruption no longer exists (config format is now v30; no migration step rewrites toolset names). This change is the durable defense against the silent-failure mode regardless of cause, matching the issue's "Expected: log a warning". Salvaged from #39207 by @lEWFkRAD (authorship preserved via cherry-pick). Tests: 9 helper cases (incl. the #38798 corruption shape, mixed valid/invalid, zero-tools state, non-dict/scalar/non-string) + a runtime caplog test — both the helper warning and the runtime guard mutation-verified to fail without the fix. Closes #38798. Supersedes #39581 (prevent-in-v25→v26 — that path is gone), #41006 / #40208 (repair-migration for already-corrupted configs).
kshitijk4poor
force-pushed
the
salvage/38798-toolset-validation
branch
from
June 26, 2026 08:37
ae36d36 to
41ede84
Compare
This was referenced Jun 26, 2026
kshitijk4poor
enabled auto-merge
June 26, 2026 08:42
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…-toolset-validation fix(config): surface invalid platform_toolsets instead of silently dropping tools (NousResearch#38798)
HexLab98
added a commit
to HexLab98/hermes-agent-fork
that referenced
this pull request
Jul 5, 2026
…silently dropping tools
A YAML indentation slip nests toolset names under a mapping, e.g.
discord:
- hermes-discord:
- browser
- terminal
which parses to `[{'hermes-discord': ['browser', 'terminal', ...]}]`.
`_get_platform_tools` normalised with `[str(ts) for ts in toolset_names]`,
turning the mapping into the literal string `"{'hermes-discord': [...]}"` —
a name that matches no toolset. `has_explicit_config` stayed False for the
real toolsets and every nested toolset was silently dropped, so the platform
loaded almost no tools (the model could reach cronjob/tts but not
terminal/file/web/browser) while config.yaml looked correct, with no warning
anywhere.
Sibling fixes (NousResearch#38798/NousResearch#52920 invalid names -> zero tools, NousResearch#57063 missing
plugin bundles / list-stored-as-string) don't cover this shape: the result
is non-empty-but-wrong, so their zero-tools guards never fire. Add
`_flatten_toolset_names`, which recovers the intended names (mapping keys +
nested values) so the platform still works and logs a loud warning so the
malformed config is visible and fixable. Well-formed flat lists pass through
unchanged with no warning.
This resolves through the shared resolver, so it fixes CLI, messaging
gateway, and TUI alike.
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…-toolset-validation fix(config): surface invalid platform_toolsets instead of silently dropping tools (NousResearch#38798)
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…-toolset-validation fix(config): surface invalid platform_toolsets instead of silently dropping tools (NousResearch#38798)
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…-toolset-validation fix(config): surface invalid platform_toolsets instead of silently dropping tools (NousResearch#38798)
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…-toolset-validation fix(config): surface invalid platform_toolsets instead of silently dropping tools (NousResearch#38798)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A corrupted or hand-edited
platform_toolsetsentry no longer silently disables all tools — invalid toolset names are now surfaced loudly.Root cause (#38798)
When
platform_toolsetscontains an invalid toolset name (the reported case: a migration rewrotehermes-cli→ the non-existenthermes),resolve_toolset()returns[]for that name. The platform ends up with zero tools, the system prompt is built without tool guidance, and the agent silently degrades to text-only replies — no error, no warning, no log entry. The cause took significant debugging to find.Fix
Surface the failure at two points (
hermes_cli/):migrate_config) — validateplatform_toolsetsand record/print a warning per unknown name, with ahermes-<platform>suggestion when that would have been valid (the exact bug: config migration (v25->v26) corrupts platform_toolsets, silently kills all tools #38798 shape:cli: [hermes]→ "did you mean 'hermes-cli'?")._get_platform_tools) — if a platform was explicitly configured but every toolset name is invalid, log a warning when tools are resolved for a session. This catches an already-corrupted config at startup, not only on the nexthermes update.The logic lives in a new pure, side-effect-free helper (
hermes_cli/toolset_validation.py) withvalidate_toolsetinjected, so it's unit-testable without the tool registry or launching Hermes.Scope note (verify-before-build)
The original v25→v26 migration that caused the corruption no longer exists — config format is now v30, and a full-history pickaxe shows no migration step that rewrites toolset names. So the prevent PR (#39581) is obsolete and the repair-migration PRs (#41006/#40208) target a corruption main no longer produces. This change is the durable defense against the silent-failure mode regardless of cause, which is exactly the issue's stated "Expected: log a warning."
Validation
tests/hermes_cli/test_toolset_validation.py(new) +test_tools_config.py— 113 passed, no regression.hermes-cli), mixed valid/invalid, zero-tools end state, non-dict / scalar / non-string entries, a real-validate_toolsetintegration test, and a runtimecaplogtest.Salvaged from #39207 by @lEWFkRAD (authorship preserved via cherry-pick; AUTHOR_MAP entry added).
Closes #38798. Supersedes #39581 (prevent-in-v25→v26 — that path is gone), #41006 / #40208 (repair-migration for already-corrupted configs).