Repository navigation
Conversation
resolve_toolset() synthesizes a hermes-<platform> bundle for any platform in
the platform registry, so plugin platforms resolve to a real tool list even
though the name is absent from TOOLSETS. validate_toolset() had no matching
branch, so the two disagreed for every platform shipped as a plugin rather
than a built-in: 10 of 22 registered platforms on the reporting install.
Config validation reads validate_toolset() as authoritative, which made
`hermes update` report a resolvable bundle as unknown and then suggest the
name it had just rejected ("references unknown toolset 'hermes-teams', did
you mean 'hermes-teams'?"), followed by a false claim that the platform would
have no tools at all.
The new branch is registry-gated, so an unregistered name is still rejected
and typos are still caught.
Refs NousResearch#71650
Duplicate of #57230. Both PRs implement the same registry-gated |
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
validate_toolset()andresolve_toolset()disagree abouthermes-<platform>bundles for platforms that ship as plugins.
resolve_toolset()synthesizes a bundle for any platform in the platformregistry (
toolsets.py:812-828), giving it the core tools plus whatever theplugin registered.
validate_toolset()checks onlyTOOLSETS, the registry'stoolset names, and registry aliases, so it rejects a name that resolves to a
full working tool list. On this install that is 10 of 22 registered platforms:
Config validation treats
validate_toolset()as authoritative(
hermes_cli/config.py:2776feeds it intovalidate_platform_toolsets()), sohermes updateemits two false warnings per affected platform, one of whichsuggests the exact name it just rejected:
The second message is the damaging one: it tells the user their platform is
broken when its tools resolve fine, and the remedy it suggests is a no-op.
The fix adds a registry-gated branch to
validate_toolset()so the predicateagrees with the resolver. Because it is gated on
platform_registry.is_registered(), an unregistered name is still rejected, sotypos are still caught, and the lazy import is wrapped so a registry failure
degrades to
Falserather than propagating into config load.Scope note
This is not the plugin-discovery-ordering bug (#71650, #91757, #95529, and
PRs #97374, #91761, #89351, #89345, #86233, #84499). That one is about when
validation runs relative to plugin load; those PRs make plugin toolset names
like
a2avalidate by deferring validation or by consultingknown_plugin_toolsets.This is a separate defect in what
validate_toolset()knows, and it survivesany ordering fix:
hermes-teamsis missing fromTOOLSETS, from the registry'stoolset names, and from
known_plugin_toolsets(that key records plugintoolsets, not platform bundles), so it stays False even with plugins fully
loaded. I deliberately did not touch the ordering half; that work belongs to the
PRs above.
Measured on this install after full plugin discovery, so ordering is not a
factor:
Related Issue
Refs #71650
Using
Refsrather thanFixes: #71650 is the ordering bug and is not closedby this change.
Type of Change
Changes Made
toolsets.py: new_is_plugin_platform_bundle()helper, andvalidate_toolset()consults it as a final fallback.tests/test_toolsets_plugin_platform_validation.py: 5 pins covering thebundle validating, the validate/resolve invariant, an unregistered name still
being rejected, non-bundle names being unaffected, and a raising registry
degrading to
False.How to Test
Add a plugin platform's bundle to
platform_toolsetsinconfig.yaml(e.g.
teams: [hermes-teams]), then runhermes update. Before thischange it prints the two warnings quoted above; after, it prints neither.
Reproduced end-to-end against a copy of a real
config.yamlin an isolatedHERMES_HOME, running the samevalidate_platform_toolsets()callhermes_cli/config.pymakes, afterimport model_toolshas forced plugindiscovery:
Negative control on the new tests: with
toolsets.pyreverted toupstream/main(git diff --quiet upstream/main -- toolsets.pyconfirmedclean), 2 of the 5 pins fail:
The 3 that pass either way are the guard tests, which is what I want from
them: they must hold before and after.
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passNot ticking the full-suite box. I ran the affected suites, not the whole tree:
The 6 failures in that last run (
test_api_server.py::TestToolsetsEndpoint,test_mcp_reload_refreshes_cached_agents.py, and 4 intest_multiplex_toolsets_profile_isolation.py) are pre-existing: they failidentically with
toolsets.pyreverted toupstream/main. Two collectionerrors under
tests/gateway/relay/are a missingpytest_asyncioin myenvironment, also present on unmodified
main. CI is the authority on the fulltree.
Documentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/ANo config keys, docs, or tool schemas change. The logic is a pure string check
plus a registry membership test with no platform-specific behaviour, but I ran
it only on Linux.
Blast radius
validate_toolset()gets strictly more permissive, and only forhermes--prefixed names backed by a registered platform. Callers that gate onit (
model_tools.py:442/:466,hermes_cli/oneshot.py,cli.py:5532,toolset_distributions.py,tui_gateway/server.py) previously dropped thesebundles or warned about them; they now accept a name whose
resolve_toolset()already returned real tools, which is the behaviour thosecall sites assume. Nothing that validated before stops validating.