Skip to content

fix(tools): don't drop a toolset from platform inference when a tool is registered into it - #56480

Closed
designnotdrum wants to merge 2 commits into
NousResearch:mainfrom
designnotdrum:fix/platform-toolset-subset-inference
Closed

designnotdrum wants to merge 2 commits into
NousResearch:mainfrom
designnotdrum:fix/platform-toolset-subset-inference

Conversation

@designnotdrum

Copy link
Copy Markdown
Contributor

What does this PR do?

On inference-based platforms — api_server (the OpenAI-compatible HTTP surface used by Open WebUI) and plugin platforms with no explicit platform_toolsets entry — _get_platform_tools() reverse-maps the platform's static composite (e.g. hermes-api-server) back to configurable toolsets using an all-tools set.issubset() test.

get_toolset() merges registry-registered tools into a toolset's resolution, so resolve_toolset("terminal") grows to include the desktop-only read_terminal after discovery, and resolve_toolset("delegation") grows once a plugin registers a tool into delegation. Platform composites enumerate static tool names and gain nothing from the registry. So the moment a tool is registered into a toolset that the static composite never listed, the subset test fails and the entire toolset is silently dropped — no log, no error. On a default api_server this drops terminal (issue #49622).

The fix compares each candidate toolset's static (pre-registry-merge) membership during reverse-mapping, via a new include_registry flag on resolve_toolset()/get_toolset(). Registry-added tools stop gating toolset inference but remain exposed (subject to their own check_fn) once the toolset is enabled — matching how explicitly-configured platforms (cli/slack/gateway) already behave. This fixes the class at the reverse-map layer rather than data-patching one composite at a time.

Related Issue

Fixes #49622

Supersedes #49626 (which adds read_terminal to the hermes-api-server composite — one symptom). This PR fixes the underlying reverse-map, so no composite needs per-tool maintenance; the composite is left unchanged.

Out of scope (distinct root causes, not this class): #35527 (discord platform-name collision with _TOOLSET_PLATFORM_RESTRICTIONS + _DEFAULT_OFF_TOOLSETS) and #55793 (plugin known_plugin_toolsets discoverability, framed by its reporter as docs + a log breadcrumb, not a behavior change). Left for separate follow-ups.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • toolsets.py — added include_registry: bool = True kwarg to get_toolset() and resolve_toolset(); threaded through recursion, the all/* alias, includes, and the hermes-<plugin> fallback. include_registry=False returns the static TOOLSETS view (no registry-merged tools).
  • hermes_cli/tools_config.py — at the three reverse-map candidate comparisons in _get_platform_tools() (no-explicit-config branch, has_explicit_config mixed-composite expansion, and the non-configurable recovery loop), resolve the candidate toolset with include_registry=False. Universes stay registry-merged so a platform never under-counts what it actually provides.
  • tests/test_toolsets.py — behavior-contract tests for the include_registry flag (static view excludes registry extras, threads through includes, all alias, registry-only toolset → empty).
  • tests/gateway/test_api_server_toolset.py — regression tests: default api_server keeps terminal ([Bug]: API server default toolset drops terminal/process after terminal toolset includes desktop-only read_terminal #49622); registering a tool into an existing toolset does not drop that toolset from inference; default-off / platform-restricted toolsets stay off.

How to Test

  1. _get_platform_tools({}, "api_server") now contains terminal (previously dropped because desktop-only read_terminal is registered into the terminal toolset but absent from the hermes-api-server composite).
  2. Registering any tool into an existing configurable toolset no longer removes that toolset from api_server inference (covered by test_registering_tool_into_toolset_does_not_drop_toolset_from_inference).
  3. Default-off (homeassistant without HASS_TOKEN, x_search without creds) and platform-restricted (discord) toolsets remain off on api_server.
  4. scripts/run_tests.sh tests/test_toolsets.py tests/gateway/test_api_server_toolset.py tests/hermes_cli/test_tools_config.py -q

Verified: the two new regression tests fail without the tools_config change and pass with it; the negative-contract test passes both ways; the toolset/platform/model_tools/coding/cli/cron/web_server test surface stays green.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(tools): ...)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix
  • I've run the relevant test modules and they pass
  • I've added tests for my changes
  • I've tested on my platform: macOS (Python 3.13)

Documentation & Housekeeping

  • I've updated relevant documentation (docstrings on get_toolset/resolve_toolset) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A (no config keys)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact — N/A (pure resolution logic, no OS-specific paths)
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

designnotdrum and others added 2 commits July 1, 2026 11:09
…orm inference

Adds include_registry=True kwarg to resolve_toolset/get_toolset. When False,
returns only the static TOOLSETS view with no registry-merged tools — the
composite-authored membership platform reverse-mapping must compare against.
Default True preserves all existing behavior; this is the enabling half of
the api_server toolset-drop fix (NousResearch#49622).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…is registered into it

_get_platform_tools reverse-maps a platform composite to configurable
toolsets with an all-tools subset test. Because get_toolset() merges
registry-registered tools into a toolset, a tool added to a toolset
(delegate_cli -> delegation; desktop-only read_terminal -> terminal) that the
static composite never listed made the subset test fail, silently dropping the
entire toolset on api_server and other inference-based platforms. Compare the
toolset's static membership at all three reverse-map sites.

Fixes NousResearch#49622.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/tools Tool registry, model_tools, toolsets comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists labels Jul 1, 2026
kshitijk4poor added a commit that referenced this pull request Jul 2, 2026
…stry-derived aliases too

Follow-up on the #56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
kshitijk4poor added a commit that referenced this pull request Jul 2, 2026
…vage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
@kshitijk4poor

Copy link
Copy Markdown
Contributor

Merged via #56887 — your commits were cherry-picked with authorship preserved (both fix(tools): expose static (pre-registry-merge) toolset view and fix(tools): don't drop a toolset from platform inference are now first-class commits on main under your name).

Thanks for the clean root-cause fix — solving this at the reverse-map layer (include_registry=False) rather than data-patching composites is exactly right, so no composite needs per-tool maintenance going forward. Verified: default api_server regains terminal, static view correctly excludes registry-only read_terminal/close_terminal, and default-off/restricted toolsets stay off. Mutation-checked the regression tests (reverting the fix makes them fail).

Closing this in favor of the salvage PR. Appreciate the contribution!

hashbender added a commit to hashbender/hermes-agent that referenced this pull request Jul 2, 2026
…is registered into it (salvage NousResearch#56480) (#722)

Co-authored-by: qbit-mirror-bot <qbit-mirror-bot@users.noreply.github.com>
Jasper6439 pushed a commit to Jasper6439/hermes-agent that referenced this pull request Jul 5, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
Jasper6439 pushed a commit to Jasper6439/hermes-agent that referenced this pull request Jul 5, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
…stry-derived aliases too

Follow-up on the NousResearch#56480 salvage: the include_registry=False docstring said
None is returned only for registry/MCP-only toolsets; it also applies to
registry-derived aliases, which have no static TOOLSETS counterpart.
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
…ch#56480 salvage

Attribution audit gate: the salvaged contributor commits carry
kiljadn@gmail.com (Nick Mason / @designnotdrum). Add the mapping so
contributor_audit.py resolves the author on this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: API server default toolset drops terminal/process after terminal toolset includes desktop-only read_terminal

3 participants