fix(cli): don't warn on plugin toolsets at startup; accept them in subagent lifecycle

[adamkrawczyk]

What does this PR do?

Fixes the false-positive Warning: Unknown toolsets: ... spam on every hermes / hermes chat start for users who enabled plugin toolsets via hermes tools.

Root cause (two sites, one bug class):

1. HermesCLI.__init__ validates configured platform toolsets before background plugin discovery has landed plugin-registered toolsets in the live tool registry (startup launches discovery in a daemon thread; validation doesn't wait). MCP server names already had an exclusion at this exact call site for the same reason — plugin toolsets were missed. Live repro on current main: hermes chat -Q --max-turns 1 -q … warns Unknown toolsets: a2a, evey_autonomy, … while the plugin toolsets are real and load moments later.

2. agent/subagent_lifecycle.py: _validate_request compares allowed_toolsets against the static TOOLSETS table only — so a delegate_task request naming a plugin-registered toolset (even one the parent legitimately runs with enabled) hard-fails with Unknown toolsets: ….

Fix:

• cli.py: extend the existing MCP-name exclusion with get_plugin_toolset_keys_nowait() — live registry keys when discovery finished, the persisted key set from the previous launch (cache file already written by _persist_plugin_toolset_keys()) while background discovery is in flight. Non-blocking (no startup latency), race-free on steady state, self-healing on first launch after a plugin is added (worst case one transitional warning).
• subagent_lifecycle.py: use validate_toolset() (static table + plugin toolsets + registry aliases) instead of static-table membership.

Genuinely unknown names still warn / still raise — the guard stays real.

Related Issue

Fixes #71650
Fixes #86231
Related: #78102 (same false-positive class for MCP names in platform lists), #52382 (stale toolset names never pruned — orthogonal, config migration), #29532.

Related PRs: #84499 (awaits discovery before warning — correct but adds startup blocking and large concurrency surface), #88003 (excludes only names listed in known_plugin_toolsets config — misses plugin toolsets a user hasn't saved via hermes tools yet, and names from plugins disabled in config), #25714 (superseded). This PR is the minimal, non-blocking variant using infrastructure that already exists on main.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• 🌟 New feature
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refrefactor (no behavior change)
• 🎯 New skill

Changes Made

• cli.py: exclude plugin-declared toolset names from the startup "Unknown toolsets" warning (parallel to the existing MCP-name exclusion).
• agent/subagent_licide.py: _validate_request now validates via validate_toolset() so plugin-registered toolsets are accepted in allowed_toolsets.
• tests/cli/test_plugin_toolset_startup_validation.py: 6 regression tests — cold-registry + persisted-cache race, live-registry plugin toolset, typo still warns, cache-fallback helper contract, lifecycle accepts plugin toolset, lifecycle still rejects typos.
• contributors/emails/adam-krawczyk@outlook.com: attribution mapping.

How to Test

1. Enable a user plugin that registers a toolset; save it for cli via hermes tools.
2. hermes chat -Q --max-turns 1 -q 'Reply exactly OK without calling tools.' — before: false warning; after: clean.
3. Configure a genuinely missing toolset name — still warns.
4. delegate_task with allowed_toolsets naming a plugin toolset — no longer raises Unknown toolsets.

$ python3 -m pytest tests/cli/test_plugin_toolset_startup_validation.py tests/cli/test_cli_init.py tests/agent/test_subagent_lifecycle.py tests/hermes_cli/test_plugins.py tests/hermes_cli/test_tools_config.py tests/hermes_cli/test_tools_disable_enable.py tests/hermes_cli/test_kanban_worker_spawn_toolsets.py -q -o 'addopts='
# 233 passed, 2 skipped — all green

Real-binary canary on the affected install (15 plugin toolsets configured):

before: Warning: Unknown toolsets: a2a, evey_autonomy, evey_bridge, evey_cost_guard, ...
after:  (no warning)

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs (fix(cli): await plugin discovery before toolset warning #84499 / fix(cli): don't warn on plugin toolsets; skip kanban TERMINAL_CWD deprecation #88003 / Nous Portal subscription integration #25 done first; this is the minimal non-blocking variant)
• My PR contains only changes related to this fix
• I've run pytest tests/ -q and all tests pass (targeted suites: 233 passed — full-suite run in CI)
• I've added tests for my changes
• I've tested on my platform: Linux 7.0.0 x86_64

Documentation & Housekeeping

• Documentation changes are N/A — behavior documented in code + tests
• cli-config.yaml.example changes are N/A — no config keys changed
• CONTRIBUTING.md / AGENTS.md changes are N/A
• I've considered cross-platform impact (pure-Python threading semantics unchanged; no new deps)
• Tool descriptions/schemas are N/A

Screenshots / Logs

Startup output diff on the affected machine:

$ hermes chat -Q --max-turns 1 -q …
-Warning: Unknown toolsets: a2a, evey_autonomy, evey_bridge, evey_cost_guard, evey_delegate, evey_digest, evey_goals, evey_adam@hermes-installed-host — 15 plugin toolsets false-flagged
+(no warning)

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[adamkrawczyk]

Full-suite verification: zero regressions (A/B against pristine main)

Ran scripts/run_tests.sh (per-file sweep, ~3090 files) on this branch. It surfaced 18 files with failures + 6 collection errors — so I A/B'd every one of those 24 files against a pristine 9664e386f6 worktree (git worktree add) using the identical interpreter and flags.

Result: 23/24 byte-identical, 0 regressed, 1 improved.

	baseline 9664e386f6	this branch
files compared	24	24
regressed	—	0
improved	—	1 (test_transcription_tools.py 1→0, flaky)

The 6 "collection error" files were pure parallel-run resource contention (load avg 31 on a 12-core box) — all pass cleanly when run individually on both trees: test_doctor.py 54 passed, test_web_server.py 165 passed, test_run_agent.py 261 passed, test_authoring_standards.py 1196 passed, test_tui_gateway_server.py 585 passed.

Pre-existing failures on pristine main, reproduced identically here (env-dependent — Daytona/Modal/fal/media gateways need credentials or network):

 6 tests/plugins/memory/test_hindsight_provider.py
 8 tests/plugins/video_gen/test_fal_plugin.py
 1 tests/test_hermes_state.py
15 tests/tools/test_daytona_environment.py
 2 tests/tools/test_image_generation.py
 1 tests/tools/test_managed_media_gateways.py
 2 tests/tools/test_modal_snapshot_isolation.py
26 tests/tools/test_video_generation_tool_surface_matrix.py
 2 tests/tools/test_web_tools_config.py
 1 tests/tui_gateway/test_slash_worker_mcp_discovery.py
── 65 pre-existing failing tests, unchanged by this PR

Notably tests/run_agent/test_moa_loop_mode.py (27 passed) and tests/tools/test_mcp_discovery_cross_process.py (1 passed) — the two closest to this diff's blast radius — are green on both trees; their appearance in the parallel sweep was contention noise.

Phase-1 xdist run on this branch: 4974 passed, 149 skipped, 2 failed (test_sse_basic.py timeout + test_readme_claims.py live-measurement — both environmental, both unrelated to toolset validation).

Targeted suites covering the changed code paths, all green on this branch: tests/cli/test_plugin_toolset_startup_validation.py (6 new), test_cli_init.py (35), tests/agent/test_subagent_lifecycle.py (4), tests/hermes_cli/test_plugins.py (64), test_tools_config.py + test_tools_disable_enable.py + test_kanban_worker_spawn_toolsets.py (60) — 233 passed, 2 skipped.

[rhein1]

Live confirmation from a Hermes v0.21.1 deployment: A2A was configured and runtime-healthy, but startup still emitted the false unknown toolset 'a2a' warning. We carried the equivalent CLI validation fix locally and the focused Hermes toolset/config tests passed 25/25; after restart, the A2A agent card remained healthy on localhost and only toolset.a2a was advertised. This supports fixing validation without weakening typo detection. No secrets or paid calls were involved.

[byjaps]

Cross-linking, in case it helps move this along: the cli.py half of this PR is exactly right and still applies cleanly to current main, but the agent/subagent_lifecycle.py hunk conflicts there (that validation was rewritten upstream into set(request.allowed_toolsets) - set(TOOLSETS)).

I re-landed the startup fix against current main in #116425, with a regression test that proves the false warning on base and credited you as co-author (that approach came from your PR). If you'd rather keep this PR as the home for the fix, say so and I'll close mine and move the test here instead — whichever keeps the fix moving.

One data point on impact, since the issue is about warnings: they are written to the console, so they reach the stdout of hermes chat -q -Q. In an integration of mine that reads that stdout as the answer, Warning: Unknown toolsets: voice_stack was the reply (a TTS pipeline read it out loud instead of the user's answer), which is what got me to trace it back here.

[teknium1]

Thanks @adamkrawczyk — credited as the earliest fix; the landed change is #116425, which you co-authored.

Superseded by #118841 (merge 74f726c), which credits this PR. Closing.