Skip to content

fix(cli): discover plugins before warning about unknown toolsets - #25714

Open
rewasa wants to merge 1 commit into
NousResearch:mainfrom
rewasa:fix/cli-toolset-plugin-discovery
Open

rewasa wants to merge 1 commit into
NousResearch:mainfrom
rewasa:fix/cli-toolset-plugin-discovery

Conversation

@rewasa

@rewasa rewasa commented May 14, 2026 •

Copy link
Copy Markdown

What does this PR do?

Fixes a false-positive CLI startup warning for plugin-contributed toolsets.

When HermesCLI.__init__ validates --toolsets/configured toolsets, plugin discovery has not necessarily happened yet. That caused valid plugin toolsets (for example code_intel) to print:

Warning: Unknown toolsets: code_intel

The CLI now mirrors the existing lazy-discovery behavior from hermes_cli/oneshot.py and tui_gateway/server.py: if any requested toolset is initially unknown, it runs discover_plugins() and re-validates before warning. Truly unknown toolsets still warn.

Related plugin/example: rewasa/hermes-code-intel-plugin — AST-aware code intelligence plugin for Hermes Agent. This PR fixes the CLI validation path so plugin-provided toolsets like code_intel can be discovered before being reported as unknown.

Related Issue

Fixes false-positive plugin toolset validation warnings. No linked issue.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • cli.py: discover plugins and re-run validate_toolset() before emitting unknown-toolset warnings.
  • tests/cli/test_cli_init.py: add regression tests for:
    • plugin toolsets becoming valid after discovery without warning
    • genuinely unknown toolsets still warning after discovery

How to Test

  1. Configure a plugin-contributed toolset such as code_intel in a Hermes profile.
  2. Start Hermes with that profile.
  3. Confirm no false Warning: Unknown toolsets: code_intel is printed.
  4. Configure a deliberately missing toolset and confirm the warning is still printed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 26.4.1

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

$ pytest tests/cli/test_cli_init.py -q
bringing up nodes...
bringing up nodes...

........................................                                 [100%]
40 passed in 8.10s

$ python -m py_compile cli.py tests/cli/test_cli_init.py
# no output (success)

@rewasa
rewasa requested a review from a team May 14, 2026 13:21
@alt-glitch alt-glitch added type/feature New feature or request comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels May 14, 2026

@austinpickett austinpickett left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix merge conflicts and use .github/PULL_REQUEST_TEMPLATE.md

@rewasa
rewasa force-pushed the fix/cli-toolset-plugin-discovery branch from 943a435 to 6ad8c50 Compare May 19, 2026 12:34
@rewasa

rewasa commented May 19, 2026

Copy link
Copy Markdown
Author

Addressed the review feedback.

Changes made:

  • Rebased/squashed the branch down to a single focused commit on latest origin/main to remove the merge conflicts and unrelated commits.
  • Updated the PR body to follow .github/PULL_REQUEST_TEMPLATE.md.
  • Added regression tests for plugin-discovered toolsets and truly unknown toolsets.

Verification:

pytest tests/cli/test_cli_init.py -q
40 passed in 8.10s

python -m py_compile cli.py tests/cli/test_cli_init.py
# success

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@rewasa
rewasa force-pushed the fix/cli-toolset-plugin-discovery branch from 6ad8c50 to e70d589 Compare May 19, 2026 13:17
@rewasa
rewasa requested a review from austinpickett May 19, 2026 13:19
The toolset validation in HermesCLI runs before plugin discovery, causing
false-positive "Unknown toolsets" warnings for toolsets contributed by
plugins (e.g. code_intel). Mirror the lazy-discovery pattern already used
in hermes_cli/oneshot.py and tui_gateway/server.py: if a toolset looks
invalid, run discover_plugins() and re-validate before warning.
@rewasa
rewasa force-pushed the fix/cli-toolset-plugin-discovery branch from e70d589 to cad502c Compare May 19, 2026 14:43

@austinpickett austinpickett left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: fix(cli): discover plugins before warning about unknown toolsets

3 files, +44/-2. Fixes a bug where plugin-contributed toolsets triggered false "Unknown toolsets" warnings because validate_toolset() ran before discover_plugins().


Assessment

(item 1) Fix is correct: Inserts plugin discovery between initial validation and the warning, matching the existing pattern already used in hermes_cli/oneshot.py and tui_gateway/server.py. Only triggers discover_plugins() when there are already-invalid toolsets (lazy/on-demand). Re-validates only the previously-invalid subset. except Exception: pass matches the existing convention.

(item 2) Tests are good: Two well-structured tests — one verifies plugin discovery resolves the toolset (no warning), one confirms genuinely unknown toolsets still warn. The ordering assertion via discover_plugins.called is clean.

(item 3) Minor scope creep: Author-map entry in release.py — one line, zero risk.

Risk: Low. New code is guarded by if invalid: and wrapped in broad exception handler. discover_plugins() is idempotent.

Verdict: Approve ✅ — Clean, focused bug fix with good tests.

Reviewed by Hermes Agent

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for isolating the plugin-discovery ordering issue. The premise remains valid on current main: cli.py:3886-3888 validates before discovery and prints the warning, while hermes_cli/oneshot.py:65-76 and tui_gateway/server.py:2645-2656 already use discovery followed by re-validation.

Problems

  • tests/cli/test_cli_init.py:94 verifies that discovery runs, but does not assert that the warning is absent. enabled_toolsets is assigned before validation, so its asserted value does not distinguish the fixed behavior from a path that still warns.
  • scripts/release.py is unrelated to the CLI behavior and no longer applies cleanly to current main's moved AUTHOR_MAP block.

Suggested changes

  • Capture _console_print and assert no Unknown toolsets warning for a toolset that becomes valid after discovery; keep the existing negative-case assertion.
  • Exclude the AUTHOR_MAP hunk when salvaging.

Automated hermes-sweeper review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants