Skip to content

Make background review toolsets configurable - #45014

Closed
hunterbohm wants to merge 1 commit into
NousResearch:mainfrom
hunterbohm:fix/configurable-review-toolsets
Closed

Make background review toolsets configurable#45014
hunterbohm wants to merge 1 commit into
NousResearch:mainfrom
hunterbohm:fix/configurable-review-toolsets

Conversation

@hunterbohm

Copy link
Copy Markdown

Summary

  • Add memory.review_toolsets with the existing safe default of ["memory", "skills"].
  • Resolve the background self-improvement review whitelist from config instead of hardcoding memory/skills.
  • Whitelist both visible tool schemas and raw pre-Tool-Search schemas so configured MCP/plugin toolsets work when collapsed behind tool_search/tool_call.

Tests

  • python -m pytest tests/run_agent/test_background_review.py tests/run_agent/test_background_review_toolset_restriction.py tests/run_agent/test_background_review_summary.py tests/run_agent/test_background_review_cache_parity.py tests/hermes_cli/test_plugins.py::TestThreadToolWhitelist -q -o 'addopts='
  • python -m pytest tests/hermes_cli/test_config.py tests/hermes_cli/test_config_validation.py -q -o 'addopts='
  • python -m ruff check agent/background_review.py hermes_cli/config.py tests/run_agent/test_background_review_cache_parity.py

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Jun 12, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the runtime whitelist configurable. The current patch needs two corrections before it can preserve main's safety and Tool Search contracts.

Problems

  • agent/background_review.py:510 replaces the current profile gate at agent/background_review.py:802-808 with a default containing memory. Commit 437dcacbb added that gate because a memory-disabled profile could otherwise receive the built-in memory tool; please retain it when composing configured toolsets.
  • The MCP/plugin claim is incomplete for a review-only configured toolset. The fork still inherits the parent scope (agent/background_review.py:711-712), and agent/tool_executor.py:239-256,402-415 rejects an unwrapped tool_call outside that scope. The whitelist alone does not change it.
  • The added test only mocks schema collection and captures the whitelist (tests/run_agent/test_background_review_cache_parity.py:304-326); it does not exercise actual bridge dispatch or the memory-disabled profile case.

Suggested changes

  • Compose configured toolsets with the existing memory-enabled/user-profile-enabled gate.
  • Add real Tool Search dispatch coverage for an in-scope configured deferred tool, plus a memory-disabled regression test.

Automated hermes-sweeper review.

clear_thread_tool_whitelist,
)

review_toolsets = _resolve_background_review_toolsets()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this replaces main's current memory-disable gate (agent/background_review.py:802-808, commit 437dcacbb). Because the new default contains memory, a profile with both _memory_enabled and _user_profile_enabled false will again whitelist the built-in memory tool. Compose the configured list with that existing gate instead of replacing it.

captured["allowed"] = set(allowed)
captured["deny_msg_fmt"] = deny_msg_fmt

with patch.object(run_agent, "AIAgent", _Recorder), \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking coverage gap: this test mocks schema resolution and stops before any tool dispatch. It cannot verify the claimed deferred MCP path, whose real executor scope comes from the inherited agent.enabled_toolsets (agent/tool_executor.py:239-256,402-415), nor catch the memory-disabled-profile regression.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 14, 2026
@hunterbohm hunterbohm closed this by deleting the head repository Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants