fix(agent): use restricted toolset for background review fork - #61529
fix(agent): use restricted toolset for background review fork#61529liuhao1024 wants to merge 3 commits into
Conversation
The background review fork was constructed with the parent agent's full
toolset (enabled_toolsets) while the runtime dispatch whitelist only
permitted {memory, skills}. This caused a storm of "denied non-whitelisted
tool" errors for file tools (read_file, write_file, patch), leading to
broken self-improvement (the model never landed skill patches).
Fix: build review_toolsets BEFORE constructing the review agent, and pass
it to AIAgent() as enabled_toolsets. This eliminates the toolset/schema
mismatch. Trade-off: tools[] no longer byte-matches the parent for
prompt-cache prefix reuse — acceptable cost for functional correctness.
Refs NousResearch#61521
Related: fixes #61521 (curator fork advertises the parent's full toolset but the runtime whitelist only permits skills+memory → per-turn tool-denial storm). This PR restricts |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing the real schema/whitelist mismatch; current main still has it at agent/background_review.py:694 and agent/background_review.py:791.
Problems
agent/background_review.py:708narrows the advertised schema for every provider. That reverts the cache-parity behavior intentionally established by5fe067226: its commit records that Anthropic’s cache key includestools, and restoring the parent schema reduced measured end-to-end review cost by about 26%. Current main still documents and tests that parent-schema contract atagent/background_review.py:680-705andtests/run_agent/test_background_review_cache_parity.py:191-238.
Suggested changes
- Preserve parent-schema parity for remote cache-backed providers. If narrowing is selected, limit it to local/no-prefix-cache endpoints while retaining the existing whitelist; the linked #39997 takes that scoped approach.
Automated hermes-sweeper review.
| credential_pool=getattr(agent, "_credential_pool", None), | ||
| parent_session_id=agent.session_id, | ||
| enabled_toolsets=getattr(agent, "enabled_toolsets", None), | ||
| enabled_toolsets=review_toolsets, |
There was a problem hiding this comment.
This applies schema narrowing to remote cache-backed providers too. Commit 5fe0672 intentionally restored parent tools[] parity because Anthropic’s cache key includes tools and measured lower review cost; preserve the parent schema remotely and scope narrowing to endpoints without that cache benefit.
What does this PR do?
Fixes the background self-improvement review ("curator") by eliminating a toolset/schema mismatch. The review fork was constructed with the parent agent's full toolset (via
enabled_toolsets=getattr(agent, "enabled_toolsets", None)), but the runtime dispatch whitelist only permitted{memory, skills}. This caused the model to see and call general-purpose file tools (read_file,write_file,patch) in thetools[]schema, then be denied at dispatch every turn with "denied non-whitelisted tool" errors. Net effect: the review loop ran continuously but almost never landed skill patches, breaking self-improvement.Related Issue
Fixes #61521
Type of Change
Changes Made
agent/background_review.py: Buildreview_toolsets(restricted to["skills"]or["memory", "skills"]) BEFORE constructing the review agent, and pass it asenabled_toolsetstoAIAgent(). This eliminates the toolset/schema mismatch.review_toolsetsconstruction that occurred after agent creation (it usedreview_agent._memory_enabledwhich is now unreachable since the agent is already constructed with the restricted toolset).tools[]no longer byte-matches the parent for prompt-cache prefix reuse (the original reason for inheriting the parent's full toolset). The functional correctness fix takes priority over cache parity.How to Test
Run existing background review tests to verify no regression:
All tests pass.
Verification (manual or automated):
tools[]schema now only lists memory/skill tools (noread_file,write_file,patch).Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/A