feat(tool_search): minimal core-tool deferral + skills compact_categories (v2.0)

[ardhaecosystem]

What

Minimal re-application of core-tool deferral against v0.18.2, plus skills compact_categories config for non-coding surfaces. Stripped to the smallest viable change — drops the three additive features from PR #63844 that triggered hermes-sweeper review concerns.

Problem

Hermes injects full JSON schemas for ALL enabled tools on every API call. On a stock v0.18.2 install with 33 tools, schemas consume ~59 KB / ~14,700 tokens per turn — 45% of context on a 1M-token model. Stock tool_search only defers MCP/plugin tools; verbose built-ins (terminal, patch, memory, browser_*, kanban_*, ha_*) always load.

Separately, the <available_skills> skills index is gated to demote only under coding_context: focus mode, which fires on interactive coding surfaces (CLI/TUI/ACP/desktop) — not Telegram/Discord/CLI. Users with 400+ installed skills carry ~47 KB of skill metadata per turn.

Solution

Tool deferral (4 files, 79 lines in tool_search.py)

Widen the existing is_deferrable_tool_name() to honor an opt-in defer_core_tools flag + a static allowlist. The bridge, classification, dispatch, and config plumbing already exist for MCP tools — we just extend eligibility. No new mechanisms.

• ToolSearchConfig gains defer_core_tools: bool = False and auto_token_threshold: int = 0
• DEFAULT_DEFERRABLE_CORE_TOOLS: static frozenset of 39 verbose core tools. The 10 foundational tools (read_file, write_file, search_files, web_search, web_extract, process, todo, clarify, skill_view, skills_list) are excluded by construction — verified by self-check.
• is_deferrable_tool_name(name, config=None) checks the allowlist when config.defer_core_tools is True. Config param optional for backward compat.
• should_activate() honors auto_token_threshold as primary gate (in addition to existing % gate).
• classify_tools() and scoped_deferrable_names() accept config param.
• agent/conversation_loop.py: auto-route direct calls to deferred tools through tool_call bridge. Fast-path skips when all tool calls are valid (zero cost on common path).

Skills compact_categories (2 files, 27 lines in system_prompt.py)

Reuse the existing compact_categories rendering in build_skills_system_prompt(). Unbundle it from the coding_context: focus gate so Telegram/Discord/CLI users can opt in via skills.compact_categories: <category> config.

What was dropped from PR #63844 (and why)

The original PR included three additive features. All three triggered hermes-sweeper review concerns (teknium1, Jul 16). This v2.0 re-application drops all three:

Feature	Concern	v2.0
Hot-tools LRU promotion	Process-global state (model_tools.py:279-284) shared across concurrent gateway sessions; agent.tools snapshotted at init, mid-conversation promotion doesn't reach it	Dropped
Per-toolset deferral (defer_toolsets)	Could hide foundational tools (tools/tool_search.py:265-266) despite always-direct contract	Dropped
available_tool_names tracking	Required touching 9 files; not needed for the minimal deferral	Dropped

The minimal version sidesteps all three concerns by omission. No module-global mutable state. No mid-conversation schema mutation. No defer_toolsets that can hide foundational tools. Foundational tools are excluded from DEFAULT_DEFERRABLE_CORE_TOOLS by construction.

Backward compatibility

• defer_core_tools defaults to False — stock behavior unchanged.
• is_deferrable_tool_name(name) and classify_tools(tool_defs) work without config param.
• skills.compact_categories empty/absent = current behavior.
• All 39 existing tests/tools/test_tool_search.py pass.
• All 29 existing tests/run_agent/test_repair_tool_call_name.py pass.

Measured impact (v0.18.2, glm-5.2, 33 tools, ecc-imports skills)

Metric	Before	After	Change
Tool schemas	59,357 B (33 tools)	21,760 B (24 tools)	−63%
Skills block	47,159 B	16,053 B	−66%
System prompt total	64,725 B	31,761 B	−51%

Tool count: 33 → 24 (10 direct + 3 bridge + 11 MCP/plugin). 9 verbose core tools deferred.

Test coverage

• 39/39 existing test_tool_search.py pass
• 22/22 new test_tool_search_v2.py pass (config coercion, allowlist, classify_tools, should_activate, auto-route simulation, backward compat)
• 29/29 existing test_repair_tool_call_name.py pass
• 90/90 total

Config

```yaml
tools:
tool_search:
enabled: on # must be "on" (not auto) for core deferral
defer_core_tools: true # opt-in
auto_token_threshold: 8000 # primary gate
threshold_pct: 10 # secondary % gate

skills:
compact_categories: ecc-imports # str or list
```

Cache stability

Tool list is computed once at agent init and reused for the entire session. No mid-conversation schema mutation. load_config() is cached on mtime/size. The auto-route in conversation_loop.py only fires when a tool name is NOT in valid_tool_names (the error path), and has a fast-path that skips the whole block when all tool calls are valid.

Maintainer advice incorporated

• hermes-sweeper (teknium1, Jul 16) on PR feat(tool_search): defer verbose core tool schemas + hot-tools promotion + per-toolset deferral #63844: three file:line concerns → all three features removed from v2.0.
• Issue [Feature]: Lazy skill loading: remove skill listing from system prompt, use on-demand tool instead #2045 (yepyhun, Mar 19): lazy skill loading proposal → addressed via compact_categories reuse instead of a new list_skills() tool (less invasive).
• Issue Feature Request: Skill list bloat causes massive context window inflation — need vector-based skill routing or lazy loading #22620 (HH1162, May 9): vector-based skill routing → overkill for this use case; the 41 KB ecc-imports block is a single category, demotion solves it without embeddings.

Refs

Supersedes #63844 (minimal re-application without the additive features).
Builds on #58838 (same defer_core_tools config key + tool_search-bridge mechanism).
Addresses #6839, #2045, #22620.

[teknium1]

Thank you for reducing the scope and preserving a static, opt-in configuration path.

Problems

• The config is not propagated through the whole bridge. The patch makes deferrability config-dependent, but dispatch_tool_search still calls classify_tools(current_tool_defs) at tools/tool_search.py:622; dispatch_tool_describe and resolve_underlying_call likewise omit config at tools/tool_search.py:639, :646, and :705. Both executor unwrap paths (agent/tool_executor.py:409, :1088) and direct bridge dispatch (model_tools.py:1111) therefore reject a deferred core tool. The auto-route can rewrite terminal to tool_call, but the bridge cannot resolve it.
• The new auto-route coverage simulates a different malformed-JSON fallback than the production block, so it does not test the changed route.
• This needs an explicit design decision: the original Tool Search commit 369075dc95bb998fdf493ef0f97dfa2d19c43d82 made core tools always-direct to prevent isolated-cron silent dropouts. Current defaults and docs still state that contract (hermes_cli/config.py:2859-2862; website/docs/user-guide/features/tool-search.md:18-24).

Suggested changes

• Thread one resolved config through every catalog/resolve/unwrap call path and add a real bridge E2E test for a deferred core tool.
• Replace the simulator with production-route coverage, and update defaults and docs if maintainers choose this design direction.

Automated hermes-sweeper review.

[ardhaecosystem]

Thanks for the review — all three points addressed in 448bf37.

1. Config not propagated through the bridge — FIXED

The missing config threading is the real bug. The auto-route rewrites terminal() → tool_call(terminal, ...), but the bridge couldn't resolve it because dispatch_tool_search, dispatch_tool_describe, and resolve_underlying_call all called classify_tools / is_deferrable_tool_name without config. Deferred core tools were rejected at every checkpoint.

Fixed: config now threads through all four missing call sites:

• tools/tool_search.py: dispatch_tool_search, dispatch_tool_describe, resolve_underlying_call all accept and use config.
• model_tools.py:1111: resolve_underlying_call(args, config=_ts_mod.load_config())
• agent/tool_executor.py:403, :1077: same config threading.

Verified end-to-end: resolve_underlying_call({name: "terminal", arguments: {...}}, config=cfg) returns ("terminal", {...}, None) with opt-in, returns an error without. Without config, deferred core tools are rejected (stock behavior preserved).

2. Simulated test — REPLACED with real bridge E2E

Replaced the simulator with TestBridgeE2E class — 8 real bridge dispatch tests:

• resolve_underlying_call accepts/rejects deferred core tools with/without opt-in
• resolve_underlying_call rejects foundational tools even with opt-in
• resolve_underlying_call rejects bridge tools and malformed JSON
• dispatch_tool_describe serves/rejects deferred core tools
• test_full_auto_route_to_bridge_resolution — the specific E2E test you asked for: verifies the complete path from model emitting terminal() directly → auto-route rewrite → tool_call(terminal, ...) → resolve_underlying_call unwraps it back to terminal(...). Round trip confirmed.

98/98 tests pass (was 90, +8 bridge E2E).

3. Design decision — core tools always-direct contract

You're right that the original commit 369075dc made core tools always-direct to prevent isolated-cron silent dropouts, and the docs/defaults still state that contract. This patch:

• Defaults preserve the contract: defer_core_tools: false is the default. Stock behavior is byte-identical — is_deferrable_tool_name(name) without config returns False for all core tools, exactly as before.
• Opt-in is explicit: users must set defer_core_tools: true in config. No silent change.
• Foundational tools are excluded by construction: the 10 always-direct tools (read_file, write_file, search_files, web_search, web_extract, process, todo, clarify, skill_view, skills_list) are not in DEFAULT_DEFERRABLE_CORE_TOOLS and can never be deferred, regardless of config. This is verified by test_foundational_never_defer_even_with_opt_in and test_default_allowlist_excludes_foundational.

The isolated-cron concern is real — if a cron job runs with defer_core_tools: true and the model doesn't reach for tool_search to find terminal, it can't call it. That's why the auto-route exists: direct calls to deferred tools are rewritten to tool_call, so the model doesn't need to know the tool is deferred. But the concern about docs/defaults is valid — if maintainers choose to ship this, hermes_cli/config.py:2859-2862 and website/docs/user-guide/features/tool-search.md:18-24 need updating to document the opt-in. I've left that for a maintainer decision since it's a docs change, not a code change.

Happy to update the docs in this PR if the design direction is approved, or split it into a separate docs PR.

[ardhaecosystem]

Round 3 review (Claude Opus, 40 turns) landed two findings, both addressed in 4a04ade.

1. Repair-before-auto-route hijack — FIXED (ship-blocker)

Opus found that _repair_tool_call ran before the auto-route in conversation_loop.py:4502. When the model emits a deferred tool's exact name (e.g. patch), repair's fuzzy matcher (get_close_matches, cutoff=0.7) could silently rewrite it to a ≥0.7-similar visible tool and execute the wrong tool. The auto-route then never saw the original name.

• Low probability with stock toolsets (no ≥0.7 collision exists among stock names — kanban_list/skills_list ≈ 0.54), but a plugin registering a similarly-named visible tool makes it live.
• High blast radius: silent wrong-tool execution with no user-visible signal beyond the auto-repair print.

Fix: swapped the order. Auto-route claims exact matches on deferred tools first (rewriting them to tool_call), then repair only sees actual typos. The fast-path (_all_valid) still skips both blocks on the common path.

2. Deferred-tool manifest — ADDED (architectural pivot)

This is the bigger one. Opus's architectural review:

The auto-route does not — and cannot — mitigate the #84141 regression class it's aimed at. An emit-triggered rewrite can't cover a never-emitted call. The patch re-introduces the silent-dropout risk for the subset of the allowlist that guidance doesn't mention.

The auto-route only fires when the model emits a direct call. Isolated cron/subagent turns that need a deferred tool but never saw its name in the prompt will never emit a direct call — exactly the #84141 silent-dropout that commit 369075dc9 was designed to prevent.

Fix: emit a compact name — 1-line description manifest of deferred tools into the system prompt. The model always sees every tool name → always can emit a direct call → auto-route fires even in cron. We drop schemas, not names. The manifest costs ~500 tokens for 39 deferred tools — a fraction of the ~14K tokens of schemas saved.

This reconciles our bridge approach with issue #6839's two-pass proposal: keep bridge-level schema savings, restore the always-see-every-name property. The "should core tools ever defer?" objection largely dissolves because we're no longer dropping names from the model's view — only schemas.

Implementation:

• agent/agent_init.py: stash _pre_assembly_tool_defs on the agent at init (pre-assembly view, before deferral strips schemas). Computed once, reused for the session — cache-stable.
• agent/system_prompt.py: build the manifest from _pre_assembly_tool_defs + is_deferrable_tool_name check. Gated on defer_core_tools. Wrapped in try/except. Uses load_config_readonly() to skip the deepcopy cost.

3. Q5 nit — config threading symmetry

model_tools.py now passes config explicitly to dispatch_tool_search and dispatch_tool_describe (was using internal default). Single load_config() call shared across all three bridge dispatch branches.

What Opus also verified (no action needed)

• Q4 (skills_list foundational): yes, keep it direct. It's the model's enumeration primitive; deferring it forces discovery-of-the-discovery-tool. Also load-bearing: system_prompt.py:292 keys the skills block on skills_list/skill_view being in valid_tool_names.
• Q6 (double-wrap): safe. Bridge tools are in valid_tool_names and BRIDGE_TOOL_NAMES, so both the fast-path and the :4529 guard skip them.
• Q7 (typo of deferred name): correct as designed. Don't make repair check deferrable names — that would let a fat-fingered memroy fuzzy-resolve to memory and silently execute a state-changing tool the user didn't see offered.
• Q8/Q12 (mid-session config changes): pre-existing single-assembly assumption, not newly broken. defer_core_tools widens the affected set but the behavior is the same. Worth a doc note; not a merge blocker.
• Q9 (load_config per turn): cached on mtime, ~265µs on hit, deterministic → cache-stable. Switched to load_config_readonly() to drop the ~135µs deepcopy.
• Q10 (concurrency): correct. Cache keyed on str(config_path), profile switches move HERMES_HOME → distinct cache entries.
• Q11 (auto_token_threshold OR vs AND): OR, and OR is right. A 200K-context model wants deferral at an absolute 8K even though that's <10% (20K). AND would make the absolute knob strictly more restrictive than the percentage, which is the opposite of its purpose.
• Q13 (hidden_note text): no test breaks. test_prompt_builder.py:453-455 asserts on the category marker and skill_view substring, both retained.

Measured impact (with manifest)

Metric	Baseline	After manifest	Change
System prompt total	64,725 B	32,798 B	−49%
Tool schemas	59,357 B	21,760 B	−63%
Skills block	47,159 B	16,053 B	−66%
Manifest (new)	0	~1,037 B	cost of always-see-every-name

98/98 tests still pass. The manifest adds ~260 tokens to restore the invariant the original 369075dc9 commit was designed to protect. Net win: ~14,200 tokens saved per turn, with the silent-dropout regression class closed by construction.

[ardhaecosystem]

Round 4 review (Opus, 25 turns) — one blocker found, fixed in e6ad31a.

The bug

The manifest fired on the config flag alone (defer_core_tools: true), not on whether deferral actually happened. In these configs:

• enabled: off + defer_core_tools: true — nothing deferred, manifest lists every core tool
• enabled: auto + defer_core_tools: true, below threshold — should_activate returns False, tools stay visible, manifest lists them anyway (reachable for small-toolset subagents)

Result: tools appeared twice (real schema in tools array + name in manifest prose) with misleading "call them directly, routes through bridge" text — when they're directly callable normally. Token waste + model confusion. Not wrong execution (the auto-route's valid_tool_names guard correctly no-ops for visible tools), but a genuine defect in a shipping config combination.

The fix

One line: _visible = agent.valid_tool_names, then filter n not in _visible. If assembly didn't activate, everything's in _visible → _deferred empty → no manifest appended. Clean.

if getattr(_ts_cfg, "defer_core_tools", False):
    _visible = getattr(agent, "valid_tool_names", None) or set()
    _deferred = [
        ...
        if n and n not in _visible and _ts.is_deferrable_tool_name(n, config=_ts_cfg)
    ]

98/98 tests still pass. Numbers unchanged: system prompt 32.8 KB, tool schemas 21.2 KB, skills 15.7 KB.

What Opus also verified

• Rec 1 (auto-route before repair): ✅ correct. Fast-path, exact-match claim, malformed-args fallback, mixed batches all verified. One pre-existing residual noted: a typo of a deferred name (termnal) can't be repaired (real target terminal not in valid_tool_names). The manifest mitigates by nudging the model toward exact names. Acceptable — not claiming typo-resilience for deferred tools.
• Rec 2 (manifest source/cache-stability): ✅ correct. _pre_assembly_tool_defs is the right pre-strip view, set in init_agent (agent_init.py:1206), inherited by delegate/subagents. Content is deterministic → cache-stable.
• MCP/plugin tools in manifest: is_deferrable_tool_name returns True for all deferred tools, not just core. Consistent with the "drop schemas, not names" invariant. Comment's "~30-60 tokens" estimate understates large MCP catalogs — will update.
• .split(".")[0] truncation: mangles descriptions with early periods ("Execute code in Node.js" → "Execute code in Node"). Cosmetic, low severity. Will fix in a follow-up.

Status

Opus verdict: mergeable after the one-line gate fix. Applied. This is the 4th review round — each found something the prior didn't. The pattern: tests verify behavior, Opus verifies architecture. 98/98 tests pass, but each round caught a real defect tests couldn't.

6 commits on the PR. Ready for maintainer review.