Skip to content

feat: Lazy skill loading — skills.loading config option - #12379

Open
downwind7clawd-ctrl wants to merge 1 commit into
NousResearch:mainfrom
downwind7clawd-ctrl:feature/lazy-skill-loading
Open

feat: Lazy skill loading — skills.loading config option#12379
downwind7clawd-ctrl wants to merge 1 commit into
NousResearch:mainfrom
downwind7clawd-ctrl:feature/lazy-skill-loading

Conversation

@downwind7clawd-ctrl

Copy link
Copy Markdown

Summary

Implements lazy skill loading as proposed in #2045.

Problem

When many skills are installed, all skill names + descriptions are injected into <available_skills> in the system prompt at every session start. With 87 bundled skills, this block alone consumes ~1,500–2,000 tokens — re-sent with every API call. For messaging platform users (Discord/Telegram) paying per-token through proxies without prompt caching, this adds up fast.

Solution

Add a skills.loading config option:

skills:
  loading: lazy    # "eager" (default) or "lazy"

When lazy:

  • The full skill listing is NOT injected into the system prompt
  • Instead, a short hint tells the agent to call skills_list() on demand
  • Agent calls skill_view(name) as normal after discovering relevant skills
  • Saves ~1,500–2,000 system-prompt tokens per turn

When eager (default):

  • Current behavior — full listing in system prompt (backward compatible)

Changes (3 files, +21 lines)

File Change
hermes_cli/config.py Add skills.loading: "eager" default
agent/prompt_builder.py Add lazy param + on-demand hint branch
run_agent.py Read config, pass lazy flag to prompt builder

Token savings estimate

Setup System prompt tokens Working headroom (200k)
87 skills always loaded ~8,500 ~191k
Lazy loading ~6,500 (-2,000) ~193k

Closes #2045

When skills.loading is set to 'lazy' in config.yaml, the full skill
listing is omitted from the system prompt. Instead, a short hint tells
the agent to call skills_list() on demand, then load relevant skills
with skill_view(name).

This saves ~1,500-2,000 system-prompt tokens per turn when many skills
are installed — especially beneficial for messaging platform users who
pay per-token through API proxies without prompt caching.

Changes:
- hermes_cli/config.py: add skills.loading default 'eager'
- agent/prompt_builder.py: add lazy parameter + on-demand hint
- run_agent.py: read config and pass lazy flag to prompt builder

Closes NousResearch#2045

@wilsonchengassistant wilsonchengassistant left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hermes Agent 程式碼審核\n\n### 總結\n此 PR 實施了延遲載入技能的功能,符合 #2045 的提案。程式碼乾淨,邏輯清晰,預期能節省系統提示的 token 消耗。\n\n### 優點\n- ✅ 配置選項設計良好,預設為 'eager' 保持向後相容。\n- ✅ 在 lazy 模式下提供明確的提示,引導代理使用 skills_list()。\n- ✅ 變更最小,只影響提示建構和配置讀取。\n\n### 建議\n- 在 tests/agent/test_prompt_builder.py 中新增單元測試,驗證 lazy 模式下提示字串的生成(例如檢查是否包含 'skills_list()' 提示)。\n- 考慮在 README 或文件更新中提及此新配置選項。\n\n### 結論\nLGTM!批准合併。\n\n---\n由 Hermes Agent 審核

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/skills Skills system (list, view, manage) labels Apr 23, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #13980 — both implement lazy skill loading (skills.loading: lazy) for the same feature request #2045.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting a real prompt-size cost.

Problems

  • agent/prompt_builder.py:609 removes the catalog entirely. Current main deliberately preserves every skill name even when it compacts descriptions (agent/prompt_builder.py:1464-1468), and the regression test explains why: models do not reliably rediscover skills through skills_list after the index goes quiet (tests/agent/test_prompt_builder.py:429-455).
  • The patch has no regression coverage for either config mode.
  • The integration points have moved: current prompt assembly is agent/system_prompt.py:314-318, and skill config initialization is agent/agent_init.py:1423-1428 (introduced by 2d2cd5e904).

Suggested changes

  • Please get maintainer direction on the discoverability trade-off before salvage; the analogous proposal #19174 was closed on this design concern.
  • If the direction is approved, implement it at the current integration points and add eager/lazy prompt-builder coverage.

Automated hermes-sweeper review.

Comment thread agent/prompt_builder.py
@@ -604,6 +609,17 @@ def build_skills_system_prompt(
if not skills_dir.exists() and not external_dirs:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removes every skill name from the initial prompt. Current main intentionally keeps all names visible even in compact mode because tests/agent/test_prompt_builder.py:429-455 documents that models do not reliably rediscover skills after the index goes quiet. Please resolve that discoverability contract before adopting this branch.

@teknium1 teknium1 added the sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users label Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users tool/skills Skills system (list, view, manage) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Lazy skill loading: remove skill listing from system prompt, use on-demand tool instead

4 participants