Skip to content

feat(skills): add lazy load mode to reduce startup token usage - #19174

Closed
LastKnight1881 wants to merge 1 commit into
NousResearch:mainfrom
LastKnight1881:feat/skills-lazy-load
Closed

feat(skills): add lazy load mode to reduce startup token usage#19174
LastKnight1881 wants to merge 1 commit into
NousResearch:mainfrom
LastKnight1881:feat/skills-lazy-load

Conversation

@LastKnight1881

Copy link
Copy Markdown

Summary

Adds a skills.lazy config flag (default: false) to reduce token usage at session startup for agents with large skill libraries.

Problem

The full skills index is injected into every session's system prompt unconditionally. On a typical install with 100+ skills across 30+ categories, this adds ~3,500–5,000 tokens to every session. For local models with smaller context windows (32k–64k), this causes aggressive context compression after only a handful of messages.

Solution

When skills.lazy: true is set in cli-config.yaml, the full skills index is replaced with a compact ~60-token notice:

Skills are available but not pre-loaded. Use skill_view(name) to load a specific skill when you need it, or skills_list() to browse available skills.

The agent retains full access to all skills via skill_view() and skills_list() — it just doesn't have them all in context at startup.

Config

skills:
  lazy: true  # default: false

Changes

  • run_agent.py: Read skills.lazy from config into self._skills_lazy; gate build_skills_system_prompt() on it; inject compact notice when lazy mode is active.
  • cli-config.yaml.example: Document the new lazy flag with comments.

Backwards compatibility

Default is false — existing behavior is fully preserved for all users who don't opt in.

Add a `skills.lazy` config flag (default: false). When enabled, the full
skills index is not injected into the system prompt at session start.
Instead, a compact notice instructs the agent to call skill_view()/
skills_list() on demand when a task matches a known workflow.

This addresses context bloat for agents with large skill libraries
(100+ skills). On a typical install the skills index adds ~3-5k tokens
to every session; lazy mode reduces that to ~60 tokens.

Config usage:
  skills:
    lazy: true

Existing behavior is preserved when the flag is absent or false.
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/skills Skills system (list, view, manage) area/config Config system, migrations, profiles duplicate This issue or pull request already exists labels May 3, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #13980 (and #12379) — same lazy skill loading feature with different config key name (skills.lazy vs skills.loading: lazy).

@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #13980 (and #12379) — same lazy skill loading feature with different config key name.

@ether-btc

Copy link
Copy Markdown
Contributor

14 lines in run_agent.py — this is a targeted fix for the startup token cost problem. Given it's labeled "duplicate", worth checking if there's a canonical version merged or in review that this should close in favor of.

If not superseded: the approach of gating skill loading behind a config flag (skills.lazy_load: true) is the right abstraction. The concern would be: does this introduce a lazy-import pattern that breaks existing synchronous skill_view() calls elsewhere in the codebase?

Recommend: check all call sites of skill_view and get_skill for synchronous expectations. If any are in the startup path and now become lazy, the first call after startup could be slower than before for cold evaluation.

@teknium1

teknium1 commented May 7, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR. Closing based on a standing design rule in our contributor guide: for tools that serve instructional / procedural content (skills, system prompts, playbooks), full-content loading should be the default AND only option. Models will use any excuse to read less — if we add a lazy mode, they'll use it, skip the bulk of the content, and miss critical pitfalls and verification steps that only appear later in a SKILL.md.

The context-pressure concern is real for 32k–64k context models, but the right lever there is skill set curation (fewer installed skills / hermes curator), not lazy-loading the index. The skills index itself is the small part — it's the full SKILL.md reads that cost tokens, and those are driven by skill invocation, not session startup.

For reference: the same pattern was rejected in April 2026 when pagination was proposed for skill_view (see AGENTS.md 'Agent-Facing Tool Design: No Lazy-Reading Escape Hatches').

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have tool/skills Skills system (list, view, manage) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants