Skip to content

perf(skills): speed up snapshot prompt builds - #60339

Closed
dexhunter wants to merge 1 commit into
NousResearch:mainfrom
dexhunter:agent/speed-up-skills-snapshot-prompt
Closed

perf(skills): speed up snapshot prompt builds#60339
dexhunter wants to merge 1 commit into
NousResearch:mainfrom
dexhunter:agent/speed-up-skills-snapshot-prompt

Conversation

@dexhunter

@dexhunter dexhunter commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Build the local skills snapshot manifest with one directory walk instead of separate index scans.
  • Avoid importing gateway.session_context during normal CLI prompt startup.
  • Reuse a direct platform-list matcher for snapshot entries and cover it with existing platform-gating tests.

Fixes #3356.

Benchmark

I ran autoresearch with Weco against a strict synthetic skills snapshot harness for the remaining first-turn skills/frontmatter bottleneck.

Weco dashboard: https://dashboard.weco.ai/share/DHk95734hgGIOpYoaXJxh-Vj6TDgWZA7

The harness builds 700 synthetic skills, seeds the disk snapshot, then measures fresh build_skills_system_prompt child-process calls. It also checks prompt hash stability, platform gating, tool fallback/requirement filtering, and snapshot invalidation after edit/add/delete.

Case hermes_skills_snapshot_prompt_ms
Fresh baseline 69.07 ms
This patch 12.21 ms

That is about an 82% reduction on the measured snapshot prompt-build path.

Validation

  • uvx ruff check --fix .
  • uv run python -m py_compile agent/prompt_builder.py agent/skill_utils.py tests/agent/test_skill_utils.py
  • uv run --with pytest --with pytest-timeout python -m pytest tests/agent/test_skill_utils.py::TestSkillMatchesPlatformTermux tests/agent/test_prompt_builder.py -q
    • 171 passed
  • uv run python ~/osspr/weco_metric_scripts/NousResearch__hermes_agent_skills_prompt_snapshot/evaluate.py --repo-dir . --iterations 7
    • hermes_skills_snapshot_prompt_ms: 12.2136902064085

Fixes NousResearch#3356

Build the skills snapshot manifest in one directory walk, avoid importing gateway session context during CLI prompt startup, and reuse direct platform-list matching for snapshot entries.
@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/skills Skills system (list, view, manage) P3 Low — cosmetic, nice to have labels Jul 7, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via #61131 — your commit was cherry-picked onto current main with your authorship preserved (now e21ba91 on main). Thanks for the contribution!

Validation that went into the salvage PR: manifest output byte-identical to main on a nested/support-dir/excluded-dir fixture, ~5.1x faster builds on a 300-skill fixture (24.9ms -> 4.9ms), and an AUTHOR_MAP entry so release attribution resolves to you. Closing this original in favor of the merged salvage.

santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
zach-hermes pushed a commit to zach-hermes/hermes-agent that referenced this pull request Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have tool/skills Skills system (list, view, manage) type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(ttft): eliminate the remaining cold-start skills frontmatter bottleneck

3 participants