feat(skills): compact system prompt index via sidecar-driven pinned skills

[sontianye]

Re-spin of #14319 along the path @teknium1 outlined when closing it.

Why

The skills section of the system prompt currently dumps every skill name + description on every turn — ~3,500 tokens on a fully-loaded install. The PR shrinks that to a fixed-budget compact format while keeping the existing framing the dispatcher relies on.

Mapping to the closing review on #14319

Reviewer point	This PR
"read pinned candidates from tools/skill_usage.py (agent_created_report() + sidecar lookup) instead of state.db"	_get_pinned_candidates() reads only via agent_created_report(), filters pinned=True && state != archived, orders by activity_count desc. No state.db access, no duplicate tracking.
"keep the existing 'MUST load' framing intact in the compact format"	Entire MUST-load paragraph kept verbatim. Guarded by test_preserves_must_load_and_hermes_agent_framing.
"preserve the hermes-agent instruction"	Entire hermes config set … paragraph kept verbatim. Same test.
"the query param on skills_list is independently nice but small enough we can do it ourselves"	Dropped from this PR.

What it looks like

## Skills (mandatory)
You have access to 71 specialized skills across 19 categories.
[full MUST-load + hermes-agent framing — unchanged]

<pinned_skills>
  - plan: Create and manage task plans for multi-step work
  - systematic-debugging: Structured approach to diagnosing bugs
</pinned_skills>

<skill_categories>
  software-development (6): …
  github (6): …
  …
</skill_categories>

Load any pinned skill listed above directly with skill_view(name).
Otherwise, list a category with skills_list(category="<name>") to see
what's available, then load the chosen skill with skill_view(name).
Only proceed without loading a skill if genuinely none are relevant to the task.

When nothing is pinned, the <pinned_skills> block is omitted entirely — pinned means pinned, never an alphabetical placeholder.

Implementation notes

• _PINNED_SKILLS_CHAR_BUDGET = 1200 keeps the pinned block bounded regardless of how many skills get pinned.
• <skill_categories> reuses the existing category_descriptions mapping; counts are deduped per top-level category.
• Cache key includes _skill_usage_epoch() (sidecar mtime_ns) so toggling pin state takes effect on the next build with no manual invalidation.
• Bundled / hub-installed skills are intentionally not in the sidecar; the categories block (+ skills_list(category=...)) is how they stay reachable.
• No new settings, no new dependencies, signature of build_skills_system_prompt() unchanged.

Tests

• All existing TestBuildSkillsSystemPrompt and TestBuildSkillsSystemPromptConditional tests updated to assert the new contract (category presence + count) instead of raw name dumps.
• New TestCompactSkillsPrompt:
  • test_preserves_must_load_and_hermes_agent_framing
  • test_pinned_candidates_from_sidecar
  • test_no_pinned_block_when_no_pins
  • test_no_pinned_block_when_skill_set_is_empty
  • test_skill_categories_summary_shows_counts
  • test_pin_change_invalidates_prompt_cache
• tests/agent/test_prompt_builder.py: 127 passed, 1 skipped.

Test plan

• uv run pytest tests/agent/test_prompt_builder.py — 127 passed
• Full suite: 5021 passed (5 pre-existing failures in unrelated gateway_service / anthropic_adapter OAuth / kanban_cli / e2e/test_platform_commands tests, none touching skills code)

Closes #14319

[swen-chan]

I hit the exact same problem — the skills block is eating ~4,500 tokens every turn on my install (90 skills across 20 categories), and there's no config workaround. This PR's approach (pinned from skill_usage.py + category summary, keeping the MUST-load and hermes-agent framing intact) is the right fix. Would love to see this land. Anything I can do to help move it forward?

[sontianye]

Friendly ping — happy to rebase or adjust if you have feedback.

[teknium1]

Thanks for preserving the MUST-load and hermes-agent guidance and for using the existing sidecar rather than state.db. The token-cost premise is still present on current main, but the implementation needs re-scoping against later skill-index decisions.

Problems

• agent/prompt_builder.py:1246 removes every unpinned skill name from the default prompt. Current main deliberately preserves all names after a real capability-loss regression: commit ee1a744ac and agent/prompt_builder.py:1622-1667 state that models do not reliably rediscover hidden skills through skills_list. Commit 4d6a133a9 also makes even names-only demotion opt-in.
• agent/prompt_builder.py:1270 aggregates nested categories, but the footer calls skills_list(category="<name>"). tools/skills_tool.py:827-830 filters categories by exact equality, so a top-level mlops listing omits mlops/evaluation, mlops/inference, and mlops/models skills included in its count.

Suggested changes

• Rework this on the current demote-never-hide contract, retaining names and treating any broader default compaction as a deliberate design decision.
• Align category summaries with exact filtering or add prefix-aware filtering, with nested-category coverage.

Automated hermes-sweeper review.

[teknium1]

Blocking: this global replacement removes unpinned skill names from the default prompt. Current main intentionally preserves every name after a real capability-loss regression (ee1a744ac; see current agent/prompt_builder.py:1622-1667), and even names-only demotion is opt-in (4d6a133a9). Please re-scope onto that demote-never-hide contract.

[teknium1]

Blocking: these counts combine nested categories under top, but the footer later directs skills_list(category="<name>"). Current skills_list uses exact equality (tools/skills_tool.py:827-830), so mlops will not return skills in mlops/evaluation, mlops/inference, or mlops/models even though they are counted here.

[sontianye]

Reworked and force-pushed. The branch is now a single clean commit on top of current main — only agent/prompt_builder.py and tests/agent/test_prompt_builder.py are touched, nothing else.

What changed vs the previous push

The earlier push had unintended noise from a soft-reset accident. This is the correct minimal patch.

Demote-never-hide — Every skill name stays in <available_skills>. The index format is unchanged from main; only the inner loop changes: pinned skills (pinned=True && state != archived in the sidecar) show name: description, everything else shows name only. No skills removed, no skills_list() needed.

No category mismatch — The previous approach's <skill_categories> with top-level aggregation is gone entirely. Category headers now use the exact skills_by_category keys, so mlops/evaluation appears as mlops/evaluation: and its skills are directly reachable by name.

Cache — _skill_usage_epoch() adds sidecar mtime_ns to the cache key so pin state changes take effect on the next build without manual invalidation.

Tests — Four existing assertions updated (Debug Python scripts etc. now correctly assert absent for non-pinned skills). New TestPinnedSkillsInIndex class: names-only for unpinned, description inline for pinned, cache invalidation on pin change, archived skill excluded from candidates.

Diff stat: +196 / -7 lines across two files.