fix(review): make skill curation precision-first

[MaximCrabbe]

Summary

• replace the background skill review's activity bias with a shared precision-first quality gate
• require every candidate to be novel, durable, evidence-backed, reusable, and well-scoped before writing
• make Nothing to save. a normal, healthy outcome
• route stable cross-task preferences to user memory; only task-specific learnings may update skills
• keep the existing loaded-skill → umbrella → support-file → create ladder after the gate passes

Why

The current prompt says most sessions should produce a skill update and frames a no-op as a missed opportunity. That pressure encourages one-incident references, duplicated preferences, and speculative claims to become persistent instructions.

This change preserves self-improvement while optimizing for precision instead of write volume.

Partially addresses #57057. It complements, without duplicating, the background write-approval work in #63019 and read-before-write work in #60331.

Tests

• uv run pytest tests/run_agent/test_background_review.py tests/run_agent/test_background_review_summary.py tests/run_agent/test_background_review_cost_controls.py tests/run_agent/test_background_review_cache_parity.py tests/run_agent/test_background_review_toolset_restriction.py tests/run_agent/test_review_prompt_class_first.py tests/test_background_review_list_shapes.py tests/test_background_review_session_isolation.py -q — 79 passed
• uv run ruff check agent/background_review.py tests/run_agent/test_review_prompt_class_first.py
• git diff --cached --check
• read-only adversarial review of candidate tree f06118a... — approved
• bounded micro-delta re-review of final tree f0a1c29... — approved

[teknium1]

Thanks for tightening the review policy. The current prompts do contain the reported activity bias (agent/background_review.py:181-205 and :286-307), and the shared gate addresses it without changing the foreground session or tool schema.

Problems

• The updated ladders still instruct the reviewer to use skills merely previously loaded in the conversation (agent/background_review.py:224-253, :323-348 in this PR). Background-review writes are rejected unless the exact target was read with skill_view in the current review turn (tools/skill_manager_tool.py:382-409; enforced for patch at :959-966). A quality-gated update can therefore still fail its first write attempt.

Suggested changes

• Add the explicit fresh skill_view requirement for every existing target to both ladders, and cover it with a prompt-contract assertion.

Automated hermes-sweeper review.

[teknium1]

Add the current-review-turn read-before-write instruction before this ladder, and mirror it in the combined prompt. tools/skill_manager_tool.py:382-409 rejects background-review edits, patches, removals, and overwrites unless the exact target was loaded with skill_view in this review turn; looking back at a skill loaded in the inherited conversation does not satisfy that guard.

[GottZ]

This was generated by AI during triage.

Summary

Eight PRs address or reference this skill-lifecycle complex. Their diffs separate into class-first curation (#16026), transient-failure filtering (#23004), write hygiene (#23288), memory routing (#24846), trigger repair (#55965), precision gating (#64655), and the duplicated read-before-write implementation in #59175/#75020.

Related pull requests

• feat(review): class-first skill review prompt #16026 [merged] partial — (+110/-10) — n/a: Adds class-first surveying, candidate skill_view, update-before-create guidance, and overlap detection, but does not add the support-file creation guard. Although merged, it remains relevant as the prompt foundation for later curation changes.
• fix(review): tell background reviewer not to capture transient env failures as skills #23004 [merged] best fix — (+84/-0) — n/a; recorded best fix for Skill auto-creation learns from transient failures, causing persistent tool avoidance (learned helplessness) #6051: Both review prompts reject transient environment failures, negative tool claims, resolved session errors, and one-off narratives while redirecting setup failures toward recording the fix. Although merged via e5af1dd6, it remains the reference implementation for the reported persistence cause; sampling behavior was explicitly left separate.
• fix: add write-hygiene guardrails to skill review prompts (prevent SKILL.md bloat) #23288 partial — (+326/-30) — n/a: Replaces completion bias with signal-gated updates and adds background-only checks for date stamps, duplicate headings, and suspicious growth, while leaving foreground writes unchanged. Consistent with the contributor keep_open review, its salvage path is the rebased autonomous-only hygiene implementation with the existing regression suite preserved.
• fix: route background memory review by destination #24846 related — (+67/-22) — n/a: Routes stable user, deployment, procedural, and session-only information to distinct stores rather than changing skill-write guards. Consistent with the contributor keep_open review, author action is to transplant the prompt intent from stale run_agent.py into agent/background_review.py and align it with tools/memory_tool.py:1076-1086.
• feat: trigger-wording check — detect skills that should have loaded but didn't #55965 best fix — (+87/-0) — n/a; recorded best fix for Feature: Skill Lifecycle Quality — Better Descriptions, Proactive Improvement Loop, and Writing Principles (inspired by Anthropic skill-creator) #429: Adds an unloaded-skill comparison pass that broadens trigger descriptions using observed task wording. The contributor keep_open review is controlling: salvage requires skill_view(name) before every patch and reporting through the surfaced action-summary path rather than only the fork's final reply.
• [codex] Fix background review skill read-before-write #59175 [closed] best fix — (+175/-14) — n/a; recorded best fix for [Bug]: Background review agent repeatedly gets stuck — prompt ambiguity, vague tool whitelist, and incomprehensible error message #58475: Requires a full umbrella SKILL.md read before support-file creation, enumerates exact review tools, expands the guard, and returns corrective skill_view(...) calls. It remains relevant although closed because it was explicitly superseded by fix: require skill_view before background skill writes #75020, which carries the implementation forward with original-author attribution.
• fix(review): make skill curation precision-first #64655 partial — (+110/-77) — n/a: Replaces activity bias with a shared novel, durable, evidence-backed, reusable, and well-scoped persistence gate, normalizing Nothing to save. and routing cross-task preferences to memory. Consistent with the contributor keep_open review, salvage requires an explicit fresh skill_view for every existing write target in both ladders plus prompt-contract coverage.
• fix: require skill_view before background skill writes #75020 best fix — (+195/-14) — n/a; recorded best fix for [Bug]: Background review agent repeatedly gets stuck — prompt ambiguity, vague tool whitelist, and incomprehensible error message #58475: Reapplies [codex] Fix background review skill read-before-write #59175 on current main, covering the umbrella-read prerequisite, exact tool contract, actionable guard errors, and patch/create/overwrite regressions. Its MAINTAINER-BOT keep_open verdict is supported by the diff and identifies no substantive correctness or design-fit issue.

Duplicates

#59175 and #75020 implement materially the same #58475 fix; the explicit duplicate chain is #59175 → #75020, with #59175 already closed as superseded. #23288 and #64655 overlap on reducing low-quality writes but retain distinct hygiene and persistence-gate scopes.

Suggested consolidation

Keep #64655 open with a salvage path: retain its precision-first quality gate and memory/skill routing, but require a fresh skill_view for every existing write target and add the requested prompt-contract test, explicitly following its contributor keep_open review. Keep #75020 open as the focused current-main continuation of #59175 under its MAINTAINER-BOT keep_open verdict; leave #59175 closed as duplicate of #75020, and keep #23288, #24846, and #55965 only on their documented contributor-review salvage paths rather than combining their broader concerns into #64655.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I429(["issue #429 (open)"])
    I6051(["issue #6051 (closed)"])
    I58475(["issue #58475 (open)"])
    P64655["PR #64655 (open)"]
    P64655 -.->|partial| I429
    P64655 -.->|partial| I6051
    P64655 -.->|partial| I58475
    class I429 open
    class I6051 closed
    class I58475 open
    class P64655 open
    class P64655 target
    click I429 "https://github.com/NousResearch/hermes-agent/issues/429"
    click I6051 "https://github.com/NousResearch/hermes-agent/issues/6051"
    click I58475 "https://github.com/NousResearch/hermes-agent/issues/58475"
    click P64655 "https://github.com/NousResearch/hermes-agent/pull/64655"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 8 pull requests and 3 issues in this complex. Each diff was read against this issue; Assessment working set: 100 kB of PR diffs, 37 kB of issue/PR text, 14 kB of discussion (14 comments), 15 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.