fix(review): teach skill-review prompts the v2026.7.1 read-before-write guard - #60331
fix(review): teach skill-review prompts the v2026.7.1 read-before-write guard#60331kkwills13 wants to merge 1 commit into
Conversation
…te guard Commit 20871c1 (shipped in v2026.7.1) made skill_manage refuse background- review writes unless the target file was loaded via skill_view in the same review turn. The _SKILL_REVIEW_PROMPT and _COMBINED_REVIEW_PROMPT still told the fork to patch skills straight from conversation context, so every review fork's first skill_manage call was refused and retried — ~300 wasted refusal round-trips/day on the crane profile (60-80k input tokens each). Add an explicit read-before-write rule to both prompts and amend option 1 to re-load via skill_view before patching. Local fix pending upstream.
|
I prepared a fresh support branch for this existing PR rather than opening a duplicate: 2001Y/hermes-agent@07be37d...support/pr-60331-read-before-write-scope It preserves @kkwills13's implementation commit and adds the scope corrections raised in the duplicate review cluster:
Verification:
Current support-branch head: |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing the prompt/guard mismatch. The premise is confirmed on current main: tools/skill_manager_tool.py:382-408 rejects un-read background-review mutations, while agent/background_review.py:208-215 and 309-317 still instruct patching without a current-turn reload.
Problems
agent/background_review.py:234-243applies the pre-read to every support-file write. Currentwrite_fileonly invokes the guard when the target exists (tools/skill_manager_tool.py:1167-1172), so a new support file cannot be pre-read and should be exempt.- The same rule says
deleteis guarded, but_delete_skilldoes not call_background_review_read_before_write_guard(tools/skill_manager_tool.py:1040-1115). - The PR diff changes only the prompt; add contract coverage for both prompt variants.
Suggested changes
- The linked support branch at
7a767e7fd34c9e1c0cc5151aa0e5d1a36f14baafhas the needed narrower wording: existing-file-only coverage, new-target exemptions, no delete claim, one bounded recovery retry, and prompt-contract tests.
Automated hermes-sweeper review.
| "session artifact. If the proposed name only makes sense for " | ||
| "today's task, it's wrong — fall back to (1), (2), or (3).\n\n" | ||
| "Read-before-write (ENFORCED — skill_manage refuses otherwise): " | ||
| "before ANY patch/edit/delete of an existing skill, or " |
There was a problem hiding this comment.
Please scope this to existing-file mutations. write_file only enforces the read guard when the support-file target exists (tools/skill_manager_tool.py:1167-1172), so a new support file cannot be pre-read; _delete_skill also has no read-before-write guard. Exempt new skills/files and omit delete from this enforcement description.
Problem
Commit 20871c1 (shipped in v2026.7.1) made
skill_managerefuse background-review writes unless the target skill was loaded viaskill_viewin the same review turn — a good guard. But_SKILL_REVIEW_PROMPTand_COMBINED_REVIEW_PROMPTinagent/background_review.pystill tell the review fork to patch skills straight from conversation context, so every review fork's firstskill_managecall is refused and retried.On one of our production profiles this produced ~300 refusal round-trips/day at 60–80k input tokens each — pure waste, and the curator effectively stopped landing skill updates.
Example refusal (repeats forever):
Fix
Prompt-only change, mirrored in both prompts:
skill_viewthe exact target in this review turn before any patch/edit/delete or supporting-file write; handle multiple files one at a time).skill_viewbefore patching it.Deployed on our fleet since 2026-07-07: refusals went from ~300/day to zero (13+ hours clean at time of writing).
Test plan
tests/test_background_review_list_shapes.py+tests/test_background_review_session_isolation.py: 21 passed, 1 pre-existing failure also present on vanillamain(TestPersistDisabledHardStop::test_flush_is_a_noop_when_persist_disabled), unrelated to this change.Related observation (not addressed here)
There appears to be a latent race in the concurrent tool executor: ContextVar read-marks set by parallelized
skill_viewcalls can be dropped (tools/thread_context.pyctx.runsnapshot), which would intermittently re-trigger refusals even with correct prompt guidance. Happy to file a separate issue with details.