Skip to content

feat(guardrails): repeated-mutation halt + destructive-overwrite guard - #43930

Closed
enzo-adami wants to merge 1 commit into
NousResearch:mainfrom
enzo-adami:feat/mutation-loop-and-safe-write-guards
Closed

feat(guardrails): repeated-mutation halt + destructive-overwrite guard#43930
enzo-adami wants to merge 1 commit into
NousResearch:mainfrom
enzo-adami:feat/mutation-loop-and-safe-write-guards

Conversation

@enzo-adami

Copy link
Copy Markdown
Contributor

What

Two opt-out guards for the per-turn tool guardrail controller
(agent/tool_guardrails.py), both born from real incidents running a
long-lived local agent on a small model:

1. Repeated-mutation halt. The existing idempotent_no_progress guard
only watches read-only tools, and the failure counters only advance when a
call fails. A mutating tool that keeps succeeding on the exact same
call is invisible to all current guards — we observed a model issue the
identical write_file 38 times in a single turn. The new check counts
identical successful mutating calls per signature (tool + full args, content
included — so legitimate iterative edits never match, since different
content means a different signature). It reuses the existing
no_progress_warn_after / no_progress_block_after thresholds: warn, then
halt (repeated_mutation_halt). Gated behind hard_stop_enabled like the
other halts, and reset per turn.

2. Destructive-overwrite guard. Blocks a write_file that would replace
a non-empty file (≥ safe_write_min_bytes, default 200) with content
smaller than safe_write_shrink_ratio (default 0.5) of its current size.
Motivation: the model accidentally blanked a status file and a full research
report by overwriting them with empty scaffold content. Design points:

  • Data-loss prevention, not loop detection — independent of
    hard_stop_enabled, controlled by its own safe_write_enabled flag
    (default on).
  • Blocks the call, never halts the turn — the model recovers in place by
    writing the full intended content.
  • Intent override — re-issuing the identical call once confirms a
    genuine shrink/replace, which then proceeds.

Config

New keys under tool_loop_guardrails, all with safe defaults:
safe_write_enabled (true), safe_write_min_bytes (200),
safe_write_shrink_ratio (0.5, validated to (0, 1]).

Tests

tests/run_agent/test_mutation_and_safe_write_guards.py (9 tests): warn →
halt progression, no false positive on iterative edits with changing
content, per-turn reset, hard_stop_enabled gating, shrink block + identical
re-issue override, small/new/growing files allowed, independence from
hard_stop_enabled, flag off. Existing guardrail suite passes unchanged
(9/9).

Field experience

Both guards have been running patched into a production-like local install
since 2026-06-08/09: the mutation halt has caught real config-edit loops,
and the overwrite guard stopped further accidental file blanking with no
false positives on normal edit traffic at the 0.5 ratio (real accidental
blankings kept ~22% of the original size; a 0.2 ratio would have missed
them).

🤖 Generated with Claude Code

…e guard

Two data-protection guards for the per-turn tool guardrail controller,
both born from real incidents on a long-running local agent:

1. Repeated-mutation halt: the existing no-progress guard only watches
   idempotent (read-only) tools, and the failure counters only advance on
   failed calls. A mutating tool that keeps *succeeding* on the exact same
   call is never caught — we observed a model issue the identical
   write_file 38 times in one turn. The guard counts identical successful
   mutating calls per signature (tool + args, content included, so
   legitimate iterative edits never match), warns at the existing
   no_progress warn threshold and halts at the block threshold.

2. Destructive-overwrite guard: blocks a write_file that would replace a
   non-empty file (>= safe_write_min_bytes, default 200) with content
   smaller than safe_write_shrink_ratio (default 0.5) of its current
   size. This is data-loss prevention, not loop detection: it is
   independent of hard_stop_enabled, does not halt the turn (the model
   can recover in place by writing the full intended content), and an
   identical re-issue confirms intent so genuine shrinks still go
   through. Motivated by a model accidentally blanking a status file and
   a full report with empty scaffold content.

New config keys under tool_loop_guardrails (all with safe defaults):
safe_write_enabled, safe_write_min_bytes, safe_write_shrink_ratio.

Existing test suite passes unchanged (tests/run_agent/
test_tool_call_guardrail_runtime.py, 9/9).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jun 11, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Feature: Repeated-mutation halt + destructive-overwrite guard

  • repeated_mutation_halt: When a mutating tool (e.g. write_file) is called identically 5+ times without change, the guard halts with a descriptive message. Different content = different signature, so iterative edits are never blocked. Counter resets each turn.
  • destructive_overwrite_guard: Blocks write_file when the new content is <50% of the existing file size and the file is >=200 bytes. Independent of hard_stop_enabled — data-loss prevention. Re-issuing the identical call once acts as explicit confirmation.
  • Both features are well-isolated and have comprehensive tests (139-line test file). The destructive overwrite guard is particularly valuable given the recent accidental-blanking incident (STATUS.md, research report).

Reviewed by Hermes Agent

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for addressing a real guardrail gap: current main lets successful non-idempotent calls return without tracking progress (agent/tool_guardrails.py:350-352).

Problems

  • The destructive-overwrite decision is action="block", but blocked guardrail calls set a halt (agent/tool_executor.py:477-480, run_agent.py:5669-5671), and the loop exits immediately (agent/conversation_loop.py:4717-4724). The advertised same-turn confirmation cannot run.
  • The proposed os.path.getsize(raw_path) runs before backend-aware resolution. write_file_tool resolves by task at tools/file_tools.py:1696 and executes via the configured backend at tools/file_tools.py:1723-1724; this is incorrect for relative paths and remote backends.
  • The new success counter applies to every mutating tool, including terminal, browser actions, messaging, delegation, and process operations (agent/tool_guardrails.py:41-59), without comparing results. The tests only exercise write_file.
  • The advertised config keys are absent from hermes_cli/config.py:1397-1410 and website/docs/user-guide/configuration.md:1348-1359.

Suggested changes

  • Add a non-halting safe-write rejection path and cover recovery through run_conversation.
  • Perform backend-aware size checking after task path resolution in tools/file_tools.py, with remote and relative-path tests.
  • Limit repeated-success detection to verified no-progress semantics or compare results before halting.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 14, 2026
@enzo-adami

Copy link
Copy Markdown
Contributor Author

Closing instead of rebasing this implementation. Current main now has result-aware idempotent no-progress guardrails, which supersede the generic repeated-mutation counter without false positives across terminal/browser/messaging operations. The destructive-overwrite half needs a separate design: backend-aware path resolution in tools/file_tools.py and a non-halting confirmation path covered through run_conversation. This branch blocks the turn before its advertised confirmation can occur and stats unresolved local paths, so carrying it forward would preserve the wrong contract.

@enzo-adami enzo-adami closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants