Skip to content

fix(guardrails): soft-block exact repeated tool-call failures by default - #57303

Open
lEWFkRAD wants to merge 2 commits into
NousResearch:mainfrom
lEWFkRAD:fix/tool-call-loop-breaker
Open

fix(guardrails): soft-block exact repeated tool-call failures by default#57303
lEWFkRAD wants to merge 2 commits into
NousResearch:mainfrom
lEWFkRAD:fix/tool-call-loop-breaker

Conversation

@lEWFkRAD

@lEWFkRAD lEWFkRAD commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes the tool-loop guardrail's block tier actually fire in default configuration — as a soft block that skips execution but keeps the turn alive — instead of being dead code unless hard_stop_enabled is set.

Today, with default config, a model that repeats a byte-identical failing tool call gets a repeated_exact_failure_warning appended to every failure… and nothing else, forever. In a production session on a local model (qwen3.6-27b), the same rejected terminal call was re-sent 20+ consecutive times; the stored transcript shows the guardrail warning firing at count=11, 13, 15, 17 with zero effect, burning the iteration budget (default 300) on a call that failed deterministically in ~3ms. Context compaction did not break the loop — the compaction summary described the loop and the model resumed it as its first post-compaction action.

This PR adds a middle tier between "warn" and "halt the turn":

  • After hard_stop_after.exact_failure (default 5) identical failed calls, the exact call is no longer executed. The model receives a synthetic error result and the turn continues, so it can recover on its own.
  • Same treatment for the idempotent no-progress axis (hard_stop_after.idempotent_no_progress identical results from a read-only call) — this is the exact scenario in [Bug]: Agent loops on identical tool calls despite being blocked — need better re-prompting after repeat detection #41490.
  • Soft-block wording rotates between attempts. This is load-bearing, not cosmetic: replaying a real stuck context against the same model showed the loop is self-conditioning — with 3 identical failed rounds in context the model fixed the call 6/6 samples, at 8 rounds it started repeating (2/6), and a prefix containing one fresh interleaved user message sampled 0/6 repeats. Identical guidance gets absorbed into the repeated pattern; varied guidance breaks it. (Anti-repetition samplers were also tested and made things worsedry_multiplier 0.8 / repeat_penalty 1.1 penalize re-sending the command string, so the model mangles the command instead of changing the failing argument.)
  • Hard stops (hard_stop_enabled: true) are completely unchanged: blocks still end the turn with the controlled halt response, thresholds and messages identical.
  • Opt-out via tool_loop_guardrails.block_enabled: false restores today's warn-only behavior.

Blocked attempts are recorded inside before_call (blocked calls never reach after_call), so consecutive blocks keep escalating the count and, when hard stops are enabled, streaks stay accurate.

Known limitation (shared with the existing opt-in hard block): a soft-blocked read-only signature stays blocked for the rest of the turn even if a later mutation would have changed its result; the model can always issue a modified call.

Relationship to nearby work: #37490 adds turn-continuation redirect guidance at the conversation-loop layer for tool-reported loop blocks; #54340 addresses the same-tool (varying-args) failure axis; #49189 flips hard stops on for non-interactive platforms. This PR is orthogonal to all three — it makes the exact-identical block tiers exist at all in default mode — and implements the failure/no-progress subset of RFC #35573 within the existing guardrail framework.

Related Issue

Fixes #41490

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/tool_guardrails.py: block_enabled config flag (default true); soft field on ToolGuardrailDecision (should_halt excludes soft decisions, to_metadata() reports it); before_call returns soft blocks for exact-failure and idempotent-no-progress streaks when hard stops are off; rotating message templates for both axes.
  • hermes_cli/config.py: block_enabled: True in DEFAULT_CONFIG["tool_loop_guardrails"] + comment.
  • cli-config.yaml.example: document the new tier and flag.
  • tests/agent/test_tool_guardrails.py: soft-block coverage (escalation, message variation, reset-on-success, metadata, opt-out, config parsing); updated the two default-behavior tests that pinned warn-only forever.
  • tests/run_agent/test_tool_call_guardrail_runtime.py: sequential-path soft block (no execution, no halt) and a full run_conversation recovery test (5 executed failures → 2 soft blocks with distinct wording → model answers, turn ends normally, every tool_call keeps a paired tool result).

How to Test

  1. python -m pytest tests/agent/test_tool_guardrails.py tests/run_agent/test_tool_call_guardrail_runtime.py -q → 35 passed.
  2. python -m pytest tests/agent tests/run_agent -q → passes (see PR checks).
  3. Manual repro: with default config, force any tool to fail 6+ times with identical arguments; observe the 6th attempt returns repeated_exact_failure_block guardrail JSON without executing, wording varies on subsequent attempts, and the turn continues (no guardrail_halt).

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11 (native, Python 3.11)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — module + config docstrings
  • I've updated cli-config.yaml.example if I added/changed config keys
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure-Python controller change, no platform surface
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

Production transcript excerpt (message ids from the local session DB) showing warn-only failing at scale:

terminal {"command":"cd ~/hermes-forge && npm run dev 2>&1","timeout":600}
→ {"exit_code": -1, "error": "This foreground command appears to start a long-lived server/watch process. Run it with background=true, ..."}
   [Tool loop warning: repeated_exact_failure_warning; count=11; ...]
(identical call re-sent)
   [Tool loop warning: ... count=13 ...]
(identical call re-sent)
   [Tool loop warning: ... count=15 ...]
(identical call re-sent)
   [Tool loop warning: ... count=17 ...]
... model never once changed the arguments; its own reasoning text correctly
identified the fix ("I need to set background=true") every single turn.

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have labels Jul 2, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the default guardrail gap and covering the sequential recovery path.

Problems

  • The new default is documented in cli-config.yaml.example, but the public configuration docs still describe warning-only defaults and say hard_stop_enabled is what blocks calls: website/docs/user-guide/configuration.md:1392-1410 and website/docs/user-guide/docker.md:73-82 on current main. With block_enabled: true at hermes_cli/config.py:1320 in this PR, those statements become inaccurate.

Suggested changes

  • Update both user-facing documents to distinguish default soft blocks, the block_enabled: false warn-only opt-out, and hard stops that end the turn.
  • Add a concurrent-path soft-block test; the current new runtime coverage is sequential, while current execution has separate sequential and concurrent paths in agent/tool_executor.py:482-484 and :1132-1141.

Automated hermes-sweeper review.

Comment thread hermes_cli/config.py
The block tier of the tool-loop guardrail only ever fired with
hard_stop_enabled=true, so default sessions got an identical warning
appended to every repeated failure and nothing else. On smaller local
models this is a proven non-fix: a production transcript shows the same
rejected terminal call re-sent 20+ times with warnings firing at
count=11..17, burning the max_iterations budget on a call that failed
deterministically in milliseconds.

Add a middle tier: with block_enabled (default true), an exact call that
has failed hard_stop_after.exact_failure times (or a read-only call that
returned an identical result hard_stop_after.idempotent_no_progress
times) is no longer executed; the model receives a synthetic error
result and the turn continues. Soft-block wording rotates between
attempts on purpose: replay experiments show these loops are context
self-conditioning, and identical guidance gets absorbed into the pattern
while varied guidance breaks it.

Hard-stop behavior is unchanged; block_enabled=false restores warn-only.

Fixes NousResearch#41490
@lEWFkRAD
lEWFkRAD force-pushed the fix/tool-call-loop-breaker branch from 8703d86 to 1406187 Compare July 15, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Agent loops on identical tool calls despite being blocked — need better re-prompting after repeat detection

3 participants