feat(guardrails): detect repeated tool results independent of args/failure

[Vissirexa]

What does this PR do?

Adds a content-only repetition guard to ToolCallGuardrailController: detection of a tool-call loop where the arguments vary on every call but the result never changes.

The existing loop guards are keyed on tool-call signature (tool name + canonical args) or on classified failure. That misses a real loop shape observed running a local model in an unattended/gateway session: a tool call that "succeeds" with different arguments each time but keeps returning the same blocked/empty/error-page body — e.g. execute_code wrapping a web fetch against a source that consistently 404s or soft-blocks a scraper. Neither the exact-failure counter (never fires — failed is false) nor the _no_progress idempotent-result tracker (keyed by signature, and only applied to a fixed idempotent-tool allowlist) catches this.

A second, related gap: a repeated vision/multimodal tool result (shape {"_multimodal": True, "content": [...]}) was never detected as repetition because str(result) embeds a base64 image payload that's unique per call even when the meaningful content (a placeholder caption like "Image loaded into your context") is identical — one session re-loaded the same image via a vision tool 6 times with zero guard activity.

The guard follows the existing axes' design: soft warning by default, hard stop only when hard_stop_enabled is set.

Related Issue

Fixes #60084

Type of Change

• ✨ New feature (non-breaking change that adds functionality)

Changes Made

• agent/tool_guardrails.py:
  • _track_result_repetition() — a content-hash-only repetition tracker, independent of tool name, arguments, and classified failure. Warns at repeated_result warn threshold (default 3), halts at the hard-stop threshold (default 5, only when hard_stop_enabled). Results under repeated_result_min_chars (default 200) are exempt so trivial outputs ("[]", "OK") never trip it.
  • _repetition_text() — normalizes a tool result before hashing. Multimodal results keep text parts verbatim and reduce each non-text payload (e.g. an image_url data URI) to a short digest, so re-loading the same image counts as repetition while distinct images (legitimate page-scroll screenshots) count as progress. Plain string results pass through unchanged.
  • Wired into after_call() ahead of the existing failure/no-progress branches; a halt short-circuits immediately, a warn is returned if no stronger decision applies.
• hermes_cli/config.py: repeated_result thresholds under warn_after/hard_stop_after plus repeated_result_min_chars in DEFAULT_CONFIG["tool_loop_guardrails"].
• cli-config.yaml.example: document the new keys alongside the existing guardrail axes.
• website/docs/user-guide/configuration.md: short section for the new repeated_result keys.
• tests/agent/test_tool_guardrails.py: 5 new tests (see below) plus repeated_result coverage in the existing config-parsing test.

How to Test

1. pytest tests/agent/test_tool_guardrails.py -q — 18 passed. New tests:
   • test_repeated_identical_result_halts_successful_varying_arg_loop (the varying-args/fixed-result loop shape)
   • test_repeated_result_ignores_short_and_distinct_outputs (no false positives on short or genuinely distinct results)
   • test_repeated_multimodal_result_same_image_trips_guard (the vision-result blind spot)
   • test_repeated_multimodal_result_distinct_images_is_progress (distinct images never halt)
   • test_default_config_guardrail_block_matches_dataclass_defaults (DEFAULT_CONFIG stays in sync with the parser defaults)
2. Downstream consumers unaffected: pytest tests/agent/test_turn_context.py tests/run_agent/test_tool_call_guardrail_runtime.py tests/hermes_cli/test_config.py -q — all green (184 passed across the four files combined).
3. Manual: configure tool_loop_guardrails.hard_stop_enabled: true, then have an agent repeatedly fetch a URL that returns the same blocked/error page body with varying query args — the guard warns after 3 identical results and halts after 5.

Note: two pre-existing tests/agent failures on a clean upstream/main checkout (test_anthropic_adapter.py, test_coding_context.py) reproduce identically without this change — they're environment-related, not introduced here.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 15 (Apple Silicon), Python 3.11

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A (no architecture/workflow change)
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A (pure-Python logic, no file I/O / process / terminal handling)

[teknium1]

Thanks for carrying the canonical implementation for #60084; the current-main premise is real (agent/tool_guardrails.py:347-352). I found blocking issues before this can safely land.

Problems

• agent/tool_guardrails.py:430-431 accumulates each hash for the whole turn. Interleaved results such as A/B/A/C/A still warn, although the messages at :438-456 say the last calls were identical.
• Native multimodal results reach _append_guardrail_observation as dictionaries (agent/tool_executor.py:891-896, :1596-1600). A warning from the new path (agent/tool_guardrails.py:448) is then passed to append_toolguard_guidance (run_agent.py:5728-5729), whose concatenation at agent/tool_guardrails.py:491 raises for a dictionary.
• The linked issue's production session_search empty-success case remains excluded by the < repeated_result_min_chars early return at agent/tool_guardrails.py:425-427.

Suggested changes

• Track a consecutive semantic-result streak and test interrupted sequences.
• Append guardrail guidance into multimodal text parts/text_summary, then cover both executor paths.
• Add explicit behavior and coverage for short empty-success envelopes from #60084.

Automated hermes-sweeper review.

[teknium1]

This is a turn-wide histogram, not a repetition streak: A/B/A/C/A reaches the warning threshold even though every duplicate was interrupted by a different result. That conflicts with the later "last {count} tool calls" message and can halt a progressing turn. Track the previous semantic hash plus a consecutive count, resetting when the result changes.

[teknium1]

For the multimodal case this warning reaches run_agent._append_guardrail_observation, which calls append_toolguard_guidance; that helper concatenates (result or "") + suffix, but the native vision result is a dict. The third repeated image therefore raises TypeError instead of returning this warning. Append to the multimodal envelope's text part/text_summary and add an executor-path regression test.

[Vissirexa]

Thanks — all three findings were real. Addressed in 37c27b6a5:

• Interleaved counting: repetition is now a single consecutive streak (last result hash + streak counter) instead of per-turn accumulation, so A/B/A/C/A never fires and the "last N calls" wording in the warn/halt messages is literal. Any different, short, or empty result breaks the streak; tests cover interleaved and interrupted sequences (test_repeated_result_interleaved_sequences_never_warn, test_repeated_result_streak_restarts_after_interruption).
• Multimodal dict results: append_toolguard_guidance now keeps the dict shape — guidance lands as a trailing {type: text} content part and is mirrored into text_summary — instead of raising on string concatenation. Both executor call sites are covered by runtime tests (test_sequential_path_appends_guidance_to_repeated_multimodal_result, test_concurrent_path_appends_guidance_to_repeated_multimodal_result).
• Short empty-success envelopes: results below repeated_result_min_chars now still count when they are an empty-success envelope (success: true plus an explicit emptiness marker like count: 0 or an empty results/items list) from a non-mutating tool — the exact session_search loop reported on [Bug]: Tool-loop guardrails miss loops where args vary but the result never changes (incl. repeated multimodal results) #60084. Bare {"success": true} acks and mutating-tool responses stay exempt, so terse legitimate successes can't trip it.

tests/agent/test_tool_guardrails.py + tests/run_agent/test_tool_call_guardrail_runtime.py green via scripts/run_tests.sh.

[Vissirexa]

Was this closed intentionally?

#60084 is still open and the guard isn't on main — repeated_result, _track_result_repetition and _repetition_text all return nothing on current upstream/main. The one recent cross-reference, #78551, fixes degenerate repetition in streaming output, which is a different path from tool-result repetition, so it doesn't cover this.

All three sweeper findings from the 2026-07-15 review were addressed in 37c27b6a5 — consecutive-streak counting so interleaved A/B/A/C/A never fires, multimodal-safe guidance append at both executor call sites, and the short empty-success envelope case from #60084 — and CI was 28/28 green after the 07-31 rebase. It has since gone conflicting against main, which I'm happy to fix.

If you'd like it reopened, I'll rebase and push. If it was closed on scope or design grounds instead, a one-line note would help — I'd rather rework it than refile something you don't want.