fix(tool-guardrails): block deterministic tool retries

[EndeavorYen]

Summary

This PR teaches the tool-loop guardrail to treat two deterministic tool failures as non-retryable for identical same-turn retries:

• memory quota overflow, where the store is already full and the same add/replace request cannot succeed unchanged.
• image_generate empty_response, where the provider returned no image_generation_call result for the completed request.

The first failure still reaches the model with a clear recovery hint. If the model repeats the same tool call with identical arguments in the same turn, Hermes blocks the duplicate execution before calling the tool again and returns a synthetic guardrail result.

Fixes #35120
Fixes #35121

Why This Matters

Hermes already has general repeated-failure guardrails, but these two failure modes are more specific than "the same tool failed a few times." They are deterministic until something changes:

• A full memory store will remain full until content is shortened, replaced, removed, or the write is skipped.
• An image_generate response with no image_generation_call is not improved by immediately replaying the exact same request inside the same turn.

Without a first-class classification, the agent can spend extra iterations repeating a call that has no new information to offer. In gateway contexts this also produces noisy logs, slower user-facing recovery, and weaker guidance for the model about what action would actually help.

The expected contribution is modest but practical: reduce avoidable tool-loop churn while preserving the agent's ability to recover by changing arguments or strategy.

Existing Upstream Work

This PR is intentionally scoped around the guardrail layer. It does not replace the existing image-generation provider work:

• fix(image_gen): drop unsupported tool_choice from openai-codex request #19979 addresses the tool_choice request-shape failure for the openai-codex image generation path, but explicitly leaves the later empty_response / no image_generation_call stage unresolved.
• fix: surface exhausted empty model responses #23505 improves exhausted empty final responses from the main agent loop. That is related at the symptom level, but it does not classify image_generate tool results or prevent same-turn duplicate image tool executions.

For memory quota overflow, I did not find an existing PR that treats the quota-exceeded result as a non-retryable same-turn tool failure.

Behavior

After this change:

1. The first matching failure returns a warn decision.
2. Hermes appends recovery guidance to the original tool result so the model sees what to change next.
3. The same tool call with the same canonical arguments is recorded as non-retryable for the rest of the turn.
4. A repeated identical call is blocked before execution and returned as a synthetic guardrail result.
5. A changed call still gets a fresh attempt.

This preserves useful retries while avoiding deterministic duplicate work.

Scope

This PR only changes the pure tool-call guardrail controller and its tests:

• agent/tool_guardrails.py
• tests/agent/test_tool_guardrails.py

It does not change the image-generation provider request shape, memory storage limits, or global hard-stop defaults.

User And Developer Impact

For users, the visible effect should be less "stuck retrying" behavior when Hermes hits a full memory store or an image provider that completed without producing an image call. The agent should move sooner to an actionable recovery path: shorten/replace/remove memory content, skip the memory write, retry image generation later, or change the prompt/model/provider.

For developers and operators, the value is sharper failure semantics in a central guardrail primitive:

• The same structured failure can be tested without invoking provider APIs.
• Logs and synthetic guardrail metadata include stable codes:
  • memory_quota_exceeded_non_retryable
  • image_generate_empty_response_non_retryable
• The fix applies to both CLI and gateway paths that already use the tool guardrail controller.

Validation

• ./scripts/run_tests.sh tests/agent/test_tool_guardrails.py - 15 passed
• ./scripts/run_tests.sh tests/run_agent/test_tool_call_guardrail_runtime.py tests/agent/test_tool_guardrails.py - 24 passed
• uv run --with ruff ruff check agent/tool_guardrails.py tests/agent/test_tool_guardrails.py - passed
• git diff --check - passed

[teknium1]

Thanks for this, and for the thorough writeup — the guardrail approach is sound and the wiring is correct.

Closing it though, because the memory half doesn't actually cover the failure mode the issue reports. The matcher keys on "exceed the limit" / "exceeds the limit", which only appears in the add quota error. The replace quota error — the one quoted first in #35121 (Replacement would put memory at 1,431/1,375 chars) — is emitted as:

Replacement would put memory at {new_total}/{limit} chars. Shorten the new content or remove other entries first.

No "exceed the limit" substring, so replace quota overflow falls straight through to the generic guardrail and the dead-loop the issue describes isn't blocked. The PR's replace-style test (test_memory_quota_failure_blocks_identical_retry_without_hard_stop) fabricates an error using "would exceed the limit" wording the real code never produces, so it passes green while the real path is still broken. #35121 also explicitly requires "unit coverage for add and replace operations" as acceptance.

We'll fold the non-retryable-classification idea into a fix that matches both real error strings (add + replace) and tests against the actual emitted text. Your design and authorship informed it — appreciate the contribution.