fix(agent): halt on non-retryable tool billing blockers

[dlkakbs]

Summary

When a tool returns a clear non-retryable billing/credits failure like:

• Payment Required
• Insufficient credits
• Insufficient quota
• similar exhausted-balance blockers

Hermes currently feeds that tool result back to the model and keeps iterating. In practice this can burn extra iterations/tokens, retry the same dead path, and delay a clear user-visible explanation of the blocker.

This PR changes that behavior:

• detect non-retryable billing/credits blockers in failed tool results
• escalate them to a controlled guardrail halt on the first failed tool call
• return a direct assistant message explaining that Hermes stopped because retrying would only waste iterations
• preserve the original tool result in history so the exact upstream error remains visible

Why

This is aimed at cases like web_search returning Payment Required / Insufficient credits, where continuing the same strategy is non-productive and the correct action is to surface the blocker
immediately.

Before this PR:

• tool error was appended to history
• model had to interpret it correctly
• Hermes could continue iterating and spend more tokens before clearly reporting the blocker

After this PR:

• Hermes stops on the first clearly non-retryable billing blocker
• the user gets a direct explanation immediately
• the exact tool error is still preserved in the tool result

Implementation

• added non-retryable billing blocker detection in agent/tool_guardrails.py
• wired those failures into the existing controlled-halt flow
• added a specific halt response for billing/credits blockers in run_agent.py
• added unit and runtime coverage for first-failure halt behavior

Tests

Ran:

• pytest tests/agent/test_tool_guardrails.py -q
• pytest tests/run_agent/test_tool_call_guardrail_runtime.py -q

Related

Related but not duplicate:

• [Bug]: Tool calling unstable / frequent hallucination and empty responses in v0.8.0 (especially with web_search and FireCrawl) #8993
• Agent Tool Loop Freeze & Executor Shutdown Crash - Gateway Stability Issues #10849
• [Bug]: Provider-side HTTP 402 can kill entire gateway service instead of failing only the request #5220

Historical context:

• feat: pre-call sanitization and post-call tool guardrails #1732 introduced post-call tool guardrails
• fix: add 402 billing error hint to gateway error handler (#5220) #10057 improved 402 handling on the gateway side

[teknium1]

Thanks for the thoughtful writeup and the tests — the underlying symptom (tools hitting a dead billing wall burning iterations) is real and worth fixing.

That said, I'm going to close this one. After reviewing, detecting billing blockers via substring-matching tool result strings inside the generic guardrail layer has too many false-positive failure modes to ship:

• _extract_tool_error_text falls back to returning the raw result when no error/message/detail key is present, so any tool whose output happens to contain phrases like "Payment Required" or "Insufficient credits" in a JSON blob triggers a halt. E2E-tested cases that incorrectly halt:
  • terminal showing HTTP/1.1 402 Payment Required when the user is debugging their own 402
  • read_file on a log/test fixture that contains the phrase
  • web_search returning a news snippet about credit exhaustion
  • patch failing with an error that quotes example payment text
• The halt bypasses hard_stop_enabled, which we deliberately made opt-in and warning-first in #58b89965 / #0704589. A new always-on hard stop on every install is a behavior change we don't want wired at this layer.
• The right layer for this is the tools/providers that actually emit 402s (e.g. Firecrawl in web_tools.py, browser-backend wrappers). HTTP status is unambiguous there; string-matching error text is not. fix: add 402 billing error hint to gateway error handler (#5220) #10057 did this at the gateway layer for the same reason.

If you'd like to take another pass: narrow detection to the specific tool wrappers that can actually produce 402/credit-exhausted responses, emit a typed failure from those, and have the guardrail react to the typed signal rather than a substring heuristic. Happy to review a followup along those lines.