fix(agent): isolate output-cap recovery from compression

[Vansh5632]

What does this PR do?

Fixes a retry loop for output-cap errors on OpenAI-compatible providers such as local vLLM. This is not a conversation-too-large problem: the prompt fits, but the requested output plus the prompt exceeds the model context window. Previously Hermes used a fixed 64-token reserve, rebuilt the whole outer loop after each failure, and counted those retries as compression attempts. If the provider reported even 65 extra input tokens on the next request, every retry remained just over the limit and eventually reported a misleading compression failure.

The recovery now retries inside the existing API loop with the same message payload, uses an output-cap-specific retry budget and error message, and reserves 128 tokens plus any observed input growth from the prior output-cap error.

Related Issue

Fixes #61761

Related: #61846 addresses the fixed margin only. This change also keeps output-cap recovery in the inner API retry loop and separates it from compression accounting.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

The output-cap recovery now calculates a safer one-shot output limit through compute_safe_output_tokens() rather than subtracting a fixed 64 tokens. It keeps the retry in the inner API loop, so Hermes rebuilds only request kwargs and does not rerun outer-loop hooks, middleware, or compression-related work. Output-cap retries have their own bounded counter and return an output-budget-specific error if they cannot recover, instead of claiming that context compression failed. Regression tests cover the vLLM error format, +65-token input growth, stable outbound messages, no compression, and the new terminal result.

How to Test

1. Run the targeted regression suite:

   scripts/run_tests.sh tests/test_ctx_halving_fix.py tests/test_output_cap_parsing.py tests/run_agent/test_413_compression.py -q

   Result: 78 passed.

   The full scripts/run_tests.sh suite was also run. It did not finish green because 7 tests outside this change failed and 3 unrelated test files hit the per-file timeout; this PR's targeted regression suite passes.

2. Reproduce the original failure using the local mock vLLM server and throwaway repro-outcap Hermes profile. The mock reports +65 input tokens on each retry.

   Before this change, the fixed 64-token reserve never converged:

   Attempt 1: input 134,465, requested max_tokens 65,536, sum 200,001 (over by 1)
   Attempt 2: input 134,530, requested max_tokens 65,471, sum 200,001 (over by 1)
   Attempt 3: input 134,595, requested max_tokens 65,406, sum 200,001 (over by 1)
   Attempt 4: input 134,660, requested max_tokens 65,341, sum 200,001 (over by 1) -> fails
   Context length exceeded: max compression attempts (3) reached.
   

   After this change, the same smaller local repro succeeds on its second request:

   $ HERMES_HOME="$HOME/.hermes/profiles/repro-outcap" hermes -z "Say hello in one word. Do not use tools." -t ""
   ok (mock accepted request)
   

   attempt=1 input=32,832 max_tokens=32,768 sum=65,600 context=65,536 over_by=64
   POST /v1/chat/completions -> 400
   attempt=2 input=32,897 max_tokens=32,576 sum=65,473 context=65,536 over_by=-63
   POST /v1/chat/completions -> 200
   

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate (related fix(agent): grow output-cap retry margin exponentially to compensate for per-retry input-token drift (#61761) #61846 found and referenced above)
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass (full CI-parity suite was attempted; see test note above)
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Ubuntu

Documentation & Housekeeping

• N/A — no user-facing documentation changes are needed for internal retry recovery.
• N/A — no configuration keys were added or changed.
• N/A — no contributor workflow or documented architecture contract changed.
• N/A — provider-agnostic Python retry logic; no platform-specific behavior changed.
• N/A — no model tool descriptions or schemas changed.

[alt-glitch]

This was generated by AI during triage.

Fix PR for open #61761 (output-cap retry loop never converges when the fixed margin is erased by per-retry input drift). Competing/related open fix PRs for the same issue via different mechanisms: #61228 (raise the 64->512 margin + overflow-spiral guard) and #61846 (grow the margin exponentially). This PR instead uses a 128-token + observed-input-growth reserve, a dedicated output-cap retry counter (separated from compression accounting), and keeps the retry in the inner API loop. Flagging the cluster so a maintainer can pick the canonical approach. Note: this PR also bundles an unrelated 291-line .github/workflows/dashboard-memory-investigation.yml; consider isolating it.

[teknium1]

Thanks for isolating a real recovery-path bug: current main still uses the fixed 64-token reserve and charges parsed output-cap retries to compression accounting at agent/conversation_loop.py:3451-3479.

Problems

• The new handler remains after the disabled-auto-compaction guard at agent/conversation_loop.py:3021-3076. vLLM's maximum context length wording matches _CONTEXT_OVERFLOW_PATTERNS in agent/error_classifier.py:242-253, so with compression.enabled: false the turn returns compaction_disabled before it can perform the output-cap retry. The PR's added coverage uses the shared fixture that forces compression_enabled = True at tests/run_agent/test_413_compression.py:97-102.

Suggested changes

• Route a parsed output-cap error around that guard, while leaving true input-overflow behavior unchanged.
• Add a vLLM-format regression with auto-compaction disabled that verifies a reduced-cap retry succeeds without calling _compress_context.

This is an automated hermes-sweeper review.

[teknium1]

This parsed recovery still sits after the earlier compression.enabled: false overflow return (agent/conversation_loop.py:3021-3076). vLLM wording is classified as context_overflow, so that configuration exits before this retry. Route parsed output-cap errors around the guard and add a disabled-compaction regression.

[kshitijk4poor]

Closing as superseded, with thanks. Since this was filed, main took a different direction for the drift problem: #55546 made the output-cap retry path compress alongside the clamp (input drift can no longer erase the margin unnoticed), and #90563 (salvaging #89923, auto-merge armed) removes the root cause of the per-retry drift itself — vLLM's "at least N input tokens" figure is back-computed from the constraint, so the fix detects that degenerate shape and halves the cap instead of trusting it. With those two in, the adaptive-margin machinery here would be dead weight on a path that no longer drifts.

Two ideas from this PR did survive into the merged work and are credited: treating output-cap exhaustion as its own failure class rather than "compression failed", and reserving for observed growth between retries (now unnecessary post-#89923, but it informed the review). Appreciate the thorough test coverage — the +65-token drift scenario in your tests is exactly what the convergence regression test now pins.