fix(agent): preflight output-token-budget gate to prevent context-overflow death-loop

[7MS8]

Summary

Fixes the same failure shape as #61761 (still open, no merged fix): a turn
requesting a large max_tokens against an already-large prompt hits HTTP 400
(input_tokens + max_tokens > context_length). The existing reactive retry
(parse available_output_tokens from the error, shrink max_tokens, retry)
chased a moving target on my setup — the input token estimate grew ~65 tokens
per attempt for reasons I could not fully isolate, exactly offsetting the
shrink and pinning the requested total at context_length + 1 on every
attempt:

Attempt 1: input 111,073, max_tokens 20,000 -> sum 131,073 (over by 1)
Attempt 2: input 111,138, max_tokens 19,935 -> sum 131,073 (over by 1)
Attempt 3: input 111,203, max_tokens 19,870 -> sum 131,073 (over by 1)
Attempt 4: input 111,268, max_tokens 19,805 -> sum 131,073 (over by 1)

After max_compression_attempts the whole turn was lost with no result
(Max compression attempts (3) reached.). Setup: local vLLM, Qwen3.6-35B,
context window 131072.

Change

Adds a preflight clamp on every attempt (not just reactively after a
400): compute the current request's token pressure using the existing
request_pressure_tokens estimator, cross-checked against a conservative
char/2.2 ratio, and cap max_tokens against the actual remaining budget
before the call goes out. It can only tighten an already-set reactive
ephemeral cap, never loosen it, so it composes safely with the existing
output-cap retry path — including #61846's exponential-margin approach,
which addresses the same symptom from the reactive side. I'd see the two as
complementary rather than competing (this one tries to avoid the 400 in the
first place; #61846 makes the reactive recovery converge faster if a 400
happens anyway).

Also adds debug-only instrumentation (logger.debug, never touches the
payload) to help pin down the exact source of the per-attempt input growth
next time this path fires. I was not able to isolate it with certainty on my
setup — I ruled out three candidates with code-level evidence (see below) but
couldn't verify a fourth, more plausible one (a local llm_request
middleware coupled to the same shrinking max_tokens value) without side
effects on a live production session, so I left it as an open question with
logging in place to answer it definitively next occurrence.

Candidates ruled out:

1. Error-to-model feedback mechanism only runs post-response, unreachable
   before a 400.
2. System/ephemeral/prefill prompts are turn-stable, not reassigned in this
   retry path.
3. No messages.append/.insert call site is reachable between the
   output-cap break and the retry continue; the message-repair helper
   only merges/drops, never adds.

Verification

• Full existing test suite green: 2014 passed, 3 skipped, 0 failed.
• New isolated repro (mocking the retry arithmetic with a small context
  limit) reproduces the death-loop with the old code and confirms
  convergence with the new gate, including an edge case where almost no
  output budget remains.

Note

I run a local/customized deployment (Pythia project) with some plugins that
hook llm_request, so I can't 100% rule out that my ~65-token/attempt drift
is specific to my setup rather than universal — but the failure shape
(pinned at exactly context_length + 1 every retry) matches #61761's trace
closely enough that I think this is worth sharing regardless.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #58869: both patches preflight-clamp requested output against estimated request/context budget to stop the same vLLM/Qwen retry loop. #58869 is the broader current open implementation with request-level coverage and lower-bound parsing repair.

[teknium1]

Thanks for documenting the vLLM/Qwen failure shape. Current main still applies the parsed output cap only after a rejected request in agent/conversation_loop.py:4575-4626, so the underlying retry-budget concern remains relevant.

Problems

• The submitted diff changes only agent/conversation_loop.py; it does not include the claimed regression. Existing coverage in tests/run_agent/test_run_agent.py:3859-3944 covers a static retry cap, not input growth across retries.
• The proposed char/2.2 safeguard needs redesign for current main. agent/conversation_loop.py:1759-1768 now defines total_chars as approx_tokens * 4 and labels it a rough logging proxy, so it is not an independent character-based cross-check.
• The branch base is 2,240 commits behind HEAD, and the preflight/compression path has moved substantially. The nearby open fix: clamp chat completion output cap to context window #58869 also targets this output-cap family at the outbound chat-completions layer.

Suggested changes

• Add an end-to-end changing-input retry regression.
• Reapply the guard at the final outbound-kwargs layer using a real independent estimate, then reconcile it with fix: clamp chat completion output cap to context window #58869.

Automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

Five PRs address related context-window overflow paths: #14858 guards against an untrusted context-tier shrink, #58869 clamps outbound output budgets and repairs vLLM lower-bound parsing, #58981 adds the same clamp while propagating custom-provider caps across agent constructors, #61228 adds custom-profile clamping plus retry-burst recovery, and #70152 implements a conversation-loop preflight clamp with diagnostics.

Related pull requests

• [codex] Guard untrusted context probe shrink #14858 [closed] related — (+120/-0) — n/a: The diff narrowly preserves a known context length when an untrusted probe tier falls below the active prompt estimate, with unit and conversation-level coverage. Although closed, it remains relevant as the narrower reference implementation described by contributor alt-glitch for the probe-shrink failure related to [codex] Prevent long-context probe collapse #14499.
• fix: clamp chat completion output cap to context window #58869 related — (+151/-2) — n/a: The diff adds an outbound chat-completions clamp and stops interpreting vLLM's “at least N input tokens” lower bound as an exact budget, directly targeting the reported retry staircase. Keep-open salvage remains appropriate, but the maintainer-bot review identifies conflicting existing parser/recovery expectations that this diff does not update and requests end-to-end clamp/compression coverage.
• fix: preserve custom provider output caps #58981 related — (+557/-8) — n/a: The diff contains the same parser repair and outbound clamp as fix: clamp chat completion output cap to context window #58869, plus preservation and propagation of custom-provider output caps through config, cron, background review, delegate, and TUI paths. The contributor keep_open review supports salvaging those remaining propagation paths, while requiring removal of the new delegation cap knob, preservation of main commit 8727e67 behavior, and separation of the unrelated OpenViking batching change.
• fix(agent): robust handling of context-window overflow on strict endpoints #61228 related — (+355/-21) — n/a: The diff clamps the custom-profile default, detects non-converging output-cap reductions, routes them to compression, raises the retry margin, and adds transport and loop regressions. The later diff and contributor response explicitly address the keep_open review's blocking concerns by resetting the burst counter after accepted responses and adding the requested intervening-success and immediate-non-convergence tests.
• fix(agent): preflight output-token-budget gate to prevent context-overflow death-loop #70152 related — (+75/-1) — n/a: The diff adds a conversation-loop preflight clamp and retry diagnostics for the same vLLM/Qwen output-budget loop, but its char/2.2 check derives from the existing rough proxy and it adds no regression file. Despite the maintainer-bot keep_open verdict, contributor alt-glitch identifies it as a duplicate of fix: clamp chat completion output cap to context window #58869, whose outbound-layer diff supplies the overlapping clamp with request-level tests and lower-bound parsing repair.

Duplicates

#70152 is substantially duplicated by #58869. #58981 incorporates the same clamp and parser changes as #58869 but has distinct custom-provider cap-propagation work; #61228 overlaps on strict-endpoint clamping while adding separate retry-burst and compression behavior.

Suggested consolidation

Close #70152 as a duplicate of #58869; this explicitly departs from its keep_open review because the visible diffs show the same preflight-budget remedy, while #58869 places it at the final outbound layer and includes request-level coverage. Keep #58869 and #61228 open with their documented salvage paths, and require author action on #58981 to rebase onto main, remove the prohibited delegation cap surface, split OpenViking batching, and retain only the non-overlapping provider-cap propagation.

Cross-PR triage: Reviewed 5 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 93 kB of PR diffs, 12 kB of issue/PR text, 6 kB of discussion (9 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.