Bugfix: recover from structured reasoning budget exhaustion

[HiddenPuppy]

Summary

• detect thinking-budget exhaustion for chat-completions responses that return structured reasoning fields without visible text
• record usage before length/empty-response recovery so Hermes can use real token pressure for recovery decisions
• compact context before continuation/prefill when a reasoning-only response already shows the conversation is over the compaction threshold
• add regression tests for structured reasoning truncation and proactive compression retry paths

Root Cause

Hermes already had a thinking-budget guard for inline <think> content, but OpenAI-compatible models like glm-5-turbo often return reasoning via reasoning_content/reasoning_details with empty content. Those responses skipped the guard, then walked into continuation or prefill retries that grew context further without ever giving compression a chance.

Notes

• This PR focuses on the still-real structured-reasoning recovery gap from [Bug] Thinking model (glm-5-turbo) reasoning tokens exhaust output budget, producing empty responses with no recovery path #9344.
• The separate stale _last_content_with_tools fallback issue from [Bug]: _last_content_with_tools fallback bypasses empty-response retries, causing silent agent loop termination mid-task #7968 is tracked independently in Bugfix: avoid stale tool-turn fallback on empty responses #9432 and is intentionally not duplicated here.

Validation

• git diff --check
• python3 -m py_compile run_agent.py tests/run_agent/test_run_agent.py
• Full pytest was not runnable locally in this environment because the machine does not currently have the repo's required Python 3.11 + dev test toolchain installed.

Closes #9344

[vvvwww-ai]

Complementary approach: boost max_tokens retry (works alongside compression)

Great PR — the _has_structured_reasoning detection is exactly right. I've been running a similar patch locally for a Qwen3.6-35B-A3B (Opus-distilled) on LM Studio, and wanted to share a complementary recovery strategy that addresses a different failure scenario.

The scenario compression doesn't cover

Compression helps when accumulated context causes reasoning to scale proportionally (the glm-5-turbo case in #9344). But there's a second failure mode: low max_tokens budget on short-task auxiliary services (title generation, approval, session search, etc.) where the model's distilled reasoning consumes 50-100% of the output budget even with minimal context.

Test data (Qwen3.6-35B-A3B, LM Studio, local)

With a complex prompt and varying max_tokens:

max_tokens	content empty?	reasoning_tokens	reasoning %	finish_reason
256	3/5 runs (60%)	255/256	100%	length
512	occasional	261/512	51%	length
1024	occasional	123/1024	12%	length
2048+	never	131/2048	6%	length/stop

When content is empty, the model still produced substantial reasoning_content (500-1300 chars). The existing _thinking_exhausted path fires correctly after this PR's detection fix, but compression doesn't help here — the context is already small (the problem is the output budget, not the input).

Proposed complementary logic

After the _thinking_exhausted check, boost max_tokens and retry before giving up:

# In the finish_reason == "length" block, after _thinking_exhausted detection:
_rc_exhausted = (
    not _trunc_has_tool_calls
    and agent._extract_reasoning(_trunc_msg)  # checks reasoning/reasoning_content/reasoning_details
    and (_trunc_content is None or not _trunc_content.strip())
)

if _rc_exhausted and reasoning_content_exhausted_retries < 2:
    reasoning_content_exhausted_retries += 1
    _rc_boost = (agent.max_tokens or 4096) * (2 ** reasoning_content_exhausted_retries)
    _rc_requested_cap = agent._requested_output_cap_from_api_kwargs(api_kwargs)
    if _rc_requested_cap is not None:
        _rc_boost = max(_rc_boost, _rc_requested_cap * (2 ** reasoning_content_exhausted_retries))
    agent._ephemeral_max_output_tokens = min(_rc_boost, max(65536, _rc_requested_cap or 0))
    # Don't append the empty response; re-run with larger budget
    continue

Verification

Scenario	Result
mt=256, empty → boost ×2 (512)	✅ 869 chars output
mt=64, empty → boost ×2 (128) still empty → boost ×4 (256)	✅ 231 chars output

Why both strategies coexist

Strategy	Fixes	When to use
Compression (this PR)	High context → reasoning scales	Long conversations, gateway sessions
Boost max_tokens (proposed)	Low budget → reasoning eats everything	Auxiliary services, cron jobs, short tasks

The detection logic (_has_structured_reasoning) is shared — only the recovery path differs. A combined approach would:

1. If context is high → compress first (this PR's _maybe_compress_after_empty_reasoning_response)
2. If context is low / compression didn't help → boost max_tokens and retry
3. If both exhausted → return the thinking-budget error

Happy to help refine or submit a follow-up PR if the maintainers prefer to keep this PR focused on compression.

[teknium1]

Thanks for isolating the structured-reasoning truncation case. The premise is still live: current main's length handler only treats inline `` content as exhaustion (agent/conversation_loop.py:1788-1804), while structured fields are handled later only for stop-response prefill (`agent/conversation_loop.py:4920-4952`).

Problems

• The proposed compression pressure uses total_tokens / last_completion_tokens (run_agent.py:8814 and the later prefill addition). Current main intentionally uses prompt tokens only because reasoning/completion tokens do not consume the next request context and cause premature compaction (agent/conversation_loop.py:4748-4754).
• The target loop was extracted after this branch: run_agent.py:5775-5787 forwards to agent/conversation_loop.py, whose accounting path has since gained additional behavior (agent/conversation_loop.py:2044-2106).

Suggested changes

• Salvage the narrow structured-reasoning detection and recovery into agent/conversation_loop.py:1756-1835 rather than transplanting the retired run_agent.py accounting block.
• Use the existing prompt/request-pressure rules and add current-head regression tests for both reasoning_content and reasoning_details length responses.

Automated hermes-sweeper review.

[teknium1]

total_tokens includes completion/reasoning output, but current compaction logic intentionally treats only prompt tokens as next-request context pressure because reasoning-heavy completions otherwise trigger premature compression. Please port this recovery using the current conversation_loop.py prompt/request-pressure rules instead.