Skip to content

fix: use structured reasoning field directly for thinking models - #4467

Closed
djt71 wants to merge 1 commit into
NousResearch:mainfrom
djt71:fix/thinking-model-reasoning-field
Closed

fix: use structured reasoning field directly for thinking models#4467
djt71 wants to merge 1 commit into
NousResearch:mainfrom
djt71:fix/thinking-model-reasoning-field

Conversation

@djt71

@djt71 djt71 commented Apr 1, 2026

Copy link
Copy Markdown

Summary

  • Thinking models (Kimi K2.5, DeepSeek-R1, etc.) put their full response in the reasoning_content API field with an empty content field. The existing empty-content retry loop wastes 3 API calls per response before falling back to the reasoning text — retrying doesn't change deterministic model behavior.
  • Adds an early exit in the empty-content handler: when the structured reasoning_content or reasoning field is populated (distinct from inline <think> tags in content), use it immediately as the response.
  • Changes macOS LaunchAgent KeepAlive from conditional (SuccessfulExit: false) to unconditional (true), so the gateway restarts after clean exits — not just crashes.

Context

Discovered during a 48-hour soak test running Kimi K2.5 via OpenRouter as a cron-driven orchestrator. Analytical prompts (read a file + summarize findings) triggered the issue on nearly every execution:

  • 28 think-block-only warnings
  • 22 wasted retry API calls (each a full round-trip to OpenRouter)
  • 6 max-retry exhaustions before the existing fallback kicked in

The existing #3444 fix (thinking-budget exhaustion detection) and #3010 (GLM reasoning-only handling) address related thinking-model edge cases but don't cover this specific pattern: a populated reasoning_content API field with empty content.

Before

⚠️  Response only contains think block with no content after it
   Reasoning:  Model: kimi-k2.5 via OpenRouter. Vault check: all 31 checks passed...
🔄 Retrying API call (1/3)...
⚠️  Response only contains think block with no content after it
🔄 Retrying API call (2/3)...
⚠️  Response only contains think block with no content after it
❌ Max retries (3) for empty content exceeded.
Using reasoning as response content (model wrapped entire response in think tags).

After

ℹ️  Thinking model returned response in reasoning field — using directly.

Test plan

  • Tested with Kimi K2.5 via OpenRouter (cron job: read vault-check log + summarize)
  • Heartbeat prompts (simple, non-analytical) unaffected — still produce content directly
  • Existing retry path preserved for genuinely empty responses (no structured reasoning)
  • python -c "import py_compile; py_compile.compile('run_agent.py', doraise=True)" passes
  • Verify with DeepSeek-R1 (same reasoning_content field pattern expected)
  • Verify non-thinking models (GPT-4o, Claude, Llama) are unaffected (no reasoning_content field → skip early exit, reach existing retry path as before)

🤖 Generated with Claude Code

Thinking models (Kimi K2.5, DeepSeek-R1, etc.) consistently return
their full response in the reasoning_content API field with an empty
content field. The existing empty-content retry loop wastes 3 API calls
per response before falling back to the reasoning text — retrying
doesn't change deterministic model behavior.

This adds an early exit: when the structured reasoning_content or
reasoning field is populated (not inline <think> tags), use it
immediately as the response instead of retrying.

Also changes the macOS LaunchAgent KeepAlive from conditional
(SuccessfulExit: false) to unconditional (true), so the gateway
restarts after clean exits — not just crashes.

Tested with Kimi K2.5 via OpenRouter over a 48-hour soak test where
28 think-block warnings and 22 wasted retry API calls were observed
before the fix. After the fix: zero retries, correct response on
first call.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@teknium1

teknium1 commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for the detailed investigation and soak test data — that's solid work. However, the empty-content behavior from thinking models isn't deterministic — retrying can and does produce proper responses with content in the content field. The existing retry loop (3 attempts before falling back to reasoning text) is the intended behavior here, since it gives the model a fair chance to produce a well-formed response before accepting the fallback.

Closing this one, but appreciate the contribution!

@teknium1 teknium1 closed this Apr 1, 2026
@djt71
djt71 deleted the fix/thinking-model-reasoning-field branch April 3, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants