Skip to content

fix(agent): strip reasoning replay fields for strict chat-completions providers - #64296

Open
Soju06 wants to merge 1 commit into
NousResearch:mainfrom
Soju06:upstream-pr/pr12-strict-replay
Open

fix(agent): strip reasoning replay fields for strict chat-completions providers#64296
Soju06 wants to merge 1 commit into
NousResearch:mainfrom
Soju06:upstream-pr/pr12-strict-replay

Conversation

@Soju06

@Soju06 Soju06 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Problem

Assistant messages in agent history can carry two non-standard reasoning replay fields: reasoning (trajectory reasoning text) and reasoning_details (OpenRouter's unified structured-reasoning format, e.g. signed Anthropic thinking blocks). reasoning_details is deliberately kept on API-bound messages so OpenRouter can maintain multi-turn reasoning continuity — but it is not part of the OpenAI Chat Completions message schema, and it currently reaches every chat-completions endpoint.

Strict OpenAI-compatible providers (Fireworks, Mistral, and many self-hosted/proxy gateways) reject any request whose replayed assistant messages contain these fields with HTTP 400 Extra inputs are not permitted, field: 'messages[N].reasoning_details'. Once a session contains such a turn, every subsequent request fails. The common trigger is a mixed-provider session: turns produced via OpenRouter (or an Anthropic reasoning model) persist reasoning_details, and a model switch or fallback to a strict direct provider makes the whole history unreplayable. The existing reactive recovery only fires on Anthropic-flavored thinking_signature error text, so these generic 400s classify as non-retryable format errors — and even when recovery applies, it costs a failed round trip per session.

Change

  • ChatCompletionsTransport.convert_messages() now strips reasoning / reasoning_details from the wire copy of messages, gated on the target endpoint via a new _base_url_consumes_reasoning_details() helper: the fields are kept for endpoints that consume the replay (openrouter.ai, nousresearch.com) and stripped for everything else, including when no base_url is supplied (safe default for strict providers). This mirrors the existing keep/strip pattern used for the Gemini extra_content thought_signature.
  • build_kwargs() passes base_url through to convert_messages() (it is already provided by both the profile and legacy call paths).
  • The max-iterations summary path in agent/chat_completion_helpers.py hand-builds messages and bypasses the transport; it already mirrored the other schema-foreign strips (tool_name, codex_*, timestamp) but leaked reasoning_details. The same gated strip is mirrored there.

Correctness notes

  • OpenRouter/Nous behavior is unchanged: reasoning_details is still replayed for reasoning continuity (signed Anthropic thinking blocks on tool-call turns require it). A new test pins this.
  • Only the wire copy is sanitized — convert_messages() uses the existing copy-on-write scheme, so persisted history keeps the fields and future turns on routes that consume them can still replay reasoning state. A test asserts the source list is untouched.
  • reasoning was already popped on the main-loop api_messages build for all providers ("trajectory storage only"); stripping it in the transport extends the same guarantee to direct _build_api_kwargs() entry points without changing main-loop behavior.
  • reasoning_content is intentionally not touched: it is a provider-schema echo-back field (DeepSeek/Kimi thinking mode) reconciled per-provider by reapply_reasoning_echo_for_provider().

Tests

New:

  • tests/agent/transports/test_chat_completions.py: strips reasoning/reasoning_details for a strict base_url and when base_url is unknown; keeps them for openrouter.ai / inference-api.nousresearch.com; source messages never mutated.
  • tests/run_agent/test_strict_api_validation.py: end-to-end through _build_api_kwargs() — stripped for a Fireworks agent (history retained, reasoning_content untouched), kept for an OpenRouter agent.

Runs (all green):

  • tests/agent/transports/, tests/run_agent/test_strict_api_validation.py
  • tests/run_agent/test_provider_parity.py, test_deepseek_reasoning_content_echo.py, test_thinking_only_sanitizer.py, test_thinking_sig_recovery_persistence.py, test_deepseek_v4_thinking_live.py, tests/agent/test_compressed_summary_metadata.py, tests/plugins/model_providers/
  • Full tests/agent/ + tests/run_agent/ (one pre-existing environment-dependent failure in tests/agent/lsp/test_broken_set.py reproduces identically on a clean checkout of main).

🤖 Generated with Claude Code

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API P2 Medium — degraded but workaround exists labels Jul 14, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

This PR strips reasoning/reasoning_details from assistant messages when sending to strict chat-completions providers.

Looks Good

  • Gated strip preserves reasoning continuity for OpenRouter/NousResearch.
  • Copy-on-write scheme ensures persisted history unchanged.
  • Thorough PR description with correctness notes.
  • Comprehensive test coverage added.

No Issues Found


Reviewed by Hermes Agent

@Soju06
Soju06 force-pushed the upstream-pr/pr12-strict-replay branch 2 times, most recently from f07a6b7 to 25034a3 Compare July 14, 2026 14:43

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing both the normal transport and the iteration-limit bypass; the replay leak is present on current main.

Problems

  • agent/transports/chat_completions.py:280 puts reasoning and reasoning_details behind the same OpenRouter/Nous allow-list. Current main treats them differently: agent/conversation_loop.py:832-835 always removes trajectory-only reasoning, while retaining reasoning_details for replay. The direct _build_api_kwargs() path would therefore still send reasoning to the two allowed endpoints.
  • PR_BODY.md:1 is a duplicate PR description, not a repository artifact.

Suggested changes

  • Strip reasoning unconditionally; retain only reasoning_details for the capability-gated endpoints, and update the OpenRouter/Nous regression test accordingly.
  • Add a regression test for agent/chat_completion_helpers.py:1738, since the summary path bypasses the transport and is changed separately.
  • Exclude PR_BODY.md from the salvage.

Automated hermes-sweeper review.

Comment thread agent/transports/chat_completions.py Outdated
):
out_msg = mutable_msg()
out_msg.pop("codex_reasoning_items", None)
out_msg.pop("codex_message_items", None)
out_msg.pop("tool_name", None)
out_msg.pop("effect_disposition", None)
out_msg.pop("timestamp", None) # #47868 — leak into strict providers
if strip_reasoning_fields:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reasoning should not share the reasoning_details allow-list. Current main always removes trajectory-only reasoning before dispatch (agent/conversation_loop.py:832-835); retain only reasoning_details for OpenRouter/Nous and pop reasoning unconditionally here.

@Soju06
Soju06 force-pushed the upstream-pr/pr12-strict-replay branch from 25034a3 to 5b85eb8 Compare July 16, 2026 02:19
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 16, 2026
@Soju06
Soju06 force-pushed the upstream-pr/pr12-strict-replay branch from 5b85eb8 to 420a894 Compare July 16, 2026 15:08
@Soju06

Soju06 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Addressed all three points:

  • reasoning is now stripped unconditionally in ChatCompletionsTransport.convert_messages(), matching the main-loop replay build; only reasoning_details remains behind the OpenRouter/Nous capability check, so the direct _build_api_kwargs() path no longer sends reasoning to those endpoints. The OpenRouter/Nous regression test now asserts reasoning is dropped while reasoning_details is retained.
  • Added summary-path regression tests for handle_max_iterations (the transport-bypassing path in agent/chat_completion_helpers.py): strict providers get neither field on the wire, OpenRouter keeps reasoning_details only, and persisted history is untouched in both cases.
  • Removed PR_BODY.md from the branch history.

Rebased on current main; full transport, strict-validation, and run_agent test suites pass (530 tests).

@Soju06

Soju06 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@teknium1 Gentle ping — all points from the review here have been addressed (summary in the comment above), the branch is rebased on current main, and CI is green. Ready for another look whenever convenient.

… providers

Strict OpenAI-compatible Chat Completions providers reject assistant
replay messages carrying non-standard fields (reasoning,
reasoning_details, and Codex Responses bookkeeping) with HTTP 400.
Strip them at the transport boundary for providers that opt into
strict validation, keeping the fields in the persisted transcript for
providers that consume them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants