fix(anthropic): round-trip thinking blocks to OpenAI backends on /v1/messages - #37953
Conversation
The experimental /v1/messages adapters lost prior-turn reasoning three different ways once the request left for an OpenAI-shaped backend. On the Responses path, thinking blocks were flattened into output_text inside the assistant message, so the model read its own private reasoning back as visible prose and no reasoning item was ever sent. They now become Responses reasoning input items, grouped by signature so summary parts that arrived as one item go back as one item. The response direction stops hardcoding signature=None and carries the reasoning item id, which is what lets the next turn regroup them; the streaming wrapper emits the matching signature_delta. On the chat completions path the adapter attached thinking_blocks but never set reasoning_content, so Moonshot and DeepSeek substituted a single-space placeholder and other providers sent nothing. It is now derived from the thinking blocks. With use_chat_completions_url_for_anthropic_messages and a model that itself bridges to /v1/responses, the assistant message was dropped whole: reasoning, text, and all. That branch now emits the reasoning items and the message content alongside the tool calls. Fixes #24985
Greptile SummaryThe PR preserves Anthropic thinking blocks when routing
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/responses_adapters/transformation.py | Adds bidirectional conversion between Anthropic thinking blocks and Responses reasoning items while preserving visible assistant text and tool calls. |
| litellm/litellm_core_utils/prompt_templates/common_utils.py | Adds shared helpers for extracting readable thinking text and constructing chat or Responses reasoning payloads. |
| litellm/completion_extras/litellm_responses_transformation/transformation.py | Preserves assistant reasoning and visible content when bridging Chat Completions messages into Responses input. |
| litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py | Populates reasoning_content when translating Anthropic assistant thinking blocks to chat messages. |
| litellm/llms/anthropic/experimental_pass_through/responses_adapters/streaming_iterator.py | Emits streamed thinking blocks with an empty signature instead of fabricating a provider signature. |
| litellm/llms/azure_ai/chat/transformation.py | Strips unsupported reasoning_content from Azure AI Foundry messages. |
| litellm/llms/fireworks_ai/chat/transformation.py | Strips reasoning_content alongside other unsupported assistant fields. |
| litellm/llms/hosted_vllm/chat/transformation.py | Removes reasoning_content from assistant messages before sending them to hosted vLLM. |
Reviews (5): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ning_content A reasoning item id is not an Anthropic signature. Passing it off as one got the block replayed to Anthropic and Bedrock as if it were real, and every backend that verifies signatures rejected the turn. Thinking blocks now come back unsigned, and the streaming path no longer emits a signature_delta for them. Azure AI Foundry, Fireworks, and vLLM reject unknown message fields, so they now strip reasoning_content alongside thinking_blocks the way Mistral already did. The thinking-block helpers take ChatCompletionThinkingBlock and ChatCompletionRedactedThinkingBlock instead of loose mappings.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 32bf1ab. Configure here.
…its function_call
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 5317a5a. Configure here.
…itellm_fix_24985_thinking_roundtrip
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Null summary becomes literal None
- Changed
_summary_part_textto coerce a nulltextvalue to an empty string viaor ""so callers correctly skip it instead of emitting a thinking block with the literal string "None".
- Changed
- ✅ Fixed: Thinking-only turns drop reasoning
- Added a fallback
elif role == "assistant"branch that emits_reasoning_input_items(msg)when the assistant turn has no content and no tool calls so prior-turn reasoning survives the chat-to-Responses bridge.
- Added a fallback
You can send follow-ups to the cloud agent here.
|
|
| assistant_message["thinking_blocks"] = thinking_blocks | ||
| reasoning_content = reasoning_content_from_thinking_blocks(thinking_blocks) | ||
| if reasoning_content: | ||
| assistant_message["reasoning_content"] = reasoning_content |
There was a problem hiding this comment.
Low: Thinking blocks bypass input guardrails
A user can place content in an assistant thinking block that the Anthropic guardrail translation does not add to its scanned texts, then have this code forward it to reasoning-capable backends as reasoning_content. Include readable thinking-block text in the pre-call guardrail extraction and write any masked result back before promoting it here.
There was a problem hiding this comment.
"bypass input guardrails"
Pre-existing surface: these blocks already went upstream via thinking_blocks and as flattened output_text. Scanning thinking text is a guardrail-layer follow-up.
PR overviewThis pull request updates the Anthropic One security issue remains open: readable content in thinking blocks can be forwarded as reasoning content without being included in pre-call guardrail scanning. This permits guardrail evasion, although the resulting impact depends on the configured guardrails and downstream backend behavior. No issues have yet been addressed. Open issues (1)
Fixed/addressed: 0 · PR risk: 4/10 |
…nking-only assistant turns
…trip' into litellm_fix_24985_thinking_roundtrip # Conflicts: # litellm/llms/anthropic/experimental_pass_through/responses_adapters/transformation.py
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e02f34b. Configure here.

TLDR
Problem this solves:
/v1/messagesdrops prior-turn thinking on OpenAI-family backendsthinking_blocksbut neverreasoning_contentthinking, gpt-5 loses the whole assistant turnHow it solves it:
reasoninginput itemreasoning_contentfrom the thinking blocksUser Flow
Before: a Claude Code user on an OpenAI-family model behind LiteLLM has last turn's reasoning thrown away, so the follow-up is either refused or answered blind
ANTHROPIC_BASE_URLtohttps://litellm-domainand start Claude Code on a reasoning model"thinking": {"type": "enabled", "budget_tokens": 4000}thinkingblock, atextblock, and atool_useblock, and Claude Code prints the thinking on screentool_resultthinking is enabled but reasoning_content is missing; on newer Moonshot models a 200 comes back whose answer starts the reasoning over from nothingAfter: the same session carries last turn's reasoning across, so every follow-up is accepted and answered in context
ANTHROPIC_BASE_URLtohttps://litellm-domainand start Claude Code on a reasoning model"thinking": {"type": "enabled", "budget_tokens": 4000}thinkingblock, atextblock, and atool_useblock, and Claude Code prints the thinking on screentool_resultRelevant issues
Fixes #24985
Linear ticket
Resolves LIT-6006
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup, identical on both sides. Two proxies, one per adapter path, each with a callback that writes the exact upstream request body to a file
config_responses.yaml(the default path for OpenAI):config_chat.yaml(the opt-in path):Every case runs the same two real turns against the real provider, the way Claude Code does:
Case 5 swaps curl for the real client: Claude Code 2.1.241 running interactively in tmux, keystrokes sent with
tmux send-keysand the screen read back withtmux capture-pane, pointed at the proxy withANTHROPIC_BASE_URL=http://127.0.0.1:$PORT,ANTHROPIC_AUTH_TOKEN=sk-qa6006, andANTHROPIC_MODELset to the proxy's model name. Two prompts typed in order, so the second request replays the first turn's thinking block exactly as Claude Code builds it:Before (7a1afa1)
Case 1: OpenAI on the default Responses path
MODEL=gpt5-resp, run the two turns aboveCase 2: Moonshot on the Chat Completions path
MODEL=kimi, run the two turns abovereasoning_contentis a single space, so the model is told it reasoned about nothing:Case 3: gpt-5 on the Chat Completions path, re-bridged to /v1/responses
MODEL=gpt5-chat, run the two turns aboveCase 4: streaming on the Responses path, the shape Claude Code actually sends
MODEL=gpt5-resp, same two turns with"stream": true, reassembling turn 1's blocks from the SSE eventsCase 5: real Claude Code, driven interactively
gpt5-resp, Chat proxy onkimireasoning_contentis sent at all (with a tool call in the turn, this same path sends the" "placeholder from case 2)After (e02f34b)
Case 1: OpenAI on the default Responses path
MODEL=gpt5-resp, run the same two turnsCase 2: Moonshot on the Chat Completions path
MODEL=kimi, run the same two turnsreasoning_contentnow carries the real prior reasoning instead of a placeholder:Case 3: gpt-5 on the Chat Completions path, re-bridged to /v1/responses
MODEL=gpt5-chat, run the same two turnsCase 4: streaming on the Responses path, the shape Claude Code actually sends
MODEL=gpt5-resp, same two turns with"stream": true, reassembling turn 1's blocks from the SSE eventsCase 5: real Claude Code, driven interactively
reasoning_contentnow carries the real prior reasoningType
🐛 Bug Fix
Caveats (if any)
id; the Responses API 404s on any it did not mintredacted_thinkingblocks are still dropped on the request path; they hold no readable textthinking_blocks; they now stripreasoning_contentalongside it, the way Mistral already didreasoning_contentand a signed thinking block. That is pre-existing, filed as [Bug]: Gemini receives the previous turn's reasoning twice when an assistant message has both reasoning_content and a signed thinking block #37973: replaying an assistant turn with both fields produces the same three parts on this branch and on the merge base. What changes here is that the/v1/messagespath now reaches it, since it sets the same pair a native Gemini response already setsthinking_blocksfor being a list. That estimate feeds context-window checks and router fallbacks, and it now over-counts for the providers that strip the field before the wirereasoning_content, and the revert that removed the earlier round-trip (fix(hosted_vllm): remove thinking_blocks and convert list content to strings #30475) names thinking blocks converted to content lists rather than the plain string. It is stripped here for consistency with the other two, and no vLLM endpoint was available to test either wayFinal Attestation
Note
Medium Risk
Touches multi-provider message transformation for reasoning/tool history, which can change what models see on follow-up turns. No auth or data-store changes; risk is incorrect request shaping if a conversion edge case is missed.
Overview
Preserves prior-turn Anthropic thinking when
/v1/messagesis proxied to OpenAI-family backends, so follow-ups keep reasoning instead of dropping it, flattening it into visible assistant text, or losing the whole assistant turn (including tool calls).Responses path: assistant
thinkingblocks become Responsesreasoningitems (no fabricatedid). Consecutive thinking blocks collapse into one item; a tool call splits them. Responses reasoning maps back to unsigned thinking blocks, notoutput_text.Chat Completions path: thinking text is also copied into
reasoning_contentso models like Moonshot/DeepSeek do not get a blank placeholder. The chat→Responses bridge now keeps assistant text next to tool calls and still emits reasoning for thinking-only turns. Storedreasoning_itemswin over re-derived thinking.Azure AI, Fireworks, and hosted vLLM now strip
reasoning_contentthe same way they already stripthinking_blocks. Streaming thinking blocks open with an emptysignatureand never stream a stand-in signature.Reviewed by Cursor Bugbot for commit e02f34b. Bugbot is set up for automated code reviews on this repo. Configure here.