fix(anthropic): populate reasoning_content when converting thinking blocks to OpenAI format - #28258
Conversation
…locks to OpenAI format Fixes BerriAI#27946. When LiteLLM's Anthropic-Messages -> OpenAI Chat-Completions adapter converts an assistant message that contains `thinking` content blocks, it stores them in the custom `thinking_blocks` field but never sets the standard `reasoning_content` field. Reasoning models hosted behind OpenAI-compatible endpoints — Moonshot Kimi K2.5+, DeepSeek R1, OpenAI o-series — reject multi-turn requests without `reasoning_content` on prior-turn assistant messages with: The 'reasoning_content' in the thinking mode must be passed back to the API. This change concatenates every `thinking`-type block's text into `reasoning_content` on the converted assistant message. Opaque `redacted_thinking` blocks are deliberately skipped — they're not text the upstream model can consume. `thinking_blocks` is unchanged so existing consumers stay intact. Improves on the prior approach in BerriAI#27947, which only inlined the first thinking block (Greptile review flagged the multi-block truncation). The concatenated form preserves the full reasoning chain across multi-step agent turns where models emit several thinking blocks per turn. ## Test plan - New: `test_translate_anthropic_messages_to_openai_populates_reasoning_content` — multi-block + redacted_thinking interleaved; asserts the joined string and the order of blocks. - New: `test_translate_anthropic_messages_to_openai_skips_reasoning_content_without_thinking` — ensures non-reasoning assistant messages don't get a stray `reasoning_content` key (would 400 on providers that reject unknown fields). - Existing 70 tests in `test_anthropic_experimental_pass_through_adapters_transformation.py` pass unchanged. ## End-to-end reproducer Verified live with the Claude Agent SDK (Python) -> bundled `claude` CLI -> LiteLLM proxy 1.85.0 -> Moonshot Kimi K2.5. Before the fix, every multi-turn round-trip caused Kimi to re-fetch context it had already gathered (turn 3 "which is the bank/cash account?" took 47.7 s across 4 round-trips, re-listing the chart of accounts it had returned in turn 2). LiteLLM's Moonshot adapter logged "Missing reasoning_content for assistant message" on every turn. After the fix, zero warnings across 26 calls and turn 3 dropped to 13.6 s in a single round-trip — the model now correctly references "the list I just showed you".
|
|
Greptile SummaryThis PR fixes the Anthropic→OpenAI adapter to also set
Confidence Score: 5/5Safe to merge — the change is isolated to a single conditional branch in the Anthropic→OpenAI adapter, adds no new code paths for non-thinking messages, and is covered by focused unit tests. The transformation logic is correct: ChatCompletionThinkingBlock and ChatCompletionRedactedThinkingBlock are TypedDicts (which subclass dict), so isinstance(block, dict) and .get() behave as expected. The guard No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py | Adds reasoning_content population from concatenated thinking-type blocks alongside the existing thinking_blocks field; redacted blocks are correctly excluded. Logic is sound given TypedDict inherits from dict so .get() and isinstance(block, dict) both work correctly. |
| tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py | Adds two targeted unit tests: one verifies multi-block reasoning_content concatenation (with a redacted block correctly omitted), the other guards against emitting reasoning_content on plain-text-only messages. No real network calls; pure in-memory transformation tests. |
Reviews (1): Last reviewed commit: "fix(anthropic): populate reasoning_conte..." | Re-trigger Greptile
|
🤖 litellm-agent: This PR is currently BLOCKED from merge. Score: 0/5 ❌ Why blocked:
Details: Score docked for: all Phase B agent checks non-approving (karpathy + security + coverage gap). Fix the issues above and push an update — the bot will re-review automatically.
|
The Anthropic /v1/messages -> OpenAI chat-completions pass-through adapter
(translate_anthropic_messages_to_openai) attaches the Anthropic-specific
thinking_blocks field to assistant messages unconditionally. Non-Anthropic
OpenAI-compatible backends reject it: on multi-turn conversations, models
like GLM behind an OpenAI-compatible endpoint fail with
400 invalid_request_error: Extra inputs are not permitted,
field: 'messages[1].thinking_blocks'
This breaks any multi-turn conversation once an earlier assistant turn
carried reasoning.
Verified directly against an OpenAI-compatible GLM endpoint (bypassing
litellm):
- assistant turn with thinking_blocks -> 400 (field rejected)
- assistant turn with reasoning_content -> 200 OK (the model consumes it and
reasons over the prior turn)
So the fix is to convert, not just drop: for non-Anthropic backends, strip the
raw thinking_blocks and set the OpenAI-style reasoning_content string
(concatenating the unredacted thinking blocks; redacted blocks carry no
readable text and are dropped).
Gate the thinking_blocks attachment on is_anthropic_claude_model or
is_bedrock_arn_model, the same pair of checks already used together elsewhere
in this file (e.g. for cache_control). Anthropic Claude backends (anthropic/*,
bedrock *anthropic*, vertex *claude*, and Bedrock ARNs such as Application
Inference Profiles that point at Claude) keep thinking_blocks and their signed
signatures unchanged. Everyone else gets reasoning_content instead. When the
target model is unknown (None) the prior behaviour is preserved (blocks
kept), so no existing caller changes.
This is the complete form of the half-fixes in BerriAI#27947 and BerriAI#28258, both of
which only add reasoning_content and leave thinking_blocks attached, so they
do not resolve the 400. Closes BerriAI#27946.
Fixes #27946.
Problem
When LiteLLM's Anthropic-Messages → OpenAI Chat-Completions adapter
(
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py::translate_anthropic_messages_to_openai)converts an assistant message that contains
thinkingcontent blocks,it stores them in the custom
thinking_blocksfield but never sets thestandard
reasoning_contentfield. Reasoning models hosted behindOpenAI-compatible endpoints — Moonshot Kimi K2.5+, DeepSeek R1,
OpenAI o-series — reject multi-turn requests whose prior-turn assistant
messages lack
reasoning_contentwith:(Moonshot's docs make this explicit at https://platform.kimi.ai/docs/guide/use-kimi-k2-thinking-model FAQ Q1; DeepSeek's reasoning-model guide says the same.)
Fix
When
thinking_blocksis populated, also setreasoning_contentfromthe concatenated text of every
thinking-type block (newline-joined,preserving order). Opaque
redacted_thinkingblocks are skipped —they're not text the upstream model can consume.
This improves on the prior approach in #27947, which only inlined the
first thinking block — a Greptile review on that PR
specifically flagged the single-block truncation as the remaining gap.
Concatenation preserves the full chain across multi-step agent turns
where models emit several thinking blocks per turn.
Test plan
test_translate_anthropic_messages_to_openai_populates_reasoning_content— assistant message with two real
thinkingblocks interleaved with aredacted_thinkingblock + atextblock. Asserts: (a)reasoning_contentequals the newline-joined real-thinking text, (b)
thinking_blocksstill contains all three blocks in original order, (c) redacted block
is not inlined.
test_translate_anthropic_messages_to_openai_skips_reasoning_content_without_thinking— guard against emitting an empty
reasoning_contentkey when theassistant message has no thinking. Providers that reject unknown
fields would 400.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.pypass unchanged —
make test-unitclean.End-to-end repro that motivated the patch
Verified live with the Claude Agent SDK (Python) → bundled
claudeCLI → LiteLLM proxy 1.85.0 → Moonshot Kimi K2.5. The Claude
Agent SDK path is what catches this bug in practice — it always
round-trips through the Anthropic Messages adapter even when the
upstream model is OpenAI-shape.
Before the fix: the LiteLLM proxy logged
Missing reasoning_content for assistant messageon every Moonshot call (MoonshotChatConfig.fill_reasoning_contentfalls back to a" "placeholder whenreasoning_contentis absent, which Kimi treats as empty). After the fix: zero such warnings across a 26-call session.Scope
This PR is intentionally scoped to the single behaviour fix in
transformation.pyplus its two regression tests — nothing else.(Earlier #27947 bundled unrelated security work; per CONTRIBUTING.md
"Keep scope isolated", this version separates the concerns.)
I will sign the CLA on first PR-bot prompt.