openrouter: send prior-turn assistant reasoning on the request path - #11
Open
jwbron wants to merge 1 commit into
Open
openrouter: send prior-turn assistant reasoning on the request path#11jwbron wants to merge 1 commit into
jwbron wants to merge 1 commit into
Conversation
The Anthropic adapter converts each assistant `thinking` content block into an entry on `assistant_message["thinking_blocks"]`, but nothing on the OpenRouter request path consumes that field: this module names reasoning only on the response side (`reasoning` -> `reasoning_content` on streaming deltas), and `reasoning_details` appears nowhere under `llms/` or `litellm_core_utils/`. `OpenAIGPTConfig.transform_request` puts `messages` straight into the request body, so `thinking_blocks` is transmitted as a field no provider reads and every historical assistant turn reaches the model with its reasoning missing. For a model whose chat template re-renders prior thinking that is a malformed history rather than a lost optimisation. Poolside Laguna renders `'<think>' + message.reasoning|message.reasoning_content + '</think>'` per previous assistant turn, so each one arrives as a literal empty `<think></think>`; Poolside's model card warns this degrades follow-up behaviour. Map the blocks onto `reasoning_content` in `transform_request` and drop `thinking_blocks` so no unknown field goes out. OpenRouter accepts `reasoning`, `reasoning_content` and `reasoning_details` interchangeably and documents this for exactly this multi-turn tool-calling case; the plain string form is used because `reasoning_details` exists to carry encrypted or summarised blocks and the adapter produces neither. Anthropic `signature` values are not forwarded. `redacted_thinking` blocks carry no plaintext and contribute nothing; a whitespace-only result emits no field at all rather than re-creating the empty `<think></think>`. Assistant messages only, input not mutated, and a block that cannot be parsed leaves its message untouched rather than failing the request. Verified against a real Claude Code /v1/messages body routed through the proxy: before, the outgoing payload carried `thinking_blocks` and no reasoning field; after, it carries `reasoning_content` and no `thinking_blocks`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
On the
/v1/messagesroute (Anthropic-format clients such as Claude Code, proxied at anOpenRouter model), assistant reasoning from prior turns is never sent back to the model.
thinkingcontent blocks intoassistant_message["thinking_blocks"](
llms/anthropic/experimental_pass_through/adapters/transformation.py, in_translate_anthropic_messages_to_openai).llms/openrouter/chat/transformation.pyhas no request-path consumer for that field. Itsonly reasoning line is response-path (
reasoning->reasoning_contenton streaming deltas).reasoning_detailsappears nowhere underllms/orlitellm_core_utils/.OpenAIGPTConfig.transform_requestdropsmessagesstraight into the request body, sothinking_blocksis transmitted to the provider as a field no one reads, and every historicalassistant turn arrives with its reasoning missing.
This is not egg-specific or fork-specific: it is present in stock
BerriAI/litellmand predatesevery change on this fork. PR #8 touched only
get_supported_openai_params, which acts onoptional_paramsand cannot reach a message field.Why it matters
For a model whose chat template re-renders prior thinking, this is a malformed history rather than
a lost optimisation. Poolside Laguna renders
for every previous assistant turn, so each one arrives as a literal empty
<think></think>.Poolside's model card warns that this degrades follow-up behaviour. Any OpenRouter reasoning model
with a template of that shape is affected.
The change
Map assistant
thinking_blocksontoreasoning_contentinOpenrouterConfig.transform_request,and remove
thinking_blocksso no unknown field is transmitted.OpenRouter accepts
reasoning,reasoning_contentandreasoning_detailsinterchangeably on anassistant message and documents this for exactly this multi-turn tool-calling case; Poolside's
template reads
message.reasoning/message.reasoning_content. The plain string form is usedbecause
reasoning_detailsexists to carry encrypted or summarised blocks and the Anthropicadapter produces neither.
Behaviour, all covered by tests:
redacted_thinkingblocks carry opaquedata, not text, and contribute nothing;<think></think>this exists to remove;signaturevalues are not forwarded (they mean nothing to OpenRouter);case is the current behaviour;
Direction
Worth stating explicitly, because the neighbouring code is easy to confuse with this: PR #4 is
response-path (provider -> client, how reasoning is streamed back). This is request-path
(client -> provider). They are adjacent, not the same thing.
Verification
Beyond the unit tests, verified end to end with a real Claude Code
/v1/messagesbody (capturedfrom a live two-turn tool-calling exchange, assistant message carrying
['text', 'thinking', 'tool_use']) replayed through a LiteLLM proxy with the provider endpointcaptured:
['role', 'thinking_blocks', 'tool_calls'], no reasoning field['reasoning_content', 'role', 'tool_calls'], nothinking_blocksOn a live two-turn exchange against
deepseek/deepseek-v4-pro, the reasoning restored to thefollow-up turn measured +28 tokens on an assistant turn that was 118 tokens without it (1.24x);
the magnitude scales with how much the model reasons.
tests/test_litellm/llms/openrouter/passes in full (154 tests), including the 27 added here.A separate upstream PR to
BerriAI/litellmis a follow-up, not part of this.