fix(provider): preserve reasoning_content across consecutive assistant messages - #9404
Conversation
|
We reported this exact behavior in #9397 with extensive evidence:
The root cause described here — Looking forward to testing this fix. Thank you for the PR! |
ae1d571 to
ff4578b
Compare
ff4578b to
48964d0
Compare
|
@shafqatevo was your account compromised? the latest commit contains apparently malicious code. |
|
My account was not compromised. The malicious code was not in my commit — it was already in the repo's history. Here's what I found: The malware lives in commit Commit My PR branch (
However, my fork's Summary: The obfuscated code was injected by If |
|
@jamadeo Could you please reopen this PR? The malicious code was not from my commit — as I explained above, it was already in the repo history (commit The fix in this PR is needed for the DeepSeek reasoning_content 400 error (#9402, #9397). Both CI and live testing against DeepSeek pass. Would appreciate if you can take another look. Thanks! |
|
+1 to reopening. This fix is proven needed by live testing on v1.37.0 (see #9397 for detailed analysis). The malware was in the repo history (commit We've identified 2 additional code paths where @jamadeo — please take another look. |
|
Thanks for reopening the linked issue, but this PR will remain closed. We welcome another fix in a fresh PR. I'll leave the forensic work to the experts, but to clarify a few facts:
@shafqatevo I recommend looking into https://opensourcemalware.com/blog/polinrider-attack |
Problem
DeepSeek's thinking-mode API returns a 400 when the conversation history contains an assistant message with
tool_callsbut noreasoning_content:This happens when DeepSeek streams a response that produces three consecutive assistant messages:
In
format_messages_with_options,pending_assistant_reasoningaccumulates the reasoning from message 1 and merges it into message 2 (the text message). But it then clears the pending buffer — so message 3 (the tool-call message) receives noreasoning_contentand DeepSeek rejects the request.Fixes #9402
Fixes #9397
Follow-up to #8857 (which added the
preserves_thinkingflag but missed this clearing bug)Fix
Remove the premature
pending_assistant_reasoning.clear()from the merge path informat_messages_with_options. The buffer is already properly cleared at the top of the loop whenever a non-assistant message (user / tool) arrives, which prevents stale reasoning from leaking across turn boundaries.The change is provider-agnostic — it's correct for all providers that set
preserve_thinking_context: true(DeepSeek, Kimi, OpenRouter, vLLM), and has no effect on providers where it'sfalse(OpenAI direct, Anthropic).Tests
Two new regression tests:
test_format_messages_reasoning_survives_text_before_tool_call— the exact failure scenario: thinking-only → text-only → tool-call. Asserts both emitted messages carryreasoning_content.test_format_messages_reasoning_cleared_across_user_message_boundary— verifies stale reasoning does NOT leak across a user message boundary (the original safety property).All 71 existing format tests continue to pass (the 1
test_create_request_o1_defaultfailure is pre-existing and unrelated).