fix(streaming): map reasoning to reasoning_content in Delta for gpt-oss providers - #22803
Conversation
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes a bug where reasoning content was silently dropped during streaming for OpenAI-compatible providers (e.g., Cerebras, Groq gpt-oss models) that return
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/types/utils.py | Added reasoning → reasoning_content field mapping in Delta.__init__ before super().__init__(). Clean, minimal change that normalizes field names for gpt-oss providers. |
| tests/test_litellm/types/test_types_utils.py | Added comprehensive unit test covering: basic mapping, direct reasoning_content, precedence when both present, non-leakage assertion, and absent-field case. No network calls — pure mock tests. |
| poetry.lock | Routine regeneration from Poetry 2.2.0 to 2.3.2 with markers format changes. Unrelated to the bug fix. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["Provider Streaming Response<br/>(e.g., Cerebras, Groq)"] -->|"delta.reasoning = '...'"| B["Delta.__init__(**kwargs)"]
B --> C{"reasoning_content<br/>is None?"}
C -->|"Yes"| D["Pop 'reasoning' from params<br/>Set reasoning_content = reasoning"]
C -->|"No"| E["Keep existing<br/>reasoning_content"]
D --> F["super().__init__(**params)<br/>(reasoning removed from params)"]
E --> F
F --> G{"reasoning_content<br/>is not None?"}
G -->|"Yes"| H["self.reasoning_content = value"]
G -->|"No"| I["del self.reasoning_content<br/>(OpenAI spec compliance)"]
Last reviewed commit: e48b7ae
7fea8a2 to
b6bd95b
Compare
|
|
b6bd95b to
c673752
Compare
…ss providers Providers like Cerebras return delta.reasoning in streaming responses for gpt-oss models, but LiteLLM's Delta class expects reasoning_content. This causes reasoning content to be silently dropped during streaming. Fixes BerriAI#13300
c673752 to
e48b7ae
Compare
0c2e6b5
into
BerriAI:litellm_oss_staging_03_04_2026
…ing-content-delta fix(streaming): map reasoning to reasoning_content in Delta for gpt-oss providers
Relevant issues
Fixes #13300
Pre-Submission checklist
tests/litellm/directorymake test-unitType
🐛 Bug Fix
Changes
Providers like Cerebras return
delta.reasoningin streaming responses for gpt-oss models, but LiteLLM'sDeltaclass expectsreasoning_content. This causes reasoning content to be silently dropped during streaming.The fix adds a 3-line mapping in
Delta.__init__that convertsreasoning→reasoning_contentwhenreasoning_contentis not already set. This is generic and works for any OpenAI-compatible provider that uses thereasoningfield name.Tested with:
gpt-oss-120b(streaming + non-streaming)gpt-oss-20b(streaming + non-streaming)gpt-oss-20b(streaming + non-streaming)Files changed:
litellm/types/utils.py: Mapreasoningkwarg toreasoning_contentinDelta.__init__tests/test_litellm/types/test_types_utils.py: Add unit test for the mapping