fix(gemini): deduplicate reasoning parts when assistant message has both reasoning_content and thinking_blocks - #37977
Conversation
…avoid duplicate reasoning parts When an assistant message has both reasoning_content and thinking_blocks, the previous code ran two independent if-branches and sent the same reasoning text to Gemini twice — once as a plain thought part and once as a signed thoughtSignature part. Prefer thinking_blocks when present; fall back to reasoning_content only when thinking_blocks is absent. The signed block is lossless: it already carries the full reasoning text plus the signature. Fixes BerriAI#37973
Greptile SummaryThis PR prevents duplicate Gemini reasoning parts by preferring signed thinking blocks over the plain reasoning fallback.
Confidence Score: 4/5The PR needs the fallback condition corrected before merging because present-but-unusable thinking blocks can now discard valid reasoning content. The new null-based branch suppresses Files Needing Attention: litellm/llms/vertex_ai/gemini/transformation.py
|
| Filename | Overview |
|---|---|
| litellm/llms/vertex_ai/gemini/transformation.py | Deduplicates reasoning serialization, but suppresses the fallback when the provided thinking-block list contains no replayable signed block. |
| tests/llm_translation/test_prompt_factory.py | Covers the signed-block and absent-block cases but omits empty or unusable thinking-block collections. |
Reviews (1): Last reviewed commit: "fix(gemini): prefer signed thinking_bloc..." | Re-trigger Greptile
| text=block_thinking_str, | ||
| ) |
There was a problem hiding this comment.
Unusable blocks suppress reasoning fallback
When thinking_blocks is empty or contains no block with both thinking text and a signature, the loop emits nothing but the non-null check still skips reasoning_content, causing valid reasoning context to be discarded during replay.
Knowledge Base Used: Provider adapters and capabilities
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@mateo-berri — This fixes the Gemini deduplication bug you listed as a known remaining caveat in #37953. The change is one file (litellm/llms/vertex_ai/gemini/transformation.py), three targeted tests, and CI is green across all required checks. Would appreciate a review when you get a chance. |
|
@tin-berri, @yucheng-berri, @mateo-berri — This fixes the Gemini deduplication bug you listed as a known remaining caveat in #37953. The change is one file (litellm/llms/vertex_ai/gemini/transformation.py), three targeted tests, and CI is green across all required checks. Would appreciate a review when you get a chance. |
Fixes #37973
TLDR
When a Gemini multi-turn conversation replays an assistant message that carries both
reasoning_contentandthinking_blocks, LiteLLM was sending the same reasoning text to Gemini twice. This PR makes the two branches mutually exclusive.What happened
_gemini_convert_messages_with_historyhad two independentifblocks:Both ran when both fields were set. LiteLLM's own Gemini response transformation sets both fields from the same parts, so any plain multi-turn Gemini conversation hit this as soon as the model returned a
thoughtSignature. Gemini billed the replayed reasoning twice.This is the pre-existing bug explicitly listed in the Caveats of #37953:
Fix
Changed
if thinking_blocks/if reasoning_contenttoif thinking_blocks/elif reasoning_content. Signed blocks take priority — they are lossless (they carry the full reasoning text plus the signature).reasoning_contentis emitted only whenthinking_blocksis absent.Tests
Added two tests to
tests/llm_translation/test_prompt_factory.py:test_gemini_no_duplicate_reasoning_parts_when_both_fields_present— asserts exactly one signed part is emitted and no plainthoughtpart appears when both fields are settest_gemini_reasoning_content_fallback_when_no_thinking_blocks— asserts the fallback path still works correctly when onlyreasoning_contentis present