Skip to content

fix(gemini): deduplicate reasoning parts when assistant message has both reasoning_content and thinking_blocks - #37977

Open
Priyanka06081218 wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
Priyanka06081218:fix/gemini-reasoning-duplicate-parts
Open

fix(gemini): deduplicate reasoning parts when assistant message has both reasoning_content and thinking_blocks#37977
Priyanka06081218 wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
Priyanka06081218:fix/gemini-reasoning-duplicate-parts

Conversation

@Priyanka06081218

Copy link
Copy Markdown

Fixes #37973

TLDR

When a Gemini multi-turn conversation replays an assistant message that carries both reasoning_content and thinking_blocks, LiteLLM was sending the same reasoning text to Gemini twice. This PR makes the two branches mutually exclusive.

What happened

_gemini_convert_messages_with_history had two independent if blocks:

if reasoning_content is not None:
    assistant_content.append(PartType(thought=True, text=reasoning_content))
if thinking_blocks is not None:
    for block in thinking_blocks:
        ...
        assistant_content.append(PartType(thoughtSignature=block_signature, text=block_thinking_str))

Both ran when both fields were set. LiteLLM's own Gemini response transformation sets both fields from the same parts, so any plain multi-turn Gemini conversation hit this as soon as the model returned a thoughtSignature. Gemini billed the replayed reasoning twice.

This is the pre-existing bug explicitly listed in the Caveats of #37953:

"Gemini sends a prior turn's reasoning twice when a message carries both reasoning_content and a signed thinking block ... replaying an assistant turn with both fields produces the same three parts on this branch and on the merge base."

Fix

Changed if thinking_blocks / if reasoning_content to if thinking_blocks / elif reasoning_content. Signed blocks take priority — they are lossless (they carry the full reasoning text plus the signature). reasoning_content is emitted only when thinking_blocks is absent.

if thinking_blocks is not None:
    for block in thinking_blocks:
        ...
elif reasoning_content is not None:
    assistant_content.append(PartType(thought=True, text=reasoning_content))

Tests

Added two tests to tests/llm_translation/test_prompt_factory.py:

  • test_gemini_no_duplicate_reasoning_parts_when_both_fields_present — asserts exactly one signed part is emitted and no plain thought part appears when both fields are set
  • test_gemini_reasoning_content_fallback_when_no_thinking_blocks — asserts the fallback path still works correctly when only reasoning_content is present

…avoid duplicate reasoning parts

When an assistant message has both reasoning_content and thinking_blocks,
the previous code ran two independent if-branches and sent the same
reasoning text to Gemini twice — once as a plain thought part and once
as a signed thoughtSignature part.

Prefer thinking_blocks when present; fall back to reasoning_content only
when thinking_blocks is absent. The signed block is lossless: it already
carries the full reasoning text plus the signature.

Fixes BerriAI#37973
@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR prevents duplicate Gemini reasoning parts by preferring signed thinking blocks over the plain reasoning fallback.

  • Makes signed thinking-block and reasoning_content serialization mutually exclusive.
  • Adds regression coverage for messages containing both representations and for the reasoning-only fallback.

Confidence Score: 4/5

The PR needs the fallback condition corrected before merging because present-but-unusable thinking blocks can now discard valid reasoning content.

The new null-based branch suppresses reasoning_content even when the thinking-block loop emits nothing, causing assistant reasoning context to be lost during replay.

Files Needing Attention: litellm/llms/vertex_ai/gemini/transformation.py

Important Files Changed

Filename Overview
litellm/llms/vertex_ai/gemini/transformation.py Deduplicates reasoning serialization, but suppresses the fallback when the provided thinking-block list contains no replayable signed block.
tests/llm_translation/test_prompt_factory.py Covers the signed-block and absent-block cases but omits empty or unusable thinking-block collections.

Reviews (1): Last reviewed commit: "fix(gemini): prefer signed thinking_bloc..." | Re-trigger Greptile

Comment on lines 899 to 900
text=block_thinking_str,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unusable blocks suppress reasoning fallback

When thinking_blocks is empty or contains no block with both thinking text and a signature, the loop emits nothing but the non-null check still skips reasoning_content, causing valid reasoning context to be discarded during replay.

Knowledge Base Used: Provider adapters and capabilities

@codecov

codecov Bot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing Priyanka06081218:fix/gemini-reasoning-duplicate-parts (1676c34) with litellm_internal_staging (aae36f4)

Open in CodSpeed

@Priyanka06081218

Copy link
Copy Markdown
Author

@mateo-berri — This fixes the Gemini deduplication bug you listed as a known remaining caveat in #37953. The change is one file (litellm/llms/vertex_ai/gemini/transformation.py), three targeted tests, and CI is green across all required checks. Would appreciate a review when you get a chance.

@Priyanka06081218

Copy link
Copy Markdown
Author

@tin-berri, @yucheng-berri, @mateo-berri — This fixes the Gemini deduplication bug you listed as a known remaining caveat in #37953. The change is one file (litellm/llms/vertex_ai/gemini/transformation.py), three targeted tests, and CI is green across all required checks. Would appreciate a review when you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gemini receives the previous turn's reasoning twice when an assistant message has both reasoning_content and a signed thinking block

1 participant