Skip to content

fix(gemini): strip thought_signature when falling back to Gemma (#36907) - #49218

Open
hbentel wants to merge 1 commit into
NousResearch:mainfrom
hbentel:fix/gemini-thought-signature-gemma-36907
Open

fix(gemini): strip thought_signature when falling back to Gemma (#36907)#49218
hbentel wants to merge 1 commit into
NousResearch:mainfrom
hbentel:fix/gemini-thought-signature-gemma-36907

Conversation

@hbentel

@hbentel hbentel commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Problem

A gemini-3-flash thinking-mode tool call attaches extra_content.google.thought_signature. When a fallback chain (or subagent / mid-session model switch) routes that tool-call history to a Google model that does not use the Gemini-3 thinking format — e.g. gemma-4-31b-it — the request 400s:

HTTP 400 INVALID_ARGUMENT: Request contains an invalid argument

Repro: primary gemini-3-flash makes a tool call (emits extra_content), then fails retriably so fallback activates to gemma-4-31b-it; the first fallback turn 400s. Also reproducible via any cross-provider replay of Gemini-3 tool-call history through a non-Gemini-3 model.

Root cause

convert_messages() in agent/transports/chat_completions.py decides whether to strip the Gemini-specific extra_content via _model_consumes_thought_signature(), which returned:

return "gemini" in m or "gemma" in m

So any gemma-* model matched and the stale thought_signature was kept — but Gemma rejects the unknown field. The function's own docstring says the field should be kept "only when the target model is itself Gemini-family"; Gemma is a separate model family that doesn't use the Gemini-3 thinking format, so the "gemma" term was the bug.

Fix

Narrow the predicate to genuine Gemini models:

return "gemini" in m

Now Gemma (and any non-Gemini model that inherited stale Gemini extra_content earlier in a mixed-provider session) strips the field, while real Gemini-3 targets still replay it. The Gemini native adapter reads extra_content on the response side only, so request-side stripping here does not affect legitimate Gemini replay.

Test plan

  • New test_convert_messages_strips_extra_content_for_gemmagemma-4-31b-it and google/gemma-3-27b strip the signature (original message list untouched).
  • Updated test_convert_messages_keeps_extra_content_for_gemini to assert keep for gemini-3-pro / google/gemini-3-pro-preview / gemini-3-flash (no longer asserts the buggy gemma-keeps behavior).
  • scripts/run_tests.sh tests/agent/transports/test_chat_completions.py — 82/82 pass.

Closes #36907

🤖 Generated with Claude Code

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) P2 Medium — degraded but workaround exists labels Jun 19, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the predicate and preserving the copy-on-write contract.

Problems

  • Current main deliberately restored the opposite predicate at agent/transports/chat_completions.py:128: commit 63ddd022a203e48ba0e4617f41c2661f3415e69e calls the Gemini-only behavior a reverted change requiring its own justification. Commit e7648d59129ab1709ed111eca3b1d5f11408adac then restored gemma-3-27b keep-coverage at tests/agent/transports/test_chat_completions.py:110 so a future narrowing fails loudly. This needs maintainer resolution against the claimed Gemma API behavior.
  • The shared predicate also controls AIAgent._sanitize_tool_calls_for_strict_api() (run_agent.py:5585-5588), but this PR tests only ChatCompletionsTransport.

Suggested changes

  • Add a parity regression for the run_agent.py sanitizer with a Gemma target, including preservation of canonical history.
  • Retain provider-backed evidence for the asserted Gemma 400 INVALID_ARGUMENT before reversing the intentionally restored current-main contract.

Automated hermes-sweeper review.

"""
m = str(model or "").lower()
return "gemini" in m or "gemma" in m
return "gemini" in m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shared predicate is also used by AIAgent._sanitize_tool_calls_for_strict_api() (run_agent.py:5585-5588), including the direct summary request path. Please add a run-agent parity regression for a Gemma target as well as this transport-level test.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 14, 2026
@hbentel
hbentel force-pushed the fix/gemini-thought-signature-gemma-36907 branch from d5115d4 to 4c7aae9 Compare July 19, 2026 20:38
@hbentel

hbentel commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Addressed both review points.

Provider-backed justification for reversing the deliberate gemini or gemma contract. This PR is the standalone follow-up the salvage of #40632 explicitly invited — commit 63ddd022a dropped the Gemma narrowing with the note that it "reverts deliberate e8c3ac2; belongs in its own PR with its own justification if pursued." The justification is #36907, which carries a concrete provider repro: a gemini-3-flash thinking-mode tool call emits extra_content.google.thought_signature, and when a cross-model fallback / subagent / mid-session switch replays that history to gemma-4-31b-it, Google returns HTTP 400 INVALID_ARGUMENT — Gemma does not use the Gemini-3 thinking format, so it never produced (and cannot consume) that signature. The only way the field reaches a Gemma request is a stale cross-model inheritance, which must be stripped. So the narrowing to "gemini" in m is correct: keeping Gemma in the consume-list keeps the #36907 400 live.

I've flipped the deliberately-restored gemma-3-27b keep-assertion into a strip-assertion (test_convert_messages_strips_extra_content_for_gemma, covering gemma-4-31b-it and google/gemma-3-27b) — the "fails loudly" guard from e7648d591 firing exactly as intended, now that there's justification.

Parity coverage for the second consumer of the predicate. The shared _model_consumes_thought_signature also gates AIAgent._sanitize_tool_calls_for_strict_api() in run_agent.py. Added test_sanitize_tool_calls_strips_extra_content_for_gemma, asserting Gemma strips extra_content on that path too and that canonical history is preserved (copy-on-write leaves the source tool_call dict intact).

Rebased on current main; 185 transport + provider-parity tests pass.

…Research#36907)

A gemini-3-flash thinking-mode tool call carries
extra_content.google.thought_signature. When a cross-model fallback (or
subagent / mid-session model switch) routes that tool-call history to
gemma-4-31b-it, the chat-completions transport left the field in place and
Gemma returned HTTP 400 INVALID_ARGUMENT — Gemma does not use the Gemini-3
thinking format.

_model_consumes_thought_signature() decided "keep" on "gemini" in m OR
"gemma" in m, so any gemma-* model matched and the stale signature leaked.
The docstring already said the field should be kept "only when the target
model is itself Gemini-family" — Gemma isn't, so the "gemma" term was the bug.
Narrow it to "gemini" so Gemma (and any non-Gemini model that inherited stale
Gemini extra_content) strips the field.

The Gemini native adapter reads extra_content on the response side only, so
request-side stripping here does not affect genuine Gemini replay.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@hbentel
hbentel force-pushed the fix/gemini-thought-signature-gemma-36907 branch from 4c7aae9 to e0a9698 Compare August 1, 2026 20:27
@hbentel

hbentel commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (@ 470cf66); was CONFLICTING after main advanced ~3.7k commits, now MERGEABLE. Conflict was the usual additive test-file keep-both in tests/agent/transports/test_chat_completions.py. Production predicate unchanged (_model_consumes_thought_signature"gemini" in m). Green: 90 tests across test_chat_completions.py + test_provider_parity.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Cross-model fallback 400 INVALID_ARGUMENT: gemini-3-flash → gemma-4-31b-it leaks thought_signature

3 participants