Skip to content

[AI] Update integration test for changed Gemini 3 thinking#15708

Merged
paulb777 merged 2 commits intomainfrom
pb-gemini3-thoughts
Jan 14, 2026
Merged

[AI] Update integration test for changed Gemini 3 thinking#15708
paulb777 merged 2 commits intomainfrom
pb-gemini3-thoughts

Conversation

@paulb777
Copy link
Member

Gemini 3 is now using thoughtSignature where Gemini 2.5 used separate thoughts for indicating thinking.

This PR updates the integration test accordingly.

More logs at #15701

Fix #15701

#no-changelog

@gemini-code-assist
Copy link
Contributor

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

@ncooke3
Copy link
Member

ncooke3 commented Jan 13, 2026

/gemini review

@paulb777 paulb777 requested a review from andrewheard January 13, 2026 15:14
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the integration test for Gemini thinking to accommodate changes in Gemini 3. The change correctly identifies that Gemini 3 uses thoughtSignature. However, there's a potential for a crash due to unsafe array access, and the test logic for verifying the presence of thoughts seems to be incorrect. I've provided a suggestion to fix both issues.

@paulb777 paulb777 force-pushed the pb-gemini3-thoughts branch from 21bd93c to 5383db2 Compare January 13, 2026 15:21
@paulb777
Copy link
Member Author

/gemini review

Copy link
Contributor

@andrewheard andrewheard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please hold off on merging. Haven't had time to understand the logic in the new check and the description "Gemini 3 is now using thoughtSignature where Gemini 2.5 used separate thoughts for indicating thinking." doesn't seem quite accurate to me.

The log in #15701 (comment) indicates that it's no longer including a thought summary (thought). In both cases "Mountain View" is a text response, not a thought summary.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates an integration test to accommodate a change in how Gemini 3 models indicate 'thinking'. The change correctly adds a check for thoughtSignature, which is the new mechanism, while maintaining the old check for thoughtParts for backward compatibility with older models. My feedback includes a minor suggestion to improve the readability of a boolean expression in the test, making the logic easier to understand at a glance.

@andrewheard
Copy link
Contributor

Are both of the examples in #15701 (comment) cases where includeThoughts is true? I'm hoping the 2.5 example is includeThoughts: true and the 3 example is includeThoughts: false (or nil).

We should verify that Gemini 3 isn't returning thought summaries where the thought bool isn't true. @paulb777 Based on the examples you just sent me, it looks like there just wasn't a thought summary for the Gemini 3 case. Potentially the model omitted it because it was such a simple question / answer prompt. I just want to make sure that thought summaries aren't implied by thoughtSignature now. It's hard to tell from the docs (Developer API, Vertex AI) because the example "I need to calculate the risk. Let me think step-by-step..." reads like a thought summary but potentially just unfortunately worded.

It's also possible that includeThoughts doesn't interact well with the legacy thinkingBudget on Gemini 3. We should see if the behaviour is any different with a thinkingLevel.

Copy link
Contributor

@andrewheard andrewheard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge for now to get the integration tests passing and add a TODO or issue to verify that thought summaries continue to work.

@paulb777 paulb777 merged commit 2737f20 into main Jan 14, 2026
67 checks passed
@paulb777 paulb777 deleted the pb-gemini3-thoughts branch January 14, 2026 22:18
JesusRojass pushed a commit to JesusRojass/firebase-ios-sdk that referenced this pull request Jan 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nightly Testing Report

3 participants