feat: add thinking summaries option for gemini - #1273
Conversation
WalkthroughThis change introduces support for configuring and applying a "thoughts summary" feature for Gemini models. It adds new constants, helper functions, and UI controls to manage whether a summary of the model's thought process is included in responses, depending on model capabilities and user settings. The thinking budget handling logic in Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant UI
participant Server
participant Gemini
User->>UI: Toggle "Include thoughts summary" setting
UI->>Server: Save updated Gemini settings
User->>UI: Submit chat request
UI->>Server: Send chat request with model name and settings
Server->>Server: ApplyThinkingAdaptationSettings(request, modelName)
Server->>Gemini: Forward request with thinking budget and thoughts summary config
Gemini-->>Server: Respond with (optional) thoughts summary
Server-->>UI: Return response (with/without thoughts summary)
UI-->>User: Display reply (with/without thoughts summary)
Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (7)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (6)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
relay/channel/gemini/helper.go (1)
1-26: Consider adding performance optimization for frequent calls.Since these functions may be called frequently during request processing, consider caching results or using a more efficient lookup mechanism like a map-based approach for better performance.
Here's an optimized version using map lookups:
+var ( + thoughtsSummSupportCache = make(map[string]bool) + minThinkingBudgetCache = make(map[string]bool) +) + +func buildCache() { + for _, model := range ModelsWithThoughtsSummarySupport { + thoughtsSummSupportCache[model] = true + } + for _, model := range ModelsWithMinimumThinkingBudgetLimits { + minThinkingBudgetCache[model] = true + } +} func IsModelSupportThoughtsSummary(modelName string) bool { + // Direct lookup first + if supported, exists := thoughtsSummSupportCache[modelName]; exists { + return supported + } + + // Fall back to prefix matching for _, supportedModel := range ModelsWithThoughtsSummarySupport { if strings.HasPrefix(modelName, supportedModel) { + thoughtsSummSupportCache[modelName] = true return true } } + thoughtsSummSupportCache[modelName] = false return false }relay/channel/gemini/relay-gemini.go (1)
131-131: Consider adding null check for better robustness.While unlikely, consider adding a null check for the Gemini settings to prevent potential panics.
-if model_setting.GetGeminiSettings().IncludeThoughtsSummaryEnabled && IsModelSupportThoughtsSummary(modelName) { +geminiSettings := model_setting.GetGeminiSettings() +if geminiSettings != nil && geminiSettings.IncludeThoughtsSummaryEnabled && IsModelSupportThoughtsSummary(modelName) {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
relay/channel/gemini/constant.go(1 hunks)relay/channel/gemini/helper.go(1 hunks)relay/channel/gemini/relay-gemini.go(2 hunks)setting/model_setting/gemini.go(2 hunks)web/src/components/settings/ModelSetting.js(1 hunks)web/src/i18n/locales/en.json(1 hunks)web/src/pages/Setting/Model/SettingGeminiModel.js(2 hunks)
🔇 Additional comments (15)
setting/model_setting/gemini.go (2)
14-14: LGTM! Field addition follows established patterns.The new
IncludeThoughtsSummaryEnabledfield is properly positioned, follows naming conventions, and includes the correct JSON tag.
33-33: LGTM! Default value is consistent with frontend.The default value of
truematches the frontend implementation and makes sense for user experience - users get thoughts summaries by default when the adapter is enabled.relay/channel/gemini/constant.go (1)
29-41: LGTM! Well-documented model capability constants.The new constants are properly named, well-documented with clear comments, and include a helpful reference URL. The model lists are logically organized with Gemini 2.5 series models, which aligns with their advanced thinking capabilities.
web/src/components/settings/ModelSetting.js (1)
25-25: LGTM! Default setting follows established pattern.The new setting key follows the existing naming convention and is positioned logically with other Gemini settings. The default value of
trueis consistent with the backend configuration.web/src/i18n/locales/en.json (1)
1692-1693: LGTM! Clear and informative localization strings.The English translations are clear and accurately describe the feature functionality. The explanatory text properly communicates the conditions under which the feature is effective (model support and non-zero thinking budget).
web/src/pages/Setting/Model/SettingGeminiModel.js (2)
33-33: LGTM! Initial state value is consistent.The default value matches the backend configuration and other frontend settings, ensuring consistency across the application.
225-239: LGTM! UI component follows established patterns.The new Form.Switch component is properly implemented following the same pattern as existing switches. It includes:
- Correct field binding to match backend setting
- Proper localized labels and explanatory text
- Appropriate onChange handler that updates state
- Logical placement within the thinking adaptation section
The implementation is consistent with the rest of the codebase.
relay/channel/gemini/helper.go (2)
7-15: Function logic is correct and well-implemented.The function correctly iterates through supported models and uses prefix matching to determine compatibility. The early return pattern is efficient.
17-25: Function logic is correct and mirrors the pattern above.The function follows the same efficient pattern as
IsModelSupportThoughtsSummary, ensuring consistency in the codebase.relay/channel/gemini/relay-gemini.go (6)
81-84: Good early return pattern for disabled feature.The early return when
ThinkingAdapterEnabledis false is efficient and prevents unnecessary processing.
86-96: Thinking budget parsing logic is correct.The logic properly handles the
-thinking-<number>suffix format, validates the numeric value, and applies budget clamping. The error handling for invalid numbers is appropriate (silently ignoring invalid values).
117-128: Correct handling of -nothinking suffix with minimum budget constraints.The logic properly handles the case where models with minimum thinking budget limits cannot have zero thinking budget, which aligns with the PR objective about Google's API behavior.
130-149: Thoughts summary logic correctly implements the new decoupled behavior.The implementation properly:
- Checks if thoughts summary is enabled globally
- Verifies model support
- Applies the summary setting except when
-nothinkingis used with models that allow zero thinking budgets- Handles the case where models with minimum thinking budgets still produce thoughts content
This addresses the main PR objective of decoupling thoughts summary from thinking budget.
172-173: Excellent refactoring that improves code maintainability.Replacing the complex inline logic with a single function call makes the code much cleaner and easier to maintain. The refactoring successfully centralizes all thinking adaptation logic.
81-150: Verify that all previous inline logic has been correctly migrated.The new function appears to implement all the necessary logic, but it's important to ensure that no edge cases or behaviors from the previous inline implementation were missed during the refactoring.
Run the following script to verify that no thinking-related logic remains in the old location:
#!/bin/bash # Description: Check for any remaining thinking-related logic that might have been missed during refactoring # Search for thinking-related patterns in the relay-gemini.go file echo "Searching for any remaining thinking-related logic patterns:" rg -A 5 -B 5 "(thinking|ThinkingConfig|IncludeThoughts)" relay/channel/gemini/relay-gemini.go echo -e "\nSearching for thinking budget calculations:" rg -A 3 -B 3 "(ThinkingAdapterBudgetTokensPercentage|ThinkingBudget)" relay/channel/gemini/ echo -e "\nSearching for model suffix handling:" rg -A 3 -B 3 "(-thinking|-nothinking)" relay/channel/gemini/relay-gemini.go
9Ninety
changed the title
9Ninety
changed the title
|
@Calcium-Ion I've updated the PR to align with the latest behavior on the alpha branch |
1 participant
This pull request decouples the thoughts summary feature from the thinking budget mechanism. The primary goal is to provide users with greater flexibility by treating these as two independent settings, which aligns with their underlying design.
The Problem
Previously, enabling the thoughts summary was tightly coupled with the thinking budget. This created a situation where users who wanted a summary were forced to use the
-thinkingsuffix.This forced coupling becomes problematic when a user wants to see the thinking summary and the client also passes a
max_completion_tokensvalue. The-thinkingsuffix would then also trigger its specific thinking budget behavior. This meant the model's default, automatic thinking budget behavior would be overridden in favozr of a calculated value.This approach limited flexibility, as it was impossible to request a summary while ensuring the default thinking budget strategy remained untouched.
The Solution
This PR introduces an independent toggle for the thoughts summary. The implementation details are as follows:
IncludeThoughtsSummaryEnabledsetting is added. It is only active when the mainThinkingAdapterEnabledfeature is also turned on. If the adapter is disabled, there is no change in behavior.-thinkingsuffix). They can explicitly disable this by turning the new toggle off.By decoupling these two features, users can get a thinking summary without using the suffix. This ensures that unless the user explicitly chooses the
-thinkingsuffix, the thinking budget is not interfered with, aligning the behavior with Google's official automatic thinking strategy.A Note on Models with Minimum Thinking Budgets
For models with a minimum thinking budget limit (like
gemini-2.5-pro),includeThoughtsis still sent when the summary toggle is on, even if the-nothinkingsuffix is used. This is by design and aligns with Google's official API behavior, reflecting that these models' thinking processes cannot be fully disabled. The summary is returned in the dedicatedreasoning_contentfield and does not affect response compatibility.Summary by CodeRabbit
Summary by CodeRabbit