Skip to content

feat: gemini-3-pro #2243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Calcium-Ion merged 2 commits into from
Nov 19, 2025
Merged

feat: gemini-3-pro #2243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bc1f747
feat: gemini-3-pro
seefs001 Nov 18, 2025
77c4c3e
feat: MediaResolution && VideoMetadata
seefs001 Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 17 additions & 7 deletions dto/gemini.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ func (r *GeminiChatRequest) SetTools(tools []GeminiChatTool) {
type GeminiThinkingConfig struct {
IncludeThoughts bool `json:"includeThoughts,omitempty"`
ThinkingBudget *int `json:"thinkingBudget,omitempty"`
// TODO Conflict with thinkingbudget.
// ThinkingLevel json.RawMessage `json:"thinkingLevel,omitempty"`
}

func (c *GeminiThinkingConfig) SetThinkingBudget(budget int) {
Expand Down Expand Up @@ -182,8 +184,12 @@ type FunctionCall struct {
}

type GeminiFunctionResponse struct {
Name string `json:"name"`
Response map[string]interface{} `json:"response"`
Name string `json:"name"`
Response map[string]interface{} `json:"response"`
WillContinue json.RawMessage `json:"willContinue,omitempty"`
Scheduling json.RawMessage `json:"scheduling,omitempty"`
Parts json.RawMessage `json:"parts,omitempty"`
ID json.RawMessage `json:"id,omitempty"`
}
Comment on lines 186 to 193

@coderabbitai coderabbitai Bot Nov 18, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

Additions to GeminiFunctionResponse are structurally sound; verify field names vs Gemini spec

Using json.RawMessage for WillContinue, Scheduling, Parts, and ID is consistent with the existing pattern of treating provider‑specific metadata as opaque JSON. Tags with omitempty avoid emitting unused fields.

Please double‑check that the JSON keys (willContinue, scheduling, parts, id) exactly match what Gemini 3 returns, since any mismatch would silently drop these fields during unmarshalling.

🌐 Web query:

Confirm the exact JSON field names and shapes for Gemini 3 function/tool response metadata (willContinue, scheduling, parts, id) in the latest Gemini API documentation.

💡 Result:

Summary of exact JSON field names and shapes (Gemini / Generative Language API):

  1. id — string. Optional. Unique id of the function response. [1]

  2. name — string. Required (function name). [1]

  3. response — object (JSON object / Struct). Required. Arbitrary JSON containing the function's output. [1]

  4. parts — array of FunctionResponsePart objects. Optional. Each part is an object with a data union (e.g., inlineData: { mimeType: string, data: string }) or other part types. [1]

  5. willContinue — boolean. Optional. Signals the function call continues (generator behavior) for NON_BLOCKING functions; false ends the generator. (Docs also show it may be nullable in some SDK schemas.) [1][2]

  6. scheduling — enum (Scheduling). Optional. Allowed enum values: SCHEDULING_UNSPECIFIED, SILENT, WHEN_IDLE, INTERRUPT. Controls how the response is scheduled for NON_BLOCKING calls. [1]

References

  • Google Generative Language / Gemini API — FunctionResponse and related types. [1]
  • Google ADK / SDK schema excerpt (shows willContinue nullable / schema shape). [2]

Struct field types misaligned with Gemini spec—willContinue and scheduling should be concrete types, not json.RawMessage

The JSON field names (willContinue, scheduling, parts, id) correctly match the Gemini API. However, the types are problematic:

  • willContinue should be bool, not json.RawMessage (spec: boolean)
  • scheduling should be string, not json.RawMessage (spec: enum)
  • parts should be []FunctionResponsePart or similar, not json.RawMessage (spec: array of objects)
  • id can remain json.RawMessage if intentionally opaque, but spec shows it as simple string

Using json.RawMessage for well-defined fields defers type checking to runtime and requires manual unmarshalling downstream, increasing error risk.

🤖 Prompt for AI Agents 
In dto/gemini.go around lines 184-191, several fields use json.RawMessage but
should use concrete types per the Gemini spec: change WillContinue from
json.RawMessage to bool, Scheduling from json.RawMessage to string (or a defined
enum type), Parts from json.RawMessage to a slice of the appropriate struct
(e.g., []FunctionResponsePart) so it is unmarshalled automatically, and change
ID to string (or keep as json.RawMessage only if intentionally opaque); update
any imports/usages and adjust downstream code that relied on manual
json.RawMessage unmarshalling accordingly.


type GeminiPartExecutableCode struct {
Expand All @@ -202,11 +208,15 @@ type GeminiFileData struct {
}

type GeminiPart struct {
Text string `json:"text,omitempty"`
Thought bool `json:"thought,omitempty"`
InlineData *GeminiInlineData `json:"inlineData,omitempty"`
FunctionCall *FunctionCall `json:"functionCall,omitempty"`
FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
Text string `json:"text,omitempty"`
Thought bool `json:"thought,omitempty"`
InlineData *GeminiInlineData `json:"inlineData,omitempty"`
FunctionCall *FunctionCall `json:"functionCall,omitempty"`
ThoughtSignature json.RawMessage `json:"thoughtSignature,omitempty"`
FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
// Optional. Media resolution for the input media.
MediaResolution json.RawMessage `json:"mediaResolution,omitempty"`
VideoMetadata json.RawMessage `json:"videoMetadata,omitempty"`
Comment on lines +215 to +219

@coderabbitai coderabbitai Bot Nov 19, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify field types against Gemini API spec—MediaResolution type is inconsistent.

Three new fields use json.RawMessage, following the same pattern flagged in GeminiFunctionResponse. More critically, MediaResolution at line 218 conflicts with the existing MediaResolution type definition used in GeminiChatGenerationConfig at line 284 (defined as MediaResolution string at line 292).

Using json.RawMessage for potentially well-defined fields reduces type safety and requires manual unmarshalling.

Please verify the correct types for these fields in the Gemini 3 API specification:

If MediaResolution should use the existing MediaResolution type, apply this change:

-	MediaResolution     json.RawMessage                `json:"mediaResolution,omitempty"`
+	MediaResolution     MediaResolution                `json:"mediaResolution,omitempty"`

🌐 Web query:

What are the exact field types for thoughtSignature, mediaResolution, and videoMetadata in Gemini 3 API GeminiPart response structure?

💡 Result:

  • thoughtSignature — binary (byte array); in JSON responses it's base64-encoded (Go: []byte). [1][2]
  • mediaResolution — enum / structured enum field (MediaResolution / PartMediaResolution with levels like MEDIA_RESOLUTION_LOW / MEDIUM / HIGH). In JSON it's the enum name (string). [3][1]
  • videoMetadata — object (VideoMetadata structure) containing video-specific fields (timestamps, fps, mime, duration, etc.). [2][4]

Sources: Go schema showing Part and thoughtSignature/videoMetadata. [1][2]; Gemini API docs for media_resolution enum/details. [3]; video understanding docs for VideoMetadata details. [4]

Replace json.RawMessage with concrete types per Gemini 3 API spec.

The three new fields are defined with incorrect types. According to the Gemini 3 API specification:

  • thoughtSignature should be []byte (binary, base64-encoded in JSON)
  • mediaResolution should be the enum type MediaResolution (as already defined at line 292)
  • videoMetadata should be the VideoMetadata struct

Using json.RawMessage sacrifices type safety and requires manual unmarshalling. Replace all three:

-	ThoughtSignature    json.RawMessage                `json:"thoughtSignature,omitempty"`
+	ThoughtSignature    []byte                         `json:"thoughtSignature,omitempty"`
	FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
	// Optional. Media resolution for the input media.
-	MediaResolution     json.RawMessage                `json:"mediaResolution,omitempty"`
-	VideoMetadata       json.RawMessage                `json:"videoMetadata,omitempty"`
+	MediaResolution     MediaResolution                `json:"mediaResolution,omitempty"`
+	VideoMetadata       *VideoMetadata                 `json:"videoMetadata,omitempty"`
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ThoughtSignature json.RawMessage `json:"thoughtSignature,omitempty"`
FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
// Optional. Media resolution for the input media.
MediaResolution json.RawMessage `json:"mediaResolution,omitempty"`
VideoMetadata json.RawMessage `json:"videoMetadata,omitempty"`
ThoughtSignature []byte `json:"thoughtSignature,omitempty"`
FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
// Optional. Media resolution for the input media.
MediaResolution MediaResolution `json:"mediaResolution,omitempty"`
VideoMetadata *VideoMetadata `json:"videoMetadata,omitempty"`
🤖 Prompt for AI Agents 
In dto/gemini.go around lines 215 to 219, the fields thoughtSignature,
mediaResolution and videoMetadata are typed as json.RawMessage but per Gemini 3
API they must be concrete types: change thoughtSignature to []byte (JSON
base64-encoded), mediaResolution to the existing MediaResolution enum type, and
videoMetadata to the VideoMetadata struct; update struct tags as needed to keep
",omitempty", remove manual unmarshalling for these fields and adjust any
callers/tests that rely on RawMessage to use the concrete types instead.

FileData *GeminiFileData `json:"fileData,omitempty"`
ExecutableCode *GeminiPartExecutableCode `json:"executableCode,omitempty"`
CodeExecutionResult *GeminiPartCodeExecutionResult `json:"codeExecutionResult,omitempty"`
Expand Down
1 change: 1 addition & 0 deletions relay/channel/gemini/constant.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ var ModelList = []string{
"gemini-1.5-pro-latest", "gemini-1.5-flash-latest",
// preview version
"gemini-2.0-flash-lite-preview",
"gemini-3-pro-preview",
// gemini exp
"gemini-exp-1206",
// flash exp
Expand Down