🐛 fix: Gemini FunctionResponse parts inlineData parsing

[t0ng7u]

Gemini native FunctionResponse may include media in functionResponse.parts. Previously, the Parts field was defined as json.RawMessage, preventing GeminiPart custom unmarshal logic from normalizing snake_case keys (inline_data/mime_type) to the camelCase format (inlineData/mimeType) required by Gemini REST.

This change updates GeminiFunctionResponse.Parts to []GeminiPart so nested media parts are correctly parsed and forwarded, enabling the model to read inline data.

Summary by CodeRabbit

• Refactor
  • Improved the structure of AI response handling to enable more robust validation and processing of incoming data.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

The GeminiFunctionResponse struct in dto/gemini.go is modified to change the Parts field from a json.RawMessage to a []GeminiPart slice, replacing opaque JSON handling with strongly-typed array parsing.

Changes

Cohort / File(s)	Summary
Struct field type conversion dto/gemini.go	Changed GeminiFunctionResponse.Parts field type from json.RawMessage to []GeminiPart for improved type safety and structured unmarshalling

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Verify that dependent code accessing Parts handles the new slice type correctly
• Ensure JSON unmarshalling behavior aligns with the new []GeminiPart structure
• Check for any direct json.RawMessage operations that now need to adapt to the slice type

Possibly related PRs

• feat: gemini-3-pro #2243: Previously introduced the Parts field as json.RawMessage in the same GeminiFunctionResponse struct; this PR converts it to the typed []GeminiPart slice.

Poem

🐰 A field once raw, now typed with care,
From opaque bytes to structs so fair,
The Parts array hops along the way,
Strong-typed paths brighten parsing's day!
✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title references a specific fix for Gemini FunctionResponse parts inlineData parsing, which matches the core change of updating Parts from json.RawMessage to []GeminiPart to enable proper parsing of inline data.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/gemini-functionresponse-inline-data-parts

---

📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cc3ba39 and 1d26f63.

📒 Files selected for processing (1)

• dto/gemini.go (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: RedwindA
Repo: QuantumNous/new-api PR: 1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.157Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.

📚 Learning: 2025-08-08T17:12:43.157Z

Learnt from: RedwindA
Repo: QuantumNous/new-api PR: 1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.157Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.

Applied to files:

• dto/gemini.go

🔇 Additional comments (1)

dto/gemini.go (1)

222-222: This type change is safe and requires no further action.

The change from json.RawMessage to []GeminiPart correctly enables custom unmarshaling logic to normalize snake_case keys (inline_data, mime_type) to camelCase (inlineData, mimeType) as required by Gemini REST API. This aligns with the pattern used in GeminiChatContent.Parts (line 285).

The field is optional (omitempty tag), and there is no existing code that reads or writes GeminiFunctionResponse.Parts, so the type change introduces no breaking changes. When the Gemini API sends media in functionResponse.parts, this change will ensure proper unmarshaling through GeminiPart's custom logic.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}