-
Notifications
You must be signed in to change notification settings - Fork 2.4k
feat(providers): add streaming support for Google Gemini provider #6191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@DOsinga @alexhancock Hi, Apologies for the direct ping for review. I don't seem to have permissions to request a review. Probably need an invite to join the github organization. |
|
I've tested using |
Here is what I see with gemini-2.0 flash. Are you using cli, web or some third-party UI? |
|
Only tested the desktop electron app ( |
Right, I noticed that desktop UI aggregates streaming chunks into a single message bubble with metadata (timestamp, copy button, etc.) before rendering. It does this by matching message IDs which does not come in gemini response chunks (API does not povide it unlike other providers). I did not see the issue in cli as cli does not do any aggregation. I've fixed this by generating a consistent UUID at the start of each streaming session. Also, can you elaborate what section of code you were referring to when you said "it destroys basic formatting"? This is all |
- Add stream() and supports_streaming() to GoogleProvider - Use streamGenerateContent?alt=sse endpoint for SSE streaming - Implement response_to_streaming_message for SSE format parsing - Handle text chunks, function calls, thinking content, and usage - Add mid-stream error detection and propagation - Trust thoughtSignature for thinking content classification - Add tests for streaming text, function calls, and error handling The Google/Gemini provider does not add a message ID to streaming chunks, causing the desktop UI to display each chunk as a separate message with its own timestamp instead of aggregating them. The desktop UI aggregates streaming chunks by matching message IDs. Other providers (OpenAI, Anthropic) get IDs from their APIs, but Gemini's API doesn't provide one. We now generate a UUID at stream start and use it for all chunks in that session. Signed-off-by: rabi <[email protected]>
98cfb23 to
7b77ce4
Compare
|
Looks like it's indeed working now (using
I just quickly compared some indention in the diff and it looked off, I re-compared and it's looking better now |
OpenRouter with Gemini models now properly handles thoughtSignature: - Use Google format for creating requests to include thoughtSignature - Use Google format for parsing responses to extract thoughtSignature - This enables proper thinking content echoing in the conversation loop - Disable streaming for Google models until PR block#6191 is merged Signed-off-by: rabi <[email protected]>
|
Folks PTAL. This would help us implement streaming in other providers like Vertex AI and OpenRouter etc. |
jamadeo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, thanks for the contribution!
I also ran test_providers.sh for gemini, and it passed.
|
Nice |
* 'main' of github.com:block/goose: Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) fix: prevent infinite loop of tool-input notifications in MCP Apps (#6374)
* main: (31 commits) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) fix: prevent infinite loop of tool-input notifications in MCP Apps (#6374) fix: Show platform-specific keyboard shortcuts in UI (#6323) fix: we load extensions when agent starts so don't do it up front (#6350) docs: credit HumanLayer in RPI tutorial (#6365) Blog: Goose Lands MCP Apps (#6172) Claude 3.7 is out. we had some harcoded stuff (#6197) ...
* main: (89 commits) fix(google): treat signed text as regular content in streaming (#6400) Add frameDomains and baseUriDomains CSP support for MCP Apps (#6399) fix(ci): add missing dependencies to openapi-schema-check job (#6367) feat: http proxy support Add support for changing working dir and extensions in same window/session (#6057) Sort keys in canonical models (#6403) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) ...
* main: (89 commits) fix(google): treat signed text as regular content in streaming (#6400) Add frameDomains and baseUriDomains CSP support for MCP Apps (#6399) fix(ci): add missing dependencies to openapi-schema-check job (#6367) feat: http proxy support Add support for changing working dir and extensions in same window/session (#6057) Sort keys in canonical models (#6403) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) ...
…ock#6191) Signed-off-by: rabi <[email protected]>


Summary
Type of Change
AI Assistance
Testing
Unit and manual testing with provider