Skip to content

Conversation

@angelahning
Copy link
Collaborator

@angelahning angelahning commented Aug 6, 2025

Issue
The response_to_streaming_message function in the OpenAI format provider was not returning tool call messages to the stream when AI providers send complete tool calls in a single chunk with finish_reason: "tool_calls".

Root Cause: The original logic assumed all tool calls would be streamed across multiple chunks (like OpenAI's behavior) and always entered a loop to wait for additional tool call data. When providers like Gemini send complete tool calls in one chunk, the function would:

  1. Process the complete tool call data
  2. Enter the waiting loop for more chunks
  3. Receive only empty chunks and the [DONE] signal
    4.Break out of the entire function without yielding the tool call message
    This resulted in tool calls being processed but never returned to the caller, causing streaming responses to appear empty.

Fix
Added a check to detect when tool calls are already complete in the initial chunk

Copy link
Collaborator

@jamadeo jamadeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find! This is a pretty nasty loop and could use some refactoring probably. But before taking that on, we should get some streaming-compatible scenario tests -- then we could add this as a case to test for

@alexhancock alexhancock self-requested a review August 7, 2025 15:39
@angelahning angelahning marked this pull request as ready for review August 7, 2025 15:58
@angelahning angelahning merged commit 6e6e083 into main Aug 7, 2025
11 checks passed
@angelahning angelahning deleted the aning/rr branch August 7, 2025 15:58
michaelneale added a commit that referenced this pull request Aug 8, 2025
* main:
  chore: rmcp 0.5.0 (#3935)
  add gpt-5 to openai provider format (#3924)
  added gpt5 context limit (#3927)
  show status of osx codesigning and increase timeout (#3926)
  Bump auto-compact threshold to 80% (#3925)
  FIX: gemini tool call hanging (#3898)
  feat(deps): upgrade rmcp to 0.4.1 (#3918)
  Fix dark mode rendering of config form and centered providers grid for wider screens. (#3837)
  fix: extension list not refreshing after installing from deeplink (#3878)
katzdave added a commit that referenced this pull request Aug 8, 2025
* 'main' of github.com:block/goose:
  remove fallback routing to hub/home for unknown routes (#3954)
  Use cross in linux bundle workflow (#3950)
  fix: disable signing for release branches until we figure out keys for this flow (#3951)
  Sanitize Tags Unicode Block (#3920)
  Add a message about DCO to CONTRIBUTING.md (#3741)
  Move hardcoded LLM prompts to template files (#3934)
  docs: migrate streamable config to consolidated component (#3936)
  feat: streamline list args on cli (#3937)
  mcp/developer: Refactor to use tokio SplitStream (#3894)
  feat: first time automated ollama install experience and openrouter (#3881)
  chore: rmcp 0.5.0 (#3935)
  add gpt-5 to openai provider format (#3924)
  added gpt5 context limit (#3927)
  show status of osx codesigning and increase timeout (#3926)
  Bump auto-compact threshold to 80% (#3925)
  FIX: gemini tool call hanging (#3898)
  feat(deps): upgrade rmcp to 0.4.1 (#3918)
  Fix dark mode rendering of config form and centered providers grid for wider screens. (#3837)
  fix: extension list not refreshing after installing from deeplink (#3878)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants