Skip to content

fix(gemini): prevent parallel tool call slot collisions in native streaming - #99273

Open
cycorld wants to merge 1 commit into
NousResearch:mainfrom
cycorld:fix/gemini-streaming-parallel-tool-calls
Open

cycorld wants to merge 1 commit into
NousResearch:mainfrom
cycorld:fix/gemini-streaming-parallel-tool-calls

Conversation

@cycorld

@cycorld cycorld commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes an issue where Gemini 3.x / 2.5 streaming responses containing multiple parallel tool calls with the same tool name (e.g. 3 parallel search_files calls) collided into slot index 0.

Problem & Root Cause

When Gemini emits parallel tool calls across separate SSE events in streamGenerateContent?alt=sse, each SSE event chunk delivers a single candidate with parts[0] (part_index == 0).

In translate_stream_event(), call_key previously hashed only:

call_key = json.dumps(
    {
        "part_index": part_index,
        "name": name,
        "thought_signature": thought_signature,
    },
    sort_keys=True,
)

When Gemini streamed consecutive parallel tool calls of the same name, call_key was identical across chunks, mapping every tool call to the same slot (index: 0). The streaming accumulator (chat_completion_helpers.py:tool_calls_acc) appended arguments into one string:
{"pattern": "*a*"}{"pattern": "*b*"}{"pattern": "*c*"}

Because the resulting string was invalid concatenated JSON, argument parsing failed and the loop exhausted retries, ultimately misdiagnosing the turn as:
Response truncated due to output length limit

Solution

  1. Use functionCall.id in call_key (f"id:{fc_id}") when present (standard in Gemini 3+).
  2. For legacy models without IDs (Gemini < 3), include args in the fallback call_key so different calls at part_index == 0 receive distinct slot indices.
  3. Added unit test coverage for parallel tool calls across SSE chunks for both Gemini 3+ and legacy models.

Verification

Ran full test suite:

  • pytest tests/agent/test_gemini_native_adapter.py (31/31 passed)
  • pytest tests/agent/test_gemini_*.py tests/hermes_cli/test_gemini_*.py (78/78 passed)

…eaming

When Gemini emits multiple parallel functionCall parts across separate
SSE chunks, each chunk contains a single candidate with parts[0].
Previously, call_key only hashed (part_index, name, thought_signature),
causing separate parallel tool calls with the same name (e.g. search_files)
to map to the same slot index 0. The streaming accumulator concatenated
their argument strings into invalid JSON, which failed parsing and
misclassified the turn as 'Response truncated due to output length limit'.

Use functionCall.id in call_key when present (Gemini 3+), and include
args in fallback call_key for legacy models without IDs, ensuring each
parallel tool call gets its own distinct slot index.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) area/streaming Streaming responses: gateway delivery, provider wire P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Aug 31, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Duplicate of #75528, which already fixes provider-ID slot identity and the id-less fallback while preserving incremental argument continuations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants