Skip to content

fix(agent): stop concatenated tool-call args in Gemini native streaming - #59871

Open
tsums wants to merge 1 commit into
NousResearch:mainfrom
tsums:fix/gemini-stream-tool-call-collision
Open

fix(agent): stop concatenated tool-call args in Gemini native streaming#59871
tsums wants to merge 1 commit into
NousResearch:mainfrom
tsums:fix/gemini-stream-tool-call-collision

Conversation

@tsums

@tsums tsums commented Jul 6, 2026

Copy link
Copy Markdown

What changed and why

translate_stream_event() in agent/gemini_native_adapter.py keys each
in-flight function-call slot by (part_index, name, thought_signature).
part_index is only unique within a single SSE event, and each event
typically carries a single part, so it resets to 0 on every event.

When the model calls the same tool twice in one turn (e.g. two
discord_read_messages calls for two different channel IDs), both calls
land on the same call_key. The second call's full arguments get treated
as a continuation of the first and are appended to its buffer instead of
starting a new slot — producing concatenated, invalid JSON such as:

{"channelId": "1234567890", "limit": 15}{"channelId": "0987654321...

Downstream, this fails JSON sanitization/repair and gets replaced with an
empty tool-call object, so the call fails (e.g. missing required
channelId), the model retries, and the loop repeats — in the case that
surfaced this, it ran long enough to exhaust the model's output budget and
finish with finish_reason='length'.

Fix

Track a chain of slots per call_key (most recent last) instead of a
single slot. New arguments are only treated as a continuation of the
previous call when they extend its last-seen arguments (equal or a
prefix-extension); otherwise a fresh slot is allocated. This mirrors the
existing "Ollama reuses index 0" workaround in
chat_completion_helpers.py, applied to Gemini's content-keyed slot
lookup instead of an id-keyed one.

How to test

Added test_stream_event_translation_separates_distinct_calls_across_events
in tests/agent/test_gemini_native_adapter.py, which reproduces two
different calls to the same tool name arriving in separate SSE events
(each at part_index=0) and asserts they land in distinct slots with
independently valid JSON arguments. Confirmed this test fails on the
pre-fix code (assert 0 != 0, both calls collapse to the same id) and
passes with the fix.

Ran:

scripts/run_tests.sh tests/agent/test_gemini_native_adapter.py \
  tests/agent/test_gemini_fast_fallback.py tests/agent/test_gemini_schema.py \
  tests/agent/test_gemini_free_tier_gate.py

61 passed, 0 failed.

Platforms tested

Linux (Ubuntu). No platform-specific code touched (pure Python
streaming-response translation logic).

translate_stream_event() keyed each in-flight function-call slot by
(part_index, name, thought_signature), but part_index is scoped to a
single SSE event (each event typically carries one part) so it resets
to 0 every time. When the model issued two calls to the same tool in
one turn, both landed on the same key: the second call's full
arguments were appended to the first call's buffer as if it were a
continuation, producing concatenated/invalid JSON args downstream
(e.g. two discord_read_messages calls for different channel IDs in
one turn). The malformed args then fail sanitization, the tool call
is dropped, and the agent retries in a loop.

Track a chain of slots per call_key and only treat new arguments as a
continuation when they extend the previous slot's arguments;
otherwise allocate a fresh slot.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) duplicate This issue or pull request already exists labels Jul 6, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #24676 — same function (translate_stream_event in agent/gemini_native_adapter.py), same root mechanism (part_index resets to 0 per SSE event so two calls to the same tool in one turn collide on one call_key and concatenate into invalid stacked JSON), and same fix approach (chain of slots per call_key with a value-based prefix-continuation guard). #24676 is the earliest open canonical fix. Cluster also includes #54355 (simpler key-extension, dup of #24676), #57165 (different prefix-strip mechanism), and issue #57939 with its fix #57941. Maintainer picks the canonical implementation.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused Gemini streaming regression fix. Current main still derives one slot key from event-local part_index, tool name, and thought signature in agent/gemini_native_adapter.py:698-706, while the response-wide dictionary is reused across every SSE event at agent/gemini_native_adapter.py:991-993. Distinct same-name calls arriving as separate one-part events therefore collide exactly as described.

The patch makes the slot chain per key explicit and allocates a fresh index/id only when the incoming serialized arguments are neither an equal re-send nor a prefix continuation. The added regression in tests/agent/test_gemini_native_adapter.py exercises the reported two-event collision and verifies independently valid arguments.

Member triage identifies #24676 as the earlier canonical duplicate; that affects maintainer selection, but current main does not yet provide this guarantee. The hunk context remains aligned with current main, so salvage should be mechanical.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/streaming Streaming responses: gateway delivery, provider wire labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants