fix(agent): stop concatenated tool-call args in Gemini native streaming - #59871
fix(agent): stop concatenated tool-call args in Gemini native streaming#59871tsums wants to merge 1 commit into
Conversation
translate_stream_event() keyed each in-flight function-call slot by (part_index, name, thought_signature), but part_index is scoped to a single SSE event (each event typically carries one part) so it resets to 0 every time. When the model issued two calls to the same tool in one turn, both landed on the same key: the second call's full arguments were appended to the first call's buffer as if it were a continuation, producing concatenated/invalid JSON args downstream (e.g. two discord_read_messages calls for different channel IDs in one turn). The malformed args then fail sanitization, the tool call is dropped, and the agent retries in a loop. Track a chain of slots per call_key and only treat new arguments as a continuation when they extend the previous slot's arguments; otherwise allocate a fresh slot.
Duplicate of #24676 — same function ( |
|
Thanks for the focused Gemini streaming regression fix. Current The patch makes the slot chain per key explicit and allocates a fresh index/id only when the incoming serialized arguments are neither an equal re-send nor a prefix continuation. The added regression in Member triage identifies #24676 as the earlier canonical duplicate; that affects maintainer selection, but current Automated hermes-sweeper review. |
What changed and why
translate_stream_event()inagent/gemini_native_adapter.pykeys eachin-flight function-call slot by
(part_index, name, thought_signature).part_indexis only unique within a single SSE event, and each eventtypically carries a single part, so it resets to
0on every event.When the model calls the same tool twice in one turn (e.g. two
discord_read_messagescalls for two different channel IDs), both callsland on the same
call_key. The second call's full arguments get treatedas a continuation of the first and are appended to its buffer instead of
starting a new slot — producing concatenated, invalid JSON such as:
Downstream, this fails JSON sanitization/repair and gets replaced with an
empty tool-call object, so the call fails (e.g. missing required
channelId), the model retries, and the loop repeats — in the case thatsurfaced this, it ran long enough to exhaust the model's output budget and
finish with
finish_reason='length'.Fix
Track a chain of slots per
call_key(most recent last) instead of asingle slot. New arguments are only treated as a continuation of the
previous call when they extend its last-seen arguments (equal or a
prefix-extension); otherwise a fresh slot is allocated. This mirrors the
existing "Ollama reuses index 0" workaround in
chat_completion_helpers.py, applied to Gemini's content-keyed slotlookup instead of an id-keyed one.
How to test
Added
test_stream_event_translation_separates_distinct_calls_across_eventsin
tests/agent/test_gemini_native_adapter.py, which reproduces twodifferent calls to the same tool name arriving in separate SSE events
(each at
part_index=0) and asserts they land in distinct slots withindependently valid JSON arguments. Confirmed this test fails on the
pre-fix code (
assert 0 != 0, both calls collapse to the same id) andpasses with the fix.
Ran:
61 passed, 0 failed.
Platforms tested
Linux (Ubuntu). No platform-specific code touched (pure Python
streaming-response translation logic).