Skip to content

fix(agent): prefix stripping and suffix finalization for Gemini native streaming - #932

Open
hashbender wants to merge 1 commit into
mainfrom
mirror/pr-57165
Open

fix(agent): prefix stripping and suffix finalization for Gemini native streaming#932
hashbender wants to merge 1 commit into
mainfrom
mirror/pr-57165

Conversation

@hashbender

Copy link
Copy Markdown
Owner

Summary of Changes

This PR fixes a critical bug in the native Gemini adapter (agent/gemini_native_adapter.py) that causes duplicated and concatenated tool call arguments (resulting in invalid stacked JSON like {"pattern": "*.py"}{"pattern": "*.py", "target": "files"}) during streaming tool calls.

Root Cause

Unlike OpenAI or Anthropic, Google's Gemini native SSE stream returns fully accumulated tool argument dictionaries in each chunk instead of incremental string deltas.

In the original translation logic:

  1. The adapter did string-based prefix subtraction using args_str.startswith(last_arguments).
  2. However, non-final chunks contain trailing closures (such as ", }, ]) representing the closed state of the JSON up to that point.
  3. When the subsequent event arrives with more parameters, the trailing closure characters in last_arguments mismatch with the new, expanding stream content at that position (e.g., , vs }).
  4. This causes startswith to return False, forcing the adapter to emit the entire updated argument string again. The final aggregated stream is corrupted with repeated and stacked JSON objects, crashing any stream-based JSON parser.

Solution

We implemented a robust prefix stripping and suffix finalization sliding window:

  1. Dynamic Right-Stripping: For non-final stream chunks (is_final = False), we right-strip trailing JSON closures (rstrip(' \t\n\r"}]')) before storing them in last_stripped and calculating the emitted delta. This prevents trailing closures from breaking prefix matching in subsequent stream events.
  2. Final Suffix Restoration: On the final event (marked by finishReason or EOF), we do not strip the suffixes. The adapter calculates the remaining delta from the full string, safely emitting the final trailing closures (e.g., def"}) to cleanly terminate the JSON object.
  3. End-of-Stream Guard: If for any reason the stream ends abruptly without finishReason being set on a candidate, the generator loops through and finalizes any remaining unfinalized tool calls.

Testing

We added a dedicated regression test suite in tests/agent/test_gemini_native_adapter.py:

  • test_stream_event_translation_with_prefix_stripping_and_suffix_finalization
  • test_stream_event_translation_parallel_calls_with_disappearing_parts

These tests verify that arguments are correctly diffed, stripped, and finalized across simulated sequential events, ensuring 100% valid JSON generation.

All tests passed successfully on our local environment:

pytest tests/agent/test_gemini_native_adapter.py -k "prefix_stripping or parallel"

Mirror-of: NousResearch#57165
NousResearch#57165

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant