fix(agent): prefix stripping and suffix finalization for Gemini native streaming - #932
Open
hashbender wants to merge 1 commit into
Open
fix(agent): prefix stripping and suffix finalization for Gemini native streaming#932hashbender wants to merge 1 commit into
hashbender wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
This PR fixes a critical bug in the native Gemini adapter (
agent/gemini_native_adapter.py) that causes duplicated and concatenated tool call arguments (resulting in invalid stacked JSON like{"pattern": "*.py"}{"pattern": "*.py", "target": "files"}) during streaming tool calls.Root Cause
Unlike OpenAI or Anthropic, Google's Gemini native SSE stream returns fully accumulated tool argument dictionaries in each chunk instead of incremental string deltas.
In the original translation logic:
args_str.startswith(last_arguments).",},]) representing the closed state of the JSON up to that point.last_argumentsmismatch with the new, expanding stream content at that position (e.g.,,vs}).startswithto returnFalse, forcing the adapter to emit the entire updated argument string again. The final aggregated stream is corrupted with repeated and stacked JSON objects, crashing any stream-based JSON parser.Solution
We implemented a robust prefix stripping and suffix finalization sliding window:
is_final = False), we right-strip trailing JSON closures (rstrip(' \t\n\r"}]')) before storing them inlast_strippedand calculating the emitted delta. This prevents trailing closures from breaking prefix matching in subsequent stream events.finishReasonor EOF), we do not strip the suffixes. The adapter calculates the remaining delta from the full string, safely emitting the final trailing closures (e.g.,def"}) to cleanly terminate the JSON object.finishReasonbeing set on a candidate, the generator loops through and finalizes any remaining unfinalized tool calls.Testing
We added a dedicated regression test suite in
tests/agent/test_gemini_native_adapter.py:test_stream_event_translation_with_prefix_stripping_and_suffix_finalizationtest_stream_event_translation_parallel_calls_with_disappearing_partsThese tests verify that arguments are correctly diffed, stripped, and finalized across simulated sequential events, ensuring 100% valid JSON generation.
All tests passed successfully on our local environment:
pytest tests/agent/test_gemini_native_adapter.py -k "prefix_stripping or parallel"Mirror-of: NousResearch#57165
NousResearch#57165