fix(google_genai): preserve raw bytes in Gemini streamGenerateContent SSE framing

[mateo-berri]

Relevant issues

Fixes #30471 (and the related earlier report #28777)

Linear ticket

LIT-3775

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Repro is the Google GenAI SDK pointed at the proxy's streamGenerateContent pass-through, which is what the reporter used. Run a proxy against a real Vertex/Gemini model with thinking enabled so the response carries thought/text content (the trigger is any Unicode line separator such as U+2028, U+2029 or U+0085 emitted raw in the data: JSON; large thinking responses make it likely).

1. Start the proxy with the config from the issue (a vertex_ai/gemini-3.1-pro-preview model on global)

python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log

2. Hit the endpoint exactly as the SDK does and confirm every chunk is valid JSON

curl -sN -X POST 'http://localhost:4000/v1beta/models/gemini-3.1-pro-preview:streamGenerateContent?alt=sse' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -d '{"contents":[{"role":"user","parts":[{"text":"Write a fully documented and optimized function that prints the first n primes in python"}]}],"generationConfig":{"thinkingConfig":{"thinkingLevel":"high"}}}' \
  | grep '^data: ' | sed 's/^data: //' | grep -v '^\[DONE\]' \
  | while IFS= read -r line; do echo "$line" | python -c 'import sys,json; json.loads(sys.stdin.read()); print("ok")'; done

Before the fix some events fail to parse (Failed to parse response as JSON); after the fix every line prints ok

Type

🐛 Bug Fix

Changes

The Gemini generate-content streaming iterators in litellm/google_genai/streaming_iterator.py framed SSE events using httpx aiter_lines/iter_lines. Those methods split on str.splitlines boundaries, which go beyond \n/\r to include U+2028, U+2029, U+0085, form feed and friends. Gemini emits those characters raw inside the data: JSON for thinking and text content, so a single SSE event was being sliced mid-payload and then rejoined with a \n, yielding invalid JSON and google.genai.errors.UnknownApiResponseError. It looked intermittent because it only fired when the response happened to contain one of those separators, which correlates with larger thinking responses.

The fix buffers the raw byte stream and splits only on real SSE frame delimiters (\r\n\r\n, \n\n, \r\r). Payload bytes are now preserved regardless of content, and large inlineData blobs that span multiple HTTP chunks are still reassembled into a single event, which is the case the earlier aiter_lines change (#30270) was meant to address; this keeps that behavior while removing the corruption.

Tests in tests/test_litellm/google_genai/test_google_genai_streaming_iterator.py now mock byte iteration and add a regression that feeds an event whose JSON carries raw U+2028/U+2029/U+0085 split across small byte chunks, asserting one complete event that parses back to the original text. There is also coverage for reassembling chunk-split inlineData and for splitting multiple events out of one buffer. The two streaming mocks in tests/unified_google_tests/test_google_ai_studio.py were updated from aiter_lines to aiter_bytes to match the new path.

---

Generated by Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!