chore(release): backport #31519, #31733 to stable/1.90.x and cut 1.90.2

[yuneng-berri]

Relevant issues

Backports two already-merged realtime fixes from litellm_internal_staging onto stable/1.90.x. #31519 hardens the Gemini Live realtime path: it stops sending a second setup on a follow-up session.update (Gemini Live accepts setup as the first-and-only client message and closes the socket with 1007 on a second one, which showed up as silence after the first turn and intermittent 1011s), retries a hung backend open handshake instead of surfacing a single slow handshake to the caller as a fatal 1011, and folds the transcription-guardrail auto-response disable into the one-and-only setup so a realtime_input_transcription guardrail keeps gating the turn. #31733 routes realtime success logging through the bounded logging worker instead of a bare asyncio.create_task, so a slow logging callback on a long-lived realtime websocket can no longer leave one suspended task per turn pinning that turn's response in memory

Cuts 1.90.2. The line's tip was 1.90.1 and 1.90.1 has already shipped (the litellm/litellm:v1.90.1 image is published on DockerHub, pushed 2026-06-30), so these picks take the next patch, 1.90.2. The git tag and release notes for 1.90.1 are still in flight, so the release does not yet show in the repo tags; the published image is the authoritative signal

Linear ticket

N/A

Pre-Submission checklist

• I have added meaningful tests
• My PR's scope is as isolated as possible; it only solves 1 specific problem

Screenshots / Proof of Fix

Both picks' own regression tests pass on stable/1.90.x, judged as a delta against a clean baseline captured on the line's tip before picking (144 passed, 0 failures). After both picks the same targeted suite is 151 passed, 0 failures (baseline 144 plus 7 net-new tests: 6 from #31519, 1 from #31733), so every new test the picks add passes and nothing that passed before regressed

uv run --extra proxy pytest \
  tests/test_litellm/litellm_core_utils/test_realtime_streaming.py \
  tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py \
  tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py -q
# baseline (pre-pick): 144 passed
# after both picks:    151 passed

Live proxy sanity on the picked code (per-worktree proxy on the line, real OpenAI call), before and after the picks, to confirm the line still boots and serves with these changes wired in:

# before picks
curl -sf localhost:PORT/health/liveliness            -> "I'm alive!"
curl localhost:PORT/v1/chat/completions -d '{"model":"gpt-4o",...}'  -> "Backport baseline approved."

# after picks
curl -sf localhost:PORT/health/liveliness            -> "I'm alive!"
curl localhost:PORT/v1/chat/completions -d '{"model":"gpt-4o",...}'  -> "Backport post OK."

The original bug repros are Gemini Live websocket sessions, which the default proxy config does not wire up (no Gemini Live model, and it is a websocket rather than an HTTP call), so the behavioral proof here is the picks' own tests plus the proxy loading the picked realtime code with no import or wiring errors

Type

Bug Fix

Changes

What is included, in merge order:

• fix(realtime): stop second Gemini Live setup, retry hung handshake, close guardrail bypass #31519 fix(realtime): stop second Gemini Live setup, retry hung handshake, fold guardrail auto-response disable into setup
• fix(logging): route realtime success logging through the bounded worker #31733 fix(logging): route realtime success logging through the bounded worker
• bump: version 1.90.1 -> 1.90.2
• chore: refresh uv.lock for 1.90.2

Adaptation notes

Both picks are adaptations rather than verbatim cherry-picks; 1.90.x predates the realtime refactor (#30960, which introduced _content_sent_after_setup and reshaped _handle_session_update), a realtime fix (#30446), and the formatter migration (black to ruff, width 120) that these staging commits were built on. So the picks were hand-resolved onto 1.90.x's older realtime code and its black formatting; each pick's own added or changed lines are preserved, only surrounding context and formatting differ

• fix(realtime): stop second Gemini Live setup, retry hung handshake, close guardrail bypass #31519: the guardrail-inject call is placed directly before backend_ws.send(msg) in _send_to_backend (1.90.x has no _content_sent_after_setup guard there, which is a separate test: add realtime proxy e2e suite across providers #30960 concern the inject line does not depend on); the follow-up-setup merge block in _handle_session_update is replaced wholesale with the drop logic; _open_realtime_backend_ws and the reordered auto-send are applied as in the source. In the gemini test suite the 4 obsolete follow-up-merge tests are removed and the 3 drop tests added, matching the source PR's own test changes exactly (name-equality verified)
• fix(logging): route realtime success logging through the bounded worker #31733: the source change (import plus routing log_messages success logging through GLOBAL_LOGGING_WORKER) applied cleanly on top of fix(realtime): stop second Gemini Live setup, retry hung handshake, close guardrail bypass #31519; it is marked adapted only because the surrounding file is black-formatted on this line rather than the source's ruff-120

Known noise on this line

The baseline for the three targeted test files was clean (0 failures), so there is no pre-existing test noise to discount for this PR

[greptile-apps]

Greptile Summary

This PR backports two realtime fixes from staging onto stable/1.90.x: it stops Gemini Live from receiving a second setup message (which caused 1007 disconnects), adds a per-attempt timeout with retry for hung backend open handshakes, folds the transcription-guardrail auto-response disable into the one-and-only setup, and routes realtime success logging through GLOBAL_LOGGING_WORKER instead of a bare asyncio.create_task.

• Gemini setup hardening (transformation.py, realtime_streaming.py, llm_http_handler.py): subsequent session.updates are now dropped instead of forwarded as a second setup; the RealTimeStreaming object is constructed before the initial setup is sent so _maybe_inject_guardrail_auto_response_disable can fold automaticActivityDetection.disabled=true into the one-and-only setup message.
• Handshake retry (llm_http_handler.py): _open_realtime_backend_ws wraps each connect attempt with open_timeout and retries on TimeoutError/OSError/WebSocketException while immediately re-raising deterministic status rejections (InvalidStatus/InvalidStatusCode), covering both websockets <15 and ≥15.
• Logging worker routing (realtime_streaming.py): async success logging is enqueued on GLOBAL_LOGGING_WORKER, bounding concurrency and preventing per-turn suspended-task memory growth.

Confidence Score: 4/5

Safe to merge; the three independent fixes are well-scoped and the targeted test suite passes cleanly after both picks.

The core logic is sound across all three fixes. The only callouts are a redundant map_openai_params call in the drop path of _handle_session_update (wastes a little CPU per dropped update but has no functional impact) and an assert guarding the post-loop raise in _open_realtime_backend_ws that would disappear under -O, leaving a confusing TypeError if max_attempts were ever zero. Neither affects normal operation. The four removed Gemini tests correctly reflect the intentional behaviour reversal and are replaced by equivalent new tests.

No files require special attention; litellm/llms/gemini/realtime/transformation.py and litellm/llms/custom_httpx/llm_http_handler.py carry the minor items flagged above.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/realtime_streaming.py	Adds _maybe_inject_guardrail_auto_response_disable to fold transcription-guardrail's auto-response disable into the one-and-only Gemini setup, and routes async success logging through GLOBAL_LOGGING_WORKER instead of a bare asyncio.create_task.
litellm/llms/custom_httpx/llm_http_handler.py	Extracts _open_realtime_backend_ws with retry logic for hung handshakes; reorders realtime setup so the RealTimeStreaming object exists before the initial setup is sent (enabling guardrail inject into the one-and-only setup message).
litellm/llms/gemini/realtime/transformation.py	Replaces the follow-up setup merge logic in _handle_session_update with a simple drop; subsequent session.updates now return [] with an appropriate log (warning for the create_response=False guardrail signal, debug otherwise). Minor inefficiency: map_openai_params is still computed for the drop path.
tests/test_litellm/litellm_core_utils/test_realtime_streaming.py	Adds 4 new tests: guardrail inject into setup, no-op without guardrail, non-bidi message left untouched, and regression test verifying success logging routes through GLOBAL_LOGGING_WORKER. No existing tests modified.
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Adds 3 new backend-open retry tests via the _FakeWebsocketsModule fixture: retry-then-succeed, exhaust-max-attempts, and deterministic-rejection (both websockets<15 and >=15 variants). No existing tests modified.
tests/test_litellm/llms/gemini/realtime/test_gemini_realtime_transformation.py	Removes 4 tests that verified the old follow-up-merge behaviour (now intentionally replaced with drop logic) and adds 3 new tests verifying the new drop behaviour, including a warning check for the create_response=False case.

Comments Outside Diff (1)

1. litellm/llms/gemini/realtime/transformation.py, line 442-455 (link)

   When session_configuration_request is not None, the function always returns [] and never uses new_overrides, yet the code still runs the full _normalize_session_payload_for_mapping + map_openai_params pipeline. _extract_turn_detection only needs the normalized session_payload, so the map_openai_params call is dead work for every dropped follow-up update.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "fix(logging): route realtime success log..." | Re-trigger Greptile}

[greptile-apps]

assert can be silently dropped when the interpreter runs with -O/-OO. If max_attempts=0 is ever passed, the assertion is eliminated, last_exc is None, and raise last_exc raises TypeError: exceptions must derive from BaseException rather than a useful error. A real guard is safer: if last_exc is None: raise RuntimeError(...) before raise last_exc.

[greptile-apps]

Four existing tests are removed here per the custom rule on test modifications. The removed tests (test_gemini_subsequent_session_update_forwards_tools_merged_with_original_setup, test_gemini_subsequent_session_update_with_turn_detection_only_preserves_original_tools, test_gemini_follow_up_session_update_preserves_response_modalities_on_partial_generation_config, test_gemini_subsequent_session_update_preserves_automatic_activity_detection_subfields) all asserted the old follow-up-merge behaviour, which is now replaced by the intentional drop. The three new tests assert the opposite (drop) contract, so coverage is updated rather than weakened — confirming this is a legitimate behaviour-aligned removal rather than a mask for a regression.

Rule Used: What: Flag any modifications to existing tests and... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

❌ Patch coverage is 10.63830% with 42 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/custom_httpx/llm_http_handler.py	9.09%	20 Missing ⚠️
litellm/litellm_core_utils/realtime_streaming.py	15.00%	17 Missing ⚠️
litellm/llms/gemini/realtime/transformation.py	0.00%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!