chore(release): backport #31733 to stable/1.89.x

[yuneng-berri]

Relevant issues

Backports #31733 onto stable/1.89.x. RealTimeStreaming.log_messages dispatched the realtime success handler with a bare asyncio.create_task, bypassing GLOBAL_LOGGING_WORKER (the bounded logging worker that provides a per-coroutine timeout and a concurrency cap). On a long-lived realtime websocket a slow logging callback left one suspended task per logged turn, each pinning that turn's assembled response, accumulating without bound (~12-15k in-flight under load in a repro) until the proxy ran out of memory. Routing realtime success logging through the bounded worker caps in-flight logging and cancels a hung callback at the worker timeout

No version bump. The line tip is already at 1.89.5 (bumped by the prior backport and not yet released), so this pick rides the pending 1.89.5

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem

What is included

• fix(logging): route realtime success logging through the bounded worker #31733 fix(logging): route realtime success logging through the bounded worker (cherry-picked from d4c33b2)

Originally requested alongside #31519, but #31519 was dropped from this line. #31519 is built on top of #30960, a large realtime restructure that is not on stable/1.89.x, so #31519's changes reference machinery (the setup-vs-content send loop and the renamed guardrail helper) that this line does not have. Backporting #31519 here would mean also backporting that restructure, which is out of scope for this patch; #31733 is independent of it and stands alone

Adaptation notes

#31733 is ADAPTED (context-only, no logic change). The fix's own lines are byte-identical to the source commit; two divergences come from this line predating the realtime restructure:

1. In litellm/litellm_core_utils/realtime_streaming.py the added import and the one-line change in log_messages apply verbatim; only the surrounding context differs (the line's typing import does not include Protocol, and model_call_details is formatted at the older width)
2. In tests/test_litellm/litellm_core_utils/test_realtime_streaming.py the new regression test test_log_messages_routes_async_logging_through_bounded_worker is appended after this line's last test rather than after the source's anchor test (which is not on this line). The added test body is byte-identical, and a name-equality check confirms the resolved file adds exactly the source's added test and nothing else

Known noise on this line

None. The targeted test set was green at baseline (112 passed, 0 failed) before any pick

Screenshots / Proof of Fix

Live proxy on localhost, hitting the real OpenAI API (sanity floor, judged as a delta against the pre-pick baseline):

# baseline (before pick)
$ curl -sf "localhost:4001/health/liveliness?_bp=bp-realtime-0630-base"
"I'm alive!"
$ curl -s localhost:4001/v1/chat/completions -d '{"model":"gpt-4o",...}'
CONTENT: Baseline backport approved.   USAGE: total_tokens=22

# after pick
$ curl -sf "localhost:4001/health/liveliness?_bp=bp-realtime-0630-post"
"I'm alive!"
$ curl -s localhost:4001/v1/chat/completions -d '{"model":"gpt-4o",...}'
CONTENT: Backport successfully completed.   USAGE: total_tokens=21

The bug itself is a memory leak on long-lived realtime (Gemini Live) websockets under load, which is not reproducible with a single curl. The behavioral proof is the regression test, which is mutation-checked: with the fix in place test_log_messages_routes_async_logging_through_bounded_worker passes; reverting the one-line change back to the bare asyncio.create_task makes it fail. Targeted test delta: 112 passed at baseline, 113 passed after the pick (the +1 is the new regression test), 0 new failures

Gauntlet (behavioral, universal): SURVIVED. All three sub-claims held; every identifier the pick references resolves on this line, the new regression test passes as a clean +1 delta, and the sole production caller of log_messages is structurally unaffected. Five investigator lenses (including two tasked as refuters) converged with no verified refutation; the only artifact surfaced was a benign pre-existing RuntimeWarning under certain test orderings that fails no test and is not introduced by this pick

Type

🐛 Bug Fix

Changes

Routes realtime success logging through GLOBAL_LOGGING_WORKER so in-flight logging on realtime websockets is bounded

[greptile-apps]

Greptile Summary

This backport routes realtime success logging through GLOBAL_LOGGING_WORKER instead of bare asyncio.create_task, capping the number of in-flight logging coroutines and applying a per-coroutine timeout to prevent memory accumulation on long-lived realtime websockets. A regression test is added to verify the routing change holds.

• realtime_streaming.py: single-line change in log_messages — asyncio.create_task(...) is replaced by GLOBAL_LOGGING_WORKER.ensure_initialized_and_enqueue(...), with the corresponding import added at the top of the file.
• test_realtime_streaming.py: appends test_log_messages_routes_async_logging_through_bounded_worker, which patches both GLOBAL_LOGGING_WORKER and asyncio.create_task to assert the worker path is taken and the bare-task path is not.

Confidence Score: 5/5

The change is a minimal, targeted fix — one import and one line replaced in the production file, plus one new regression test. The bounded worker path is already exercised elsewhere in the codebase; this just connects realtime logging to it.

Both changed files are focused and correct. The production change swaps an unbounded create_task for the existing bounded-worker enqueue; the test straightforwardly patches both paths and asserts the right one is taken. No pre-existing tests were modified, and no custom rules are violated.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/realtime_streaming.py	One-line fix replacing bare asyncio.create_task with GLOBAL_LOGGING_WORKER.ensure_initialized_and_enqueue for realtime success logging, preventing unbounded task accumulation on long-lived websockets.
tests/test_litellm/litellm_core_utils/test_realtime_streaming.py	Adds test_log_messages_routes_async_logging_through_bounded_worker regression test that verifies GLOBAL_LOGGING_WORKER.ensure_initialized_and_enqueue is called and bare asyncio.create_task is not used for success logging.

_{Reviews (1): Last reviewed commit: "fix(logging): route realtime success log..." | Re-trigger Greptile}