Fix: User specified async client ignored with Gemini streaming+async

[Chesars]

Relevant issues

Fixes #17148

Pre-Submission checklist

• I have Added testing in the tests/litellm/ directory
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Summary

When using litellm.acompletion() with stream=True and a custom HTTP client, the client is not beign passed:

# This did NOT work - client was ignored
response = await litellm.acompletion(
    model="gemini/gemini-2.5-pro",
    messages=[...],
    client=custom_client,  # ❌ Ignored
    stream=True,
)

The sync version worked correctly:

# This worked fine
response = litellm.completion(
    model="gemini/gemini-2.5-pro",
    messages=[...],
    client=custom_client,  # ✅ Works
    stream=True,
)

Solution

Added gemini_client parameter to make_call() (async) to preserve the user's custom client, matching the existing pattern in make_sync_call() (sync).

Tests added

• test_async_streaming_uses_custom_client
• test_sync_streaming_uses_custom_client

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 4, 2026 10:39pm

[greptile-apps]

Greptile Summary

Fixes a bug where user-specified async HTTP clients were silently ignored during Gemini streaming + async calls (litellm.acompletion() with stream=True). The fix adds a gemini_client parameter to the make_call() async function, matching the existing pattern already used in make_sync_call(). When CustomStreamWrapper.fetch_stream() invokes the partial, the user's client now correctly takes priority over the module-level fallback.

• Added gemini_client parameter to make_call() async function with override logic matching make_sync_call()
• Updated async_streaming() to pass the user-provided client as gemini_client (with AsyncHTTPHandler type check) via partial()
• Added two mock-only unit tests verifying gemini_client is correctly wired through partial() keywords

Confidence Score: 4/5

• This PR is safe to merge — it's a minimal, targeted fix that mirrors an existing working pattern.
• The code change is small (4 lines of logic) and directly mirrors the established make_sync_call pattern. The fix correctly propagates the user's async client through partial() and the CustomStreamWrapper call chain. No other callers of make_call exist, so there's no risk of breaking other code paths. Tests are present but shallow (verify partial wiring only, not actual invocation).
• No files require special attention — the change is consistent with the existing sync pattern.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Adds gemini_client parameter to make_call() async function, mirroring the existing make_sync_call() pattern to ensure user-specified async clients are passed through for streaming calls.
tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py	Adds two tests verifying gemini_client is passed through partial() keywords for both async and sync streaming paths. Tests are shallow — they verify partial wiring but don't invoke the actual functions.

Sequence Diagram

sequenceDiagram
    participant User as User Code
    participant AC as litellm.acompletion()
    participant AS as VertexLLM.async_streaming()
    participant CSW as CustomStreamWrapper
    participant MC as make_call()
    participant Client as HTTP Client

    User->>AC: acompletion(stream=True, client=custom_client)
    AC->>AS: async_streaming(client=custom_client)
    AS->>CSW: CustomStreamWrapper(make_call=partial(make_call, gemini_client=custom_client, ...))
    Note over CSW: Stores partial for lazy invocation
    CSW->>MC: await make_call(client=module_level_aclient)
    alt gemini_client is not None
        MC->>MC: client = gemini_client (user's client wins)
    end
    MC->>Client: client.post(api_base, stream=True)
    Client-->>MC: streaming response
    MC-->>CSW: ModelResponseIterator
    CSW-->>User: async stream chunks

Loading

_{Last reviewed commit: 872554d}