fix(gemini): preserve toolConfig on native generate_content

[emerzon]

Relevant issues

Fixes #23491

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix
✅ Test

Changes

This fixes the native Google/Vertex generate_content path dropping top-level toolConfig.

Before this change, LiteLLM preserved contents but rebuilt the downstream native request body without toolConfig. For affected Gemini tool-calling requests, that changed provider behavior from a normal follow-up tool call into a single empty STOP event.

This PR:

• threads toolConfig / tool_config through the native generate_content setup path
• passes toolConfig through BaseLLMHTTPHandler.generate_content_handler() and async_generate_content_handler()
• includes toolConfig in both Google-native and Vertex-native transform_generate_content_request() output
• adds regression coverage to verify native request transforms preserve toolConfig
• updates the proxy request test to assert the outbound native request body includes toolConfig

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 12, 2026 11:07pm

[greptile-apps]

Greptile Summary

This PR fixes a regression in the native Google/Vertex generate_content path where toolConfig was being silently dropped, causing tool-calling requests to receive an empty STOP response. It threads the tool_config field through every entrypoint (generate_content, agenerate_content_stream, generate_content_stream, setup_generate_content_call) down to both concrete transform_generate_content_request implementations (Gemini and Vertex AI). As a companion fix, system_instruction — which had the same omission in the sync generate_content_stream path — is also extracted and forwarded there.

Key changes:

• New _get_tool_config_from_kwargs helper uses explicit key presence checks ("toolConfig" in kwargs) instead of falsy evaluation, correctly preserving intentionally empty dict values
• tool_config parameter added to BaseLLMHTTPHandler.generate_content_handler, async_generate_content_handler, and both transform_generate_content_request overrides
• VertexAIGoogleGenAIConfig also aligns the system_instruction guard from a falsy check to is not None, removing a pre-existing inconsistency
• Regression coverage added: parameterized transformation test across both provider configs, a sync-stream forwarding test, and an updated proxy request test that asserts toolConfig in the outbound body

Confidence Score: 4/5

• This PR is safe to merge; it is a focused, additive parameter-threading fix with no backwards-incompatible changes and good mock-based test coverage.
• The fix is mechanically straightforward — every call site that was missing tool_config has been updated and backed by a test. The helper function avoids the falsy-dict footgun. The only deduction is that the agenerate_content_stream path still has no test asserting tool_config forwarding (the existing test only asserts stream=True), and the redundant transform_generate_content_request call inside setup_generate_content_call remains a maintenance footgun for future parameter additions — both noted in prior review threads.
• No files require special attention; all changed files have been reviewed and the logic is consistent.

Important Files Changed

Filename	Overview
litellm/google_genai/main.py	Adds _get_tool_config_from_kwargs helper, extracts and forwards tool_config through all four call paths (generate_content, agenerate_content_stream, generate_content_stream, setup_generate_content_call). Also backfills the missing system_instruction extraction in generate_content_stream, addressing the issue flagged in the previous review thread.
litellm/llms/custom_httpx/llm_http_handler.py	Adds tool_config parameter to both generate_content_handler and async_generate_content_handler, and correctly threads it into transform_generate_content_request in both the sync path and the _is_async delegation path.
litellm/llms/gemini/google_genai/transformation.py	Adds tool_config parameter to transform_generate_content_request and emits toolConfig into the request dict when the value is not None. Consistent with the system_instruction guard pattern.
litellm/llms/vertex_ai/google_genai/transformation.py	Adds tool_config to transform_generate_content_request, correctly guarded with if tool_config is not None. Also aligns system_instruction guard from falsy check to is not None — a pre-existing inconsistency addressed as a bonus improvement.
litellm/llms/base_llm/google_genai/transformation.py	Updates the abstract base class signature to include tool_config as an optional parameter, keeping it consistent with all concrete implementations.
tests/test_litellm/google_genai/test_google_genai_main.py	Adds test_generate_content_stream_forwards_system_instruction which asserts both tool_config and system_instruction are forwarded in the sync stream path. Fixes an existing test assertion that was using == as a statement instead of assert. No real network calls — all mocked.
tests/test_litellm/google_genai/test_google_genai_transformation.py	Adds parameterized test_transform_generate_content_request_preserves_tool_config covering both GoogleGenAIConfig and VertexAIGoogleGenAIConfig. Updates existing tests to pass tool_config=None explicitly after the new required param was added to the signature.
tests/proxy_unit_tests/test_google_gemini_proxy_request.py	Injects toolConfig into the sample payload and asserts it is present in the outbound HTTP request body, providing end-to-end regression coverage for the proxy path.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant generate_content / agenerate_content_stream / generate_content_stream
    participant setup_generate_content_call
    participant BaseLLMHTTPHandler
    participant async_generate_content_handler
    participant transform_generate_content_request

    Caller->>generate_content / agenerate_content_stream / generate_content_stream: call(kwargs incl. toolConfig)
    generate_content / agenerate_content_stream / generate_content_stream->>setup_generate_content_call: setup(..., **kwargs)
    setup_generate_content_call->>transform_generate_content_request: transform(..., tool_config)
    Note over setup_generate_content_call: request_body stored in SetupResult (unused by callers)
    setup_generate_content_call-->>generate_content / agenerate_content_stream / generate_content_stream: SetupResult

    generate_content / agenerate_content_stream / generate_content_stream->>generate_content / agenerate_content_stream / generate_content_stream: tool_config = _get_tool_config_from_kwargs(kwargs)
    generate_content / agenerate_content_stream / generate_content_stream->>BaseLLMHTTPHandler: generate_content_handler(..., tool_config)

    alt _is_async = True
        BaseLLMHTTPHandler->>async_generate_content_handler: async_generate_content_handler(..., tool_config)
        async_generate_content_handler->>transform_generate_content_request: transform(..., tool_config)
        transform_generate_content_request-->>async_generate_content_handler: {contents, tools, toolConfig, ...}
    else sync
        BaseLLMHTTPHandler->>transform_generate_content_request: transform(..., tool_config)
        transform_generate_content_request-->>BaseLLMHTTPHandler: {contents, tools, toolConfig, ...}
    end

    BaseLLMHTTPHandler->>BaseLLMHTTPHandler: HTTP POST with toolConfig in body

Loading

_{Last reviewed commit: 296ea95}

[Copilot]

Pull request overview

This PR adds support for passing toolConfig through LiteLLM’s Google GenAI generate_content request flow (including Vertex AI’s Google GenAI-compatible path), and updates unit tests to validate the field is preserved end-to-end.

Changes:

• Thread tool_config from litellm.google_genai.main into the HTTP handler and provider transform_generate_content_request(...).
• Add toolConfig mapping in both GoogleGenAIConfig and VertexAIGoogleGenAIConfig request transformations.
• Extend tests to assert toolConfig is preserved in transformed requests and outgoing HTTP JSON bodies.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
litellm/google_genai/main.py	Extracts toolConfig/tool_config from kwargs and forwards it into request construction + handler calls.
litellm/llms/custom_httpx/llm_http_handler.py	Adds tool_config plumbing into sync/async generate_content handler signatures and transform calls.
litellm/llms/base_llm/google_genai/transformation.py	Updates the base interface signature/docs to include tool_config.
litellm/llms/gemini/google_genai/transformation.py	Adds toolConfig into the Google GenAI request payload.
litellm/llms/vertex_ai/google_genai/transformation.py	Adds toolConfig into the Vertex AI Google GenAI-format request payload.
tests/test_litellm/google_genai/test_google_genai_transformation.py	Adds/updates tests to ensure toolConfig is preserved for both Google and Vertex configs.
tests/proxy_unit_tests/test_google_gemini_proxy_request.py	Verifies the outbound HTTP request JSON includes the provided toolConfig.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.