fix(gemini): pop tool_choice from optional_params when creating cached content

[enzoomoreira]

Summary

When using context caching with Gemini + tool calling, check_and_create_cache() correctly pops tools from optional_params and includes them in the cached content (added in #11989). However, tool_choice is not being popped, causing it to appear as toolConfig in the final GenerateContent request alongside cachedContent.

The Gemini API rejects this with error 400:

CachedContent can not be used with GenerateContent request setting
system_instruction, tools or tool_config

Root cause

In both check_and_create_cache() and async_check_and_create_cache():

tools = optional_params.pop("tools", None)  # tools correctly popped
# tool_choice NOT popped -- remains in optional_params

Later in _transform_request_body(), tool_choice is read from optional_params and added as toolConfig to the request, conflicting with cachedContent.

Fix

Pop tool_choice alongside tools and include it as toolConfig in the cached content request body. The Gemini CachedContent API accepts toolConfig as a valid field.

tools = optional_params.pop("tools", None)
tool_choice = optional_params.pop("tool_choice", None)
# ...
cached_content_request_body["tools"] = tools
if tool_choice is not None:
    cached_content_request_body["toolConfig"] = tool_choice

Applied to both sync and async code paths.

Reproduction

import litellm
import asyncio

litellm.drop_params = True

response = await litellm.acompletion(
    model="gemini/gemini-2.5-flash",
    messages=[
        {"role": "system", "content": [
            {"type": "text", "text": "You are an analyst...(large prompt)",
             "cache_control": {"type": "ephemeral"}}
        ]},
        {"role": "user", "content": "What is ROAE?"},
    ],
    tools=[{"type": "function", "function": {"name": "search", ...}}],
    tool_choice="auto",  # <-- this causes the 400 error
)

Before fix: BadRequestError 400: CachedContent can not be used with GenerateContent request setting ... tool_config

After fix: Cache hit works correctly -- cache_read_input_tokens > 0 in usage stats.

Tests

Added 4 new test cases (sync/async x cached/non-cached) mirroring the existing tools tests:

• test_check_and_create_cache_tool_choice_popped_from_optional_params
• test_check_and_create_cache_tool_choice_not_popped_when_no_cached_messages
• test_async_check_and_create_cache_tool_choice_popped_from_optional_params
• test_async_check_and_create_cache_tool_choice_not_popped_when_no_cached_messages

All 86 tests pass (74 existing + 12 new across 3 providers).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 14, 2026 0:42am

[CLAassistant]

All committers have signed the CLA.

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing enzoomoreira:fix/gemini-cache-tool-choice (a655c53) with main (8427534)}

[codecov]

Codecov Report

❌ Patch coverage is 80.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ex_ai/context_caching/vertex_ai_context_caching.py	80.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

Fixes a 400 BadRequest from the Gemini CachedContent API caused by tool_choice remaining in optional_params and appearing as toolConfig in the GenerateContent request alongside cachedContent. The fix pops tool_choice together with tools in both check_and_create_cache and async_check_and_create_cache, includes it in the cache key, and embeds it as toolConfig in the cached content creation request. The previous-thread concern about tool_choice missing from get_cache_key is resolved here.

Confidence Score: 5/5

Safe to merge — the fix is correct, targeted, and both sync and async paths are handled symmetrically.

All remaining findings are P2 (test coverage gaps and test realism). The core fix is straightforward and low-risk, the cache-key concern from the previous thread is addressed, and no existing tests were weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py	Pops tool_choice from optional_params alongside tools in both sync and async paths, adds it to the cache key, and includes it as toolConfig in the cached content request body. Fix is correct and complete.
tests/test_litellm/llms/vertex_ai/context_caching/test_vertex_ai_context_caching.py	Adds 4 new tool_choice tests and reformats existing tests. New tests verify the pop behavior but don't exercise the HTTP path where toolConfig is placed in the request body, and use raw string "auto" rather than the ToolConfig object that production code produces.

Sequence Diagram

sequenceDiagram
    participant C as Caller
    participant T as transform_request_body
    participant CC as check_and_create_cache
    participant G as Gemini API

    C->>T: optional_params with tools and tool_choice
    T->>CC: optional_params
    CC->>CC: pop tools from optional_params
    CC->>CC: pop tool_choice from optional_params
    CC->>CC: get_cache_key with tools and tool_choice
    alt cache miss - create new cache
        CC->>G: POST cachedContents with tools and toolConfig
        G-->>CC: cache name
    end
    CC-->>T: messages, optional_params without tools or tool_choice, cache name
    T->>T: build GenerateContent request with cachedContent
    Note over T: No toolConfig conflict because tool_choice was already removed

Loading

_{Reviews (2): Last reviewed commit: "fix(gemini): include tool_choice in cach..." | Re-trigger Greptile}

[greptile-apps]

tool_choice omitted from cache key

tool_choice is popped and embedded in the cached content, but it is not passed to get_cache_key. Two calls with the same messages, tools, and model but different tool_choice values hash to the same key. The second call hits the first call's cached content (containing the wrong toolConfig), tool_choice is then discarded from optional_params, and the GenerateContent request proceeds with the wrong calling mode silently.

The key should include tool_choice — get_cache_key accepts **kwargs so the argument can be added directly. The same fix is needed in the equivalent get_cache_key call inside async_check_and_create_cache.