feat(mistral): support native web search via the Conversations API

[ElDjeee]

TLDR

Problem this solves:

• Mistral's built-in web search is not reachable through LiteLLM
• It only exists on Mistral's Conversations API, not on chat completions
• supports_web_search was hardcoded true for every Mistral model, embeddings included
• The key/team allowed_tools allowlist ignored built-in web search tools

How it solves it:

• Web search requests are rerouted to POST /v1/conversations transparently
• Outputs map back to an OpenAI chat response with url_citation annotations
• Search calls are billed per call on top of token cost
• supports_web_search moves onto the Mistral chat entries in the cost map
• allowed_tools now covers web_search / web_search_premium / web_search_options

User Flow

Before: a developer asking a Mistral model to search the web gets a provider error, because the chat completions endpoint has no web search

1. They send POST https://litellm-domain/v1/chat/completions with "model": "mistral-medium-latest" and "tools": [{"type": "web_search"}]
2. The gateway forwards it to Mistral's chat completions endpoint as-is
3. They get back a 400 from Mistral rejecting the unknown tool type, or an answer with no live sources when they try web_search_options

After: the same request comes back with a web-grounded answer and its sources

1. They send the same POST https://litellm-domain/v1/chat/completions with "tools": [{"type": "web_search"}] (or "web_search_options": {})
2. They get a 200 with choices[0].message.content answering from the live web and choices[0].message.annotations listing url_citation entries with the source titles and URLs
3. usage.prompt_tokens_details.web_search_requests counts the searches, and the x-litellm-response-cost header includes the per-search charge
4. With "stream": true the same answer and citations arrive as SSE chunks
5. A key whose allowed_tools does not list web_search gets a 403 tool_access_denied on that request instead of a billed search

Relevant issues

Web search requested by a customer

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both runs hit the real Mistral API through a live proxy started with python litellm/proxy/proxy_cli.py --config qa_config.yaml --port 4000 (the Before run from a worktree at the merge base, the After run from this branch's tip). qa_config.yaml:

model_list:
  - model_name: mistral-medium-latest
    litellm_params:
      model: mistral/mistral-medium-latest
      api_key: os.environ/MISTRAL_API_KEY

general_settings:
  master_key: sk-1234

The restricted key used by the third case was minted once with:

curl -sS -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"metadata":{"allowed_tools":["get_weather"]}}'

Before (658f506)

Web search via tools

1. Command

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"mistral-medium-latest",
       "messages":[{"role":"user","content":"Who won the most recent UEFA European Championship, and in what year? Cite sources."}],
       "tools":[{"type":"web_search"}]}'

2. Output: HTTP 400, Mistral rejects the tool on chat completions

{"error":{"message":"litellm.BadRequestError: MistralException - {\"object\":\"error\",\"message\":\"WebSearchTool connector is not supported\",\"type\":\"invalid_tools\",\"param\":null,\"code\":\"1800\",\"raw_status_code\":400}. Received Model Group=mistral-medium-latest\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"400"}}

Streaming web search via web_search_options

1. Command

curl -sS -N http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"mistral-medium-latest",
       "messages":[{"role":"user","content":"Who is the current UN Secretary-General?"}],
       "web_search_options":{},"stream":true}'

2. Output: HTTP 400, the parameter is not supported for Mistral

{"error":{"message":"litellm.UnsupportedParamsError: mistral does not support parameters: ['web_search_options'], for model=mistral-medium-latest. To drop these, set `litellm.drop_params=True` or for proxy:\n\n`litellm_settings:\n drop_params: true`\n. \n If you want to use these params dynamically send allowed_openai_params=['web_search_options'] in your request.. Received Model Group=mistral-medium-latest\nAvailable Model Group Fallbacks=None","type":"None","param":null,"code":"400"}}

Key restricted by allowed_tools

1. Command (with the key minted above, whose allowlist is ["get_weather"])

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-<restricted-key>" -H "Content-Type: application/json" \
  -d '{"model":"mistral-medium-latest",
       "messages":[{"role":"user","content":"Who won the most recent UEFA European Championship?"}],
       "tools":[{"type":"web_search"}]}'

2. Output: the allowlist does not block the request, it reaches Mistral and fails there with the same 400 as the first case

{"error":{"message":"litellm.BadRequestError: MistralException - {\"object\":\"error\",\"message\":\"WebSearchTool connector is not supported\",\"type\":\"invalid_tools\",\"param\":null,\"code\":\"1800\",\"raw_status_code\":400}. Received Model Group=mistral-medium-latest\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"400"}}

After (f54a4c5)

Web search via tools

1. Same command as Before
2. Output: HTTP 200, the request lands on /v1/conversations, citations come back as url_citation annotations and usage.prompt_tokens_details.web_search_requests counts the search

{"id":"conv_01a06779650a769fac194f5a2eb59c08","created":1788442403,"model":"mistral-medium-latest","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The most recent UEFA European Championship was won by **Spain in 2024**.","role":"assistant","annotations":[{"type":"url_citation","url_citation":{"title":"List of UEFA European Championship finals - Wikipedia","url":"https://en.wikipedia.org/wiki/List_of_UEFA_European_Championship_finals"}},{"type":"url_citation","url_citation":{"title":"UEFA European Football Championship Winners Year by Year","url":"https://www.aworldofsoccer.com/tournaments_nt/european_cup_winners.htm"}}]}}],"usage":{"completion_tokens":53,"prompt_tokens":779,"total_tokens":6344,"prompt_tokens_details":{"web_search_requests":1}}}

3. Response headers of the same call: the $0.03 per-search charge ($30 per 1,000 web_search calls) is billed on top of the token cost

x-litellm-response-cost: 0.031566
x-litellm-response-cost-input: 0.0011685
x-litellm-response-cost-output: 0.0003975
x-litellm-response-cost-tool-usage: 0.03

Streaming web search via web_search_options

1. Same command as Before
2. Output: HTTP 200 SSE, served through the fake-stream path with the citations on the delta

data: {"id":"conv_01a06779812c768c9a7f7c692f7da59a","created":1788442459,"model":"mistral-medium-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"As of September 3, 2026, the current UN Secretary-General is António Guterres of Portugal. His second term is set to end on December 31, 2026, and the selection process for his successor is currently underway, with the new Secretary-General expected to take office on January 1, 2027.\n\nWould you like more details about the selection process or the candidates?","role":"assistant","annotations":[{"type":"url_citation","url_citation":{"title":"Secretary-General | United Nations","url":"https://www.un.org/sg/en"}},{"type":"url_citation","url_citation":{"title":"UN Secretary-General Candidates 2026: Who Will Lead Next?","url":"https://pakistanchronicle.com/un-secretary-general-candidates-2026-who-will-lead-next/"}},{"type":"url_citation","url_citation":{"title":"Two More UN Secretary-General Candidates Face Member States – SDG Knowledge Hub","url":"https://sdg.iisd.org/news/two-more-un-secretary-general-candidates-face-member-states/"}}]}}]}

data: {"id":"conv_01a06779812c768c9a7f7c692f7da59a","object":"chat.completion.chunk","created":1788442459,"model":"mistral-medium-latest","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Key restricted by allowed_tools

1. Same command as Before, with the same restricted key
2. Output: HTTP 403, the request is denied before any search is billed

{"error":{"message":"Tool(s) ['web_search'] are not in the allowed tools list for this key/team.","type":"tool_access_denied","param":"tools","code":"403"}}

Type

New Feature

Caveats (if any)

Medium

• Streaming is fake-streamed: one content chunk, since the Conversations route buffers the response
• finish_reason is inferred from the token budget because the Conversations API returns none

Low

• Conversations requests are pinned to store: false; no conversation persistence is exposed

Changes

This PR is a small preparatory refactor, the feature, and a set of follow-up fixes from review

refactor(utils): collapse plain provider dispatch in get_provider_model_info

get_provider_model_info was a flat if/elif chain already sitting at the repo's C901 complexity ceiling of 15, so adding any new provider tips it over the cap. The plain cases that just return a litellm.<X>Config() are collapsed into a read-only mapping lookup, leaving the lazy-import providers as explicit branches. Behavior is identical; this only buys headroom so the feature commit can register mistral without a drive-by refactor bundled into it

feat(mistral): support native web search via the Conversations API

Mistral's built-in web search only exists on the Conversations API, so a completion that asks for web search is routed there transparently while the public API stays as simple as

litellm.completion(
    model="mistral/mistral-medium-latest",
    messages=[{"role": "user", "content": "Who won the last Euro?"}],
    tools=[{"type": "web_search"}],   # or web_search_options={}
)

A request is detected as web search when it carries web_search_options or a tools entry of type web_search / web_search_premium. _complete_mistral swaps the resolved MistralConfig for a sibling MistralConversationsConfig for those requests, the way _complete_bedrock picks invoke vs converse. The swapped config points get_complete_url at /conversations, builds the Conversations request body (system messages to instructions, remaining turns to inputs, sampler args to completion_args, the connector tool to tools) and maps the outputs back to an OpenAI-shaped ModelResponse, surfacing tool_reference chunks as url_citation annotations. Streaming is served through the base handler's fake-stream path, and function calling on chat completions is untouched; only web-search requests are rerouted

A small cost calculator bills Mistral's published rates ($30 per 1,000 web_search calls, $50 per 1,000 web_search_premium calls) off the authoritative usage.connectors count, added on top of token cost through the shared get_cost_for_web_search_request hook

Tests: unit coverage for the Conversations request/response transformation (citation mapping, usage counting from both the connectors count and the tool-execution fallback, edge cases), the endpoint-selection routing for both trigger forms across sync and async plus the negative case that a normal call still hits /chat/completions, the fake-stream path, and the cost calculator. Three live tests in tests/llm_translation/test_mistral_api.py exercise the real wire schema; like every test in that folder they run under the Redis-backed VCR cassette cache (see the folder's Readme), recording against the live API on cache-miss and replaying keyless after that, so they give CI a deterministic signal without needing MISTRAL_API_KEY on every run

fix(mistral): source supports_web_search from the model cost map

supports_web_search was hardcoded True for every mistral model, so embeddings and OCR models wrongly reported support and the flag was invisible in the cost map. It now lives on the individual mistral chat entries in model_prices_and_context_window.json (and its backup), the way xai, gemini and anthropic declare it, so mistral-embed / mistral-ocr-* correctly resolve to False

fix(proxy): enforce allowed_tools on built-in web search tools

extract_request_tool_names only surfaced function tool names, so a key restricted with allowed_tools could still trigger built-in web search. It now also surfaces the web_search / web_search_premium tool types and the web_search_options shorthand (mapped to web_search, or web_search_premium when premium: true), so the existing allowlist check covers them. An invariant test asserts the tool names sent on the wire never exceed the names the allowlist checked

fix(mistral): stop extra_body from overriding sanitized Conversations fields and fix(mistral): reserve instructions and completion_args from the extra_body merge

extra_body was merged into the request body after the transformation ran, so a caller could clobber the allowlist-checked tools, the pinned store: false, or the built inputs / model. Configs can now declare reserved_request_body_keys (default empty) and the HTTP handler drops those keys from extra_body; the Conversations config reserves the sanitized fields: tools, store, inputs, model, plus instructions (built from guardrail-inspected system messages) and completion_args (carries proxy-enforced limits like max_tokens), so extra_body can neither inject an unscanned system prompt nor replace an enforced token cap

fix(mistral): map tool-call history into Conversations inputs

Assistant tool_calls and tool results were flattened into role+content message entries, silently dropping the call binding. They now map to the Conversations function.call / function.result input entry types, preserving id, name and arguments; verified against the live API

fix(mistral): infer finish_reason=length for truncated Conversations responses

The Conversations API returns no finish or stop reason field (verified live), so finish_reason was hardcoded to stop even when the response was truncated. It is now inferred from the token budget: length when completion_tokens fills the requested max_tokens, otherwise stop. A live test forces a real truncation with max_tokens: 1 and asserts length comes back

test(mistral): drop key-skip guards so web search tests replay under VCR

The three live tests skipped when MISTRAL_API_KEY was unset, which opted them out of the folder's VCR replay and gave keyless CI runs no signal from them. The guards are gone, matching BaseLLMChatTest and the rest of the folder, so a keyless run replays the cassette instead of silently skipping

fix(mistral): map seed directly into optional_params so it reaches completion_args

map_openai_params stuffed seed into optional_params["extra_body"], but the HTTP handler pops extra_body before transform_request runs and merges it at the top level of the final body afterwards. The chat body happened to accept top-level random_seed so the bug was invisible there, but on the Conversations route the transform never saw the seed and the merge placed random_seed at the top level of the body instead of inside completion_args. The mapping now writes optional_params["random_seed"] directly, the way watsonx and codestral map their seed equivalents; this also stops the mapping from overwriting a caller-supplied extra_body. The chat wire body is unchanged and the Conversations transform picks random_seed up as a plain completion arg. Pipeline-faithful respx tests drive litellm.completion(seed=...) through the full pipeline for both routes and assert where the seed lands

test(mistral): cover content-part flattening, get_models errors, and cost dispatch

Codecov flagged uncovered lines across the PR; each was a real untested behavior, so they are covered with meaningful assertions: content-part message lists flatten to their text with non-text and malformed parts dropped, an assistant turn with content: null plus tool_calls maps to a function.call entry without an empty message entry, a message.output with null content transforms gracefully, get_models raises cleanly without credentials and on non-2xx responses, the shared get_cost_for_web_search_request dispatcher routes provider mistral to the new cost calculator, and non-dict tool entries in a request body neither crash allowlist extraction nor shadow real tools

refactor(mistral): immutable annotations and a declared premium web search usage field for the ratcheted lint gates and refactor(mistral): satisfy the Final, immutability, and test-quality gates added on the base branch

The base branch ratcheted its lint gates while this PR was open: every local must carry a Final declaration, mutable list/dict builds are flagged, TypedDict fields must be ReadOnly, and tests may neither write litellm module globals nor assert nothing. The Conversations request is now built from ReadOnly TypedDict entries and tuples, web_search_premium_requests is a declared field on PromptTokensDetailsWrapper instead of an ad-hoc attribute, the swapped Conversations config binds to a fresh name instead of rebinding the frozen dispatch context field, and the async routing test injects an httpx-backed AsyncHTTPHandler instead of flipping litellm.disable_aiohttp_transport

test(mistral): cover an untitled web search source in the citation mapping

The rebase split the citation builder into a titled and an untitled branch, and only the titled one had a test. A tool_reference with a URL but no title now has its own case asserting it is still cited and does not carry a null title key, which brings patch coverage back to 100%

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

The PR adds native Mistral web search through the Conversations API while preserving LiteLLM’s chat-completion interface.

• Routes Mistral web-search requests through /v1/conversations and normalizes citations, usage, streaming, and costs.
• Preserves function-call history and seed placement in Conversations requests.
• Moves web-search capability metadata into the model catalog and extends proxy tool allowlist enforcement.
• Protects transformed Conversations fields from extra_body overrides.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/mistral/conversations/transformation.py	Implements Conversations request and response translation, including tool history, citations, usage, costs, and truncation inference; the previously reported issues are addressed.
litellm/llms/mistral/chat/transformation.py	Adds web-search option support and maps seed directly to random_seed so each endpoint can place it correctly.
litellm/main.py	Selects the Conversations configuration only for Mistral web-search requests while retaining normal chat routing otherwise.
litellm/llms/openai/chat/guardrail_translation/handler.py	Includes built-in web-search forms in existing key and team tool-allowlist checks.
litellm/llms/custom_httpx/llm_http_handler.py	Adds provider-declared reserved request keys so extra_body cannot replace sanitized Conversations fields.
model_prices_and_context_window.json	Declares web-search capability on individual Mistral chat model entries rather than globally.
tests/llm_translation/test_mistral_api.py	Exercises Mistral Conversations behavior through the established translation-test record/replay infrastructure.

_{Reviews (9): Last reviewed commit: "refactor(mistral): satisfy the Final, im..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 5 · PR risk: 0/10

[ElDjeee]

@greptileai

[ElDjeee]

@greptileai

[ElDjeee]

@greptileai

[ElDjeee]

@greptileai

[ElDjeee]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing ElDjeee:litellm_mistral_web_search (f54a4c5) with litellm_internal_staging (11a02b9)}

[ElDjeee]

@greptileai