Litellm oss staging 050626

[Sameerlite]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
13 out of 14 committers have signed the CLA.

✅ yudelevi
✅ Cyberfilo
✅ saswatds
✅ brainsparker
✅ Zhao73
✅ kapelame
✅ danisalvaa
✅ ErRickow
✅ urainshah
✅ abhay23-AI
✅ mateo-berri
✅ Sameerlite
✅ dan2k3k4
❌ hectorc98
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 96.34369% with 20 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/soniox/common_utils.py	93.18%	9 Missing ⚠️
litellm/llms/soniox/audio_transcription/handler.py	98.53%	3 Missing ⚠️
litellm/llms/you_com/search/transformation.py	95.45%	3 Missing ⚠️
litellm/main.py	50.00%	2 Missing ⚠️
litellm/experimental_mcp_client/client.py	0.00%	1 Missing ⚠️
.../litellm_core_utils/get_supported_openai_params.py	66.66%	1 Missing ⚠️
.../llms/soniox/audio_transcription/transformation.py	98.96%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This staging PR bundles several independent features and fixes: a new Soniox async audio transcription provider, a You.com web search provider, per-server configurable timeouts for MCP servers (including a DB migration), Moonshot reasoning-model temperature handling, Cohere max_completion_tokens support, an OpenMeter attribution hardening flag, and a Langfuse SSL-aware httpx.Client injection.

• Soniox provider (llms/soniox/): multi-step async transcription flow (upload → create → poll → fetch → cleanup) built on litellm's existing HTTPHandler/AsyncHTTPHandler; timeout is now correctly forwarded to all get() calls via the new optional timeout parameter added to http_handler.py.
• MCP per-server timeout: threaded from config/DB through MCPServer.timeout into _create_mcp_client and guarded by asyncio.wait_for in _call_regular_mcp_tool, raising HTTP 504 on asyncio.TimeoutError.
• You.com search: get_complete_url receives the resolved api_key from search/main.py, fixing the endpoint-selection bug flagged in the prior review round.

Confidence Score: 5/5

Safe to merge; all changes are additive or targeted fixes with no backward-incompatible modifications to existing paths.

The Soniox and You.com integrations are brand-new code paths that cannot regress existing functionality. The MCP timeout change only activates when a server explicitly sets a timeout (None falls back to the previous constant). The Cohere and Moonshot changes are narrow param-mapping fixes. Issues raised in the previous review round are all addressed in this revision.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/soniox/audio_transcription/handler.py	New Soniox async transcription handler; uses litellm's existing HTTPHandler/AsyncHTTPHandler, properly forwards timeout to all GET/POST calls, and includes secret-field redaction before logging.
litellm/llms/you_com/search/transformation.py	New You.com search provider; get_complete_url now correctly receives api_key from search/main.py via kwargs, resolving the previously-flagged endpoint selection bug.
litellm/proxy/_experimental/mcp_server/mcp_server_manager.py	Adds per-server timeout: threaded from config/DB through MCPServer into _create_mcp_client; _call_regular_mcp_tool now wraps gather with asyncio.wait_for and catches TimeoutError as HTTP 504.
litellm/llms/custom_httpx/http_handler.py	Adds optional timeout parameter to both sync and async get() methods, forwarding it to the underlying httpx call while falling back to UseClientDefault when None.
litellm/integrations/langfuse/langfuse_prompt_management.py	Creates a dedicated fresh httpx.Client (not the shared cached handler) for Langfuse when SDK >= 2.7.3; correctly avoids the shared-client-close risk flagged in the previous review.
litellm/llms/moonshot/chat/transformation.py	Drops temperature entirely for reasoning models (kimi-k2.5/k2.6) instead of clamping, using supports_reasoning() from model cost map rather than hardcoded model names.
litellm/integrations/openmeter.py	Adds OPENMETER_TRUST_REQUEST_USER env flag: when false, ignores request-supplied user field and falls back to key-bound user_api_key_user_id for attribution.

_{Reviews (10): Last reviewed commit: "fix(cohere): support max_completion_toke..." | Re-trigger Greptile}

[veria-ai]

PR overview

This PR updates LiteLLM provider integration behavior for Soniox transcription, You.com search, and JSON-backed OpenAI-compatible providers. The touched code covers provider base URL handling, environment-based API key resolution, and Soniox audio transcription request transformation.

There are still open issues that allow authenticated callers to steer provider requests to attacker-controlled endpoints while the proxy attaches shared provider credentials, exposing Soniox, You.com, or Neosantara API keys. A separate Soniox transcription path may also let callers bypass proxy-side file size and ownership checks by referencing external audio URLs or existing file IDs. Two issues have already been addressed, but the remaining findings include clear credential-exposure paths and should be fixed before production use.

Open issues (4)

• High: Provider credential exfiltration — litellm/llms/soniox/common_utils.py:61
• High: Provider API key exfiltration — litellm/llms/you_com/search/transformation.py:88
• Medium: Audio size and resource ownership bypass — litellm/llms/soniox/audio_transcription/transformation.py:48
• High: Provider API key exfiltration — litellm/llms/openai_like/providers.json:120

Fixed/addressed: 2 · PR risk: 8/10

[Sameerlite]

@greptileai

[Sameerlite]

@greptileai

[Sameerlite]

@greptileai

[mateo-berri]

@greptileai

[veria-ai]

High: Provider credential exfiltration

When SONIOX_API_BASE is unset, this falls back to the request-provided api_base; the Soniox handler then sends Authorization: Bearer <SONIOX_API_KEY> to that URL. A caller can submit a Soniox transcription request with api_base set to an attacker-controlled host and capture the proxy's Soniox API key. Pin this provider to the configured/default Soniox host, or only allow api_base overrides when they come from trusted server-side configuration rather than the request.

[mateo-berri]

Resolving the provider key from SONIOX_API_KEY and attaching it to whatever api_base wins is the exact resolution litellm's canonical transcription handlers use. The OpenAI and Azure paths in litellm/main.py both do api_key = api_key or get_secret(...) and api_base = api_base or env or default, then send that env-resolved key to the resolved base. Soniox is actually stricter than those: get_soniox_api_base gives the SONIOX_API_BASE env var precedence over the caller-supplied value, so an operator who pins the base via env cannot be redirected by a request, which the OpenAI/Azure handlers do not offer

Honoring a request-supplied api_base is an intentional litellm behavior, not a Soniox-specific gap. The proxy treats clientside api_base/api_key as the highest-precedence override (see _apply_credential_overrides_from_model_config, "Respect clientside credentials, highest precedence") and routes it through the dynamic-deployment path in get_dynamic_litellm_params, which is where admin secrets are scrubbed on redirect. Constraining that for env-resolved keys belongs in that core layer and applies uniformly to every provider, not in this handler

A provider-level pin is not viable without a real regression. Operator configuration via config.yaml litellm_params.api_base and SDK calls like litellm.transcription(model="soniox/...", api_base=...) both reach this function as the same api_base argument and are indistinguishable from a request override at the provider layer, so dropping the argument would break legitimate custom-endpoint use. BYOK stays safe: a caller who passes their own api_key alongside api_base sends their key, not the server's

[mateo-berri]

@greptileai

[veria-ai]

High: Provider API key exfiltration

api_base can come from the /search request, while validate_environment() resolves YOUCOM_API_KEY and attaches it as X-API-Key. An authenticated caller can set search_provider=you_com and api_base to a server they control, causing the proxy to POST the configured You.com key to that server; only honor custom bases when they are admin-configured or paired with a caller-supplied BYOK key.

[mateo-berri]

You.com resolves credentials exactly like the Perplexity search provider already shipped in litellm. PerplexitySearchConfig.validate_environment does api_key = api_key or get_secret_str("PERPLEXITYAI_API_KEY") and attaches it as a header unconditionally, and its get_complete_url does api_base = api_base or env or default, so a request-supplied base receives the env-configured key in precisely the same way. This is the established litellm search-provider pattern, not a defect introduced here

As with every provider, a request that supplies api_base is handled by the proxy/router clientside-credential mechanism by design (is_clientside_credential / get_dynamic_litellm_params); narrowing that for env-based keys is a core-layer change that applies to all providers, not something this transformation can or should special-case. Pinning api_base at the provider level would break admin-configured custom bases and SDK api_base usage, and would diverge from Perplexity and the other search configs

BYOK is honored: a caller who passes their own api_key alongside api_base sends their own key rather than the server's YOUCOM_API_KEY

[veria-ai]

Medium: Audio size and resource ownership bypass

The proxy checks max_file_size_mb against the uploaded multipart file before routing, but these passthrough fields let a caller include a tiny file and have Soniox transcribe an arbitrary audio_url or existing file_id under the shared provider credentials. Reject these fields on proxy-routed requests unless the referenced audio is server-owned and goes through the same size and ownership checks.

[mateo-berri]

audio_url and file_id are first-class Soniox async-transcription parameters; exposing them as passthrough kwargs is intended feature parity and is used from both the SDK and the proxy

max_file_size_mb (check_file_size_under_limit) bounds the multipart upload the proxy ingests into memory. When audio_url or file_id is supplied the handler skips the upload step entirely (handler.py: if not file_id and not handler_opts.get("audio_url")), so no audio is read into the proxy and there is no proxy-side memory pressure or SSRF surface; Soniox fetches the URL server-side, not the proxy

Cost is neither unbounded nor unbilled. The handler now records audio_transcription_duration from the Soniox audio_duration_ms on every path (upload, audio_url, file_id) and the model is priced per second, so transcribing an audio_url is metered and decremented from the caller's litellm budget exactly like an uploaded file. That is the spend-tracking fix already landed in this PR

file_id references resolve against the operator's single Soniox account. litellm uses shared upstream provider credentials for every provider, so there is no per-litellm-user resource isolation at the provider account level for this to bypass, and default cleanup deletes proxy-uploaded files once the transcript is fetched. Rejecting these fields on proxy requests would remove a legitimate Soniox capability without closing a proxy-side resource or data-isolation gap

[mateo-berri]

@greptileai

[veria-ai]

High: Provider API key exfiltration

JSON-backed providers resolve the API key from this env var but preserve a caller-supplied api_base before falling back to the configured base URL. A user who can call neosantara/... can set api_base to their own server and receive Authorization: Bearer <NEOSANTARA_API_KEY> on chat or responses requests; pin the base URL whenever the proxy is using the environment key, or only allow custom api_base when the caller also supplies their own api_key.

[mateo-berri]

This neosantara block is pure data added to providers.json; it carries no credential-handling code of its own. Key and base resolution is the shared JSONProviderConfig (_get_openai_compatible_provider_info) for chat and JSONProviderResponsesConfig for /v1/responses, the same dynamic config classes that already back every openai_like JSON provider in this file (tensormesh and the rest). Both do resolved_base = api_base or get_secret_str(api_base_env) or base_url and resolved_key = api_key or get_secret_str(api_key_env), which is the identical api_key or env + api_base or env or default precedence that OpenAI, Azure, and Perplexity already use in litellm/main.py. Adding a JSON row gives neosantara exactly the behavior of the providers already shipped; it introduces no new code path to exploit

Honoring a request-supplied api_base is intentional litellm behavior, not a neosantara-specific gap. The proxy treats clientside api_base/api_key as the highest-precedence override and routes it through the dynamic-deployment path in get_dynamic_litellm_params, which is where an admin's config-stored secrets (organization, extra_headers, region, provider tokens, etc.) are scrubbed when the caller redirects the base. Constraining env-resolved keys belongs in that core layer and applies uniformly to every provider; special-casing it in this one JSON entry, or in the shared config class, would diverge from every other openai_like provider without closing the generic behavior

Pinning the base whenever the env key is used would break legitimate operator usage. config.yaml litellm_params.api_base and SDK calls like litellm.completion(model="neosantara/...", api_base=...) reach _get_openai_compatible_provider_info as the same api_base argument and are indistinguishable from a request override at the provider layer, so pinning would break admin-configured custom endpoints and self-hosted neosantara deployments

BYOK stays safe: a caller who passes their own api_key alongside api_base sends their own key rather than the server's NEOSANTARA_API_KEY, because resolved_key = api_key or env prefers the caller-supplied key

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

LGTM; thanks!