fix(params): keep _litellm_* kwargs out of provider request bodies by construction by shrey-berri · Pull Request #43221 · BerriAI/litellm

shrey-berri · 2026-09-25T21:29:28Z

TLDR

Problem this solves:

LiteLLM's own internal settings could leak into provider requests
The provider then rejects the whole request with a 400

How it solves it:

A kwarg named _litellm_... now always counts as LiteLLM's own
Every place that filters kwargs for a provider now drops them

Intentional product change: a request key starting with _litellm_ is now dropped before the provider call instead of being forwarded

LiteLLM decides what to send a provider by removing the kwargs it owns. Until now, "owns" meant "appears in all_litellm_params", a list someone has to update by hand. When a new internal setting was added without updating that list, it went into the provider's request body, and strict providers like OpenAI answered Unknown parameter

This PR adds one function, is_litellm_owned_kwarg, that owns anything in that list plus any name starting with _litellm_. The prefix lives in litellm/constants.py as INTERNAL_KWARG_PREFIX. Every place that strips LiteLLM's kwargs before a provider call now asks that function: chat completions, transcription, search, video generation, embeddings, image generation, image edit, ElevenLabs text to speech and Bedrock batches. A name with _litellm_ only in the middle, like provider_litellm_knob, still goes to the provider

The tests send a _litellm_ key down each of those paths and check that the provider never sees it. One of them reads the raw HTTP body sent to six providers. If the function stops checking the prefix, 28 tests fail

User Flow

Before: a request carrying a _litellm_ key fails at the provider

A developer sends POST http://localhost:4000/v1/chat/completions for an OpenAI model with "_litellm_sentinel": "x" in the body
OpenAI rejects it, and the developer gets a 400 saying Unknown parameter: '_litellm_sentinel'

After: the same request succeeds, and the key never leaves LiteLLM

The developer sends the same POST with "_litellm_sentinel": "x"
They get a 200 chat completion

Linear ticket

Resolves LIT-8318

Resolves LIT-8319

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

On main, only chat completions leaked the key. OpenAI traffic on /v1/messages and /v1/responses goes through the Responses API, which already dropped unknown keys, so those two cases show nothing changed there. Each response below is trimmed to its reply text and HTTP status

Setup: a proxy on localhost:4417 with $LITELLM_MASTER_KEY set and this config:

model_list:
  - model_name: live-gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_key: os.environ/OPENAI_API_KEY

Before (`8327cd6`)

/v1/chat/completions

Send a chat completion with a _litellm_ key

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "messages": [{"role": "user", "content": "say hi"}], "max_tokens": 16, "_litellm_sentinel": "x"}'

Observed:

{"error":{"message":"litellm.BadRequestError: OpenAIException - Unknown parameter: '_litellm_sentinel'. ...","type":"invalid_request_error","param":"_litellm_sentinel","code":"400"}}
HTTP 400

/v1/messages

Send the same key to /v1/messages

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/messages \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "max_tokens": 16, "messages": [{"role": "user", "content": "say hi"}], "_litellm_sentinel": "x"}'

Observed:

{"type":"message","role":"assistant", ... "content":[{"type":"text","text":"Hi!"}],"stop_reason":"end_turn", ...}
HTTP 200

/v1/responses

Send the same key to /v1/responses

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "input": "say hi", "max_output_tokens": 16, "_litellm_sentinel": "x"}'

Observed:

{"object":"response", ... "output":[{"content":[{"text":"Hi!","type":"output_text", ...}], ...}],"status":"completed", ...}
HTTP 200

After (`fac0829`)

/v1/chat/completions

Send the same chat completion

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "messages": [{"role": "user", "content": "say hi"}], "max_tokens": 16, "_litellm_sentinel": "x"}'

Observed:

{"object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi!","role":"assistant", ...}}], ...}
HTTP 200

/v1/messages

Send the same key to /v1/messages

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/messages \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "max_tokens": 16, "messages": [{"role": "user", "content": "say hi"}], "_litellm_sentinel": "x"}'

Observed:

{"type":"message","role":"assistant", ... "content":[{"type":"text","text":"Hi!"}],"stop_reason":"end_turn", ...}
HTTP 200

/v1/responses

Send the same key to /v1/responses

curl -s -w '\nHTTP %{http_code}' http://localhost:4417/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "live-gpt-5.4-mini", "input": "say hi", "max_output_tokens": 16, "_litellm_sentinel": "x"}'

Observed:

{"object":"response", ... "output":[{"content":[{"text":"Hi!","type":"output_text", ...}], ...}],"status":"completed", ...}
HTTP 200

Type

🐛 Bug Fix

Caveats (if any)

Low

A client-sent _litellm_ key is dropped silently, not rejected
The live proof ran at fac0829, two revisions before this head
- Those revisions moved the prefix check into one function and covered more paths. Chat behavior did not change

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

Medium Risk
Touches shared kwarg classification used on many API paths; behavior change drops client _litellm_* keys silently, but scope is narrow and well-covered by tests.

Overview
Introduces is_litellm_owned_kwarg and INTERNAL_KWARG_PREFIX (_litellm_) so LiteLLM-owned kwargs are recognized by name prefix as well as the existing all_litellm_params list. Undeclared _litellm_* keys are stripped before provider calls instead of being forwarded (fixing 400s from strict APIs).

Provider-param filtering now uses this helper in completion/transcription/embeddings paths (filter_out_litellm_params, get_non_default_*), image generation/edit, Bedrock batch JSONL mapping, and ElevenLabs TTS. Keys like provider_litellm_knob (prefix not at start) still pass through.

Tests assert _litellm_undeclared_sentinel never appears in outbound HTTP bodies or provider payloads across those surfaces.

^{Reviewed by Cursor Bugbot for commit de2a4d1. Bugbot is set up for automated code reviews on this repo. Configure here.}

codecov · 2026-09-25T21:40:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

codspeed · 2026-09-25T21:42:18Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_internal_kwarg_prefix_v2 (de2a4d1) with main (cd1107a)¹}

No successful run was found on main (474ab91) during the generation of this report, so cd1107a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

shrey-berri · 2026-09-25T21:46:19Z

@greptileai

shrey-berri · 2026-09-25T21:46:20Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

greptile-apps · 2026-09-25T21:49:57Z

Confidence Score: 5/5

[Medium risk] Filters internal litellm parameters from provider requests.

The PR appears safe to merge based on the reviewed changes.

Summary

The PR classifies _litellm_* kwargs as internal and applies that classification across provider-facing parameter filters.

Adds request-body and classifier tests covering the prefix across multiple provider paths.
The latest revision removes two redundant docstring additions. All three previous Greptile findings are resolved or fixed.

_{Reviews (5) · Last reviewed commit: "fix(params): keep _litellm_* kwargs out ..."}

shrey-berri · 2026-09-25T22:24:17Z

@greptileai

shrey-berri · 2026-09-25T22:24:19Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

shrey-berri · 2026-09-25T23:12:02Z

@greptileai

Generated by Claude Code

shrey-berri · 2026-09-25T23:12:04Z

bugbot run

Generated by Claude Code

cursor

Stale Bugbot comment from a previous run.

shrey-berri · 2026-09-25T23:30:29Z

@greptileai

Generated by Claude Code

shrey-berri · 2026-09-25T23:30:30Z

bugbot run

Generated by Claude Code

cursor

Stale Bugbot comment from a previous run.

… construction Kwargs LiteLLM code introduces for its own use were only kept out of provider bodies if someone also listed them in all_litellm_params. Undeclared ones went into extra_body or optional_params, reached the provider, and the provider rejected the request. is_litellm_owned_kwarg in types/utils.py now defines LiteLLM-owned once: a registered name, or any name starting with INTERNAL_KWARG_PREFIX from litellm/constants.py. Every filter that builds provider params from kwargs uses it: chat completion, transcription, embedding, image generation and edit, search and video, ElevenLabs text to speech, and the Bedrock batch mapper. The two untyped shared filters now take Mapping[str, object] The stream_chunk_size wire test becomes test_internal_params_wire.py. It also sends an undeclared _litellm_ kwarg and asserts that no _litellm_ key reaches any of the six provider bodies, while extra_body passthrough keeps working Refs LIT-8318, LIT-8319

shrey-berri · 2026-09-25T23:38:40Z

@greptileai

Generated by Claude Code

shrey-berri · 2026-09-25T23:38:41Z

bugbot run

Generated by Claude Code

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit de2a4d1. Configure here.}

mateo-berri

LGTM. Thanks!

shrey-berri mentioned this pull request Sep 25, 2026

fix(params): validate stream_chunk_size once, before any provider call #43222

Merged

6 tasks done

shrey-berri marked this pull request as ready for review September 25, 2026 21:46

shrey-berri requested a review from a team September 25, 2026 21:46