Skip to content

Log when drop_params discards caller-specified params - #7

Merged
jwbron merged 2 commits into
mainfrom
drop-params-visibility-fork
Jul 25, 2026
Merged

Log when drop_params discards caller-specified params#7
jwbron merged 2 commits into
mainfrom
drop-params-visibility-fork

Conversation

@jwbron

@jwbron jwbron commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Found while investigating why egg's agents were ignoring a configured reasoning_effort (jwbron/egg#3599).

The problem

drop_params exists so an unsupported parameter does not fail the whole request, and that tradeoff is right. But dropping a parameter changes generation behaviour, and today it happens with no signal at all — the branch that pops them is a bare loop with no logging:

if litellm.drop_params is True or (drop_params is not None and drop_params is True):
    for k in unsupported_params.keys():
        non_default_params.pop(k, None)

So a reasoning_effort, temperature or penalty set in a proxy config simply never reaches the provider, and nothing in the logs or the response says so. The config and the wire disagree, silently and indefinitely.

Why it is easy to hit

Providers whose supported-param set is derived from the model-cost map fail closed on models the map doesn't know. OpenrouterConfig.get_supported_openai_params advertises reasoning_effort/thinking only when litellm.supports_reasoning(model) is true, which reads that map:

Model In model_cost supports_reasoning
qwen/qwen3-max False
moonshotai/kimi-k2-thinking False
deepseek/deepseek-r1 True

OpenRouter adds models faster than the map tracks them, so an unflagged slug is a routine state rather than an exotic one, and reasoning_effort set on one is a silent no-op. Diagnosing that today means reading get_optional_params and the model map by hand; that is how this was found, and it took a while.

This is not an OpenRouter quirk. OpenRouter is just where I hit it. Nineteen provider configs currently call supports_reasoning from inside get_supported_openai_params, and the ones that use it as a bare gate fail closed in exactly the same way for any model the map does not carry:

anthropic/chat            azure/chat/gpt_5          bedrock/chat/converse
bedrock_mantle/chat       cerebras/chat             deepinfra/chat
fireworks_ai/chat         gemini/chat               github_copilot/chat
groq/chat                 minimax/chat              openai/chat/gpt_5
openai_like/dynamic_config  openrouter/chat         perplexity/chat
tencent/chat              vertex_ai/gemini          xai/chat
zai/chat

Reproduce the list rather than trusting the count, since it moves with every release:

import ast, os
for root, _, files in os.walk('litellm/llms'):
    for f in files:
        if not f.endswith('.py'):
            continue
        p = os.path.join(root, f)
        src = open(p).read()
        try:
            tree = ast.parse(src)
        except SyntaxError:
            continue
        for node in ast.walk(tree):
            if (isinstance(node, ast.FunctionDef)
                    and node.name == 'get_supported_openai_params'
                    and 'supports_reasoning' in (ast.get_source_segment(src, node) or '')):
                print(f'{p}:{node.lineno}')

The shape matters more than the count, and not every caller fails the same way. AzureOpenAIO1Config calls supports_reasoning too but fails open: a deployment name the map does not know is assumed reasoning-capable and gets reasoning_effort advertised unconditionally. Same call, opposite failure mode. So the question to ask of any provider is not "does it consult the map" but "what does it do when the model is absent", and this warning is useful precisely because it answers that question at runtime instead of by reading source.

The change

One warning naming the dropped params, model and provider:

litellm.drop_params: dropping unsupported params ['reasoning_effort'] for
model=qwen/qwen3-max, provider=openrouter. They will NOT reach the provider, so
whatever behaviour they were meant to control is unchanged. To send them anyway,
pass allowed_openai_params=['reasoning_effort'].

Design points:

  • Warning, not debug — the user asked for something and did not get it. At debug it would be invisible in exactly the situation it exists for.
  • Deduped by (provider, model, sorted dropped params), because the same params are dropped on every request for a given route and a per-call warning would be pure noise.
  • Bounded dedupe set (1000 entries) so a long-lived proxy serving many models cannot grow it without limit. Past the cap it stops recording rather than stops warning — repeating a warning is the safe direction to fail.
  • No warning when drop_params is off, since the caller already gets a loud UnsupportedParamsError.

No behaviour change beyond the log line: the params dropped are exactly the params dropped before.

Scope

Applied to the chat-completions get_optional_params path only — the one that carries essentially all traffic. The transcription / image-gen / embeddings paths have their own near-identical drop sites and could take the same helper; happy to extend if preferred.

Testing

5 tests in tests/test_litellm/test_utils.py::TestDropParamsVisibility: warns on drop, warns once across repeat calls, warns separately per model, stays silent when all params are supported, stays silent when drop_params is off.

tests/test_litellm/test_utils.py goes 179 → 184 passed with an identical set of 21 pre-existing failures (missing optional redis/azure/gcp deps in my env), verified by diffing the failure list with and without the change.

@jwbron
jwbron force-pushed the drop-params-visibility-fork branch from bf2ea98 to 2ee533e Compare July 25, 2026 20:48
Comment thread litellm/utils.py
"they were meant to control is unchanged. To send them anyway, pass "
"allowed_openai_params=%s.",
list(dropped),
model,
Comment thread litellm/utils.py
"allowed_openai_params=%s.",
list(dropped),
model,
custom_llm_provider,
`drop_params` exists so an unsupported parameter does not fail the whole
request, and that tradeoff is right. But dropping a parameter changes
generation behaviour, and today it happens with no signal at all: the branch
that pops the params is a bare loop with no logging. A `reasoning_effort`,
`temperature` or penalty set in a proxy config simply never reaches the
provider, and nothing in the logs or the response says so. The config and the
wire disagree, silently and indefinitely.

This is easiest to hit on a provider whose supported-param set is derived from
the model-cost map. `OpenrouterConfig.get_supported_openai_params` advertises
`reasoning_effort`/`thinking` only when `litellm.supports_reasoning(model)` is
true, which reads that map — so a model ABSENT from the map answers False and
the gate fails closed:

    qwen/qwen3-max               in_map=False  supports_reasoning=False
    moonshotai/kimi-k2-thinking  in_map=False  supports_reasoning=False
    deepseek/deepseek-r1         in_map=True   supports_reasoning=True

OpenRouter adds models faster than the map tracks them, so an unflagged slug
is a routine state rather than an exotic one, and `reasoning_effort` set on
one is a silent no-op. Diagnosing that currently means reading
`get_optional_params` and the model map by hand.

Warns rather than debugs, because the user asked for something and did not get
it. Deduped by (provider, model, dropped-param set) so a route that drops the
same params on every request logs once instead of flooding; the dedupe set is
bounded so a long-lived proxy serving many models cannot grow it without
limit, and past the cap it stops recording rather than stops warning.

No behaviour change beyond the log line: the params dropped are exactly the
params dropped before.
The message only named the per-request kwarg form, but the people who hit
this are overwhelmingly proxy operators reading a config.yaml, for whom
that phrasing reads as not applicable. allowed_openai_params is settable
in a model_list entry's litellm_params (LiteLLM_Params is
ConfigDict(extra="allow")) and does reach get_optional_params from
there; verified end to end through a Router with the transport mocked:
reasoning_effort is absent from the wire by default and present once
allowed_openai_params is set in litellm_params.

Covered by a test so the message cannot silently lose either remedy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants