feat(model_prices): let a map entry declare its exact reasoning_effort levels by tin-berri · Pull Request #38481 · BerriAI/litellm

tin-berri · 2026-08-27T07:51:27Z

TLDR

Problem this solves:

Kimi K3 takes only low, high and max
The model map had no way to say that
So the dashboard cannot offer max for kimi-k3

How it solves it:

New reasoning_effort_levels array key on a map entry
Present means it wins whole, over the per-level flags
Each kimi-k3 entry declares what its own deployment takes

User Flow

Before: an admin building an auto-router cannot put Kimi K3 on the max thinking its own docs call the default

They open https://litellm-domain/ui/models-and-endpoints, go to the Auto-Routers tab and click Add Auto Router
Under Detailed Configuration they put kimi-k3 in the Complex tier
The Reasoning effort dropdown beside it offers Default, none, minimal, low, medium, high, xhigh
max is absent, so they cannot select it, and four levels Kimi does not document are offered instead
They pick high and lose the depth Kimi K3 runs at by default

After: the same dropdown offers exactly what Kimi K3 accepts

They open the same page and put kimi-k3 in the Complex tier
The Reasoning effort dropdown beside it now offers Default, low, high, max
They select max and save the router
Requests the router sends to that tier carry reasoning_effort: max

Relevant issues

The auto-router preset work that needs max on kimi-k3 is stacked on this branch

Linear ticket

Resolves LIT-6325

Pre-Submission checklist

I have added meaningful tests
The handful of test files covering my change pass locally
My PR passes all required CI/CD checks
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Two identical rigs, one per leg: before at 493bca667b (the merge base) and after at e377a0472e (the PR tip). Each leg is its own worktree running a live proxy on a random port with 2 uvicorn workers, a real Postgres database and LITELLM_LOCAL_MODEL_COST_MAP=True, so the proxy reads the map at that commit. The model list registers real Moonshot and Fireworks Kimi K3 deployments (both providers sell it, and the real calls below hit them and cost real money), plus copies of the same litellm model strings pointed at a local echo upstream that replies with the reasoning_effort it received, which makes the wire value the gateway sends observable. gpt56-stub is an openai/gpt-5.6 deployment on the echo upstream and kimi-mixed holds one Moonshot kimi-k3 plus one openai/gpt-5.6. One pre-existing gate to know about, identical at both commits: of the five providers with a declared kimi-k3 entry, only fireworks_ai lets an explicit reasoning_effort through on chat completions (perplexity appends it when supports_reasoning resolves), while moonshot, together_ai and azure_ai reject it with UnsupportedParamsError unless the caller passes allowed_openai_params: ["reasoning_effort"] or sets drop_params. That is why the stub cases below ride the Fireworks model string and the real Moonshot call carries the hatch; LIT-6330 tracks it

model_list:
  - model_name: kimi-k3
    litellm_params: {model: moonshot/kimi-k3, api_key: os.environ/MOONSHOT_API_KEY}
  - model_name: kimi-k3-fireworks
    litellm_params: {model: fireworks_ai/accounts/fireworks/models/kimi-k3, api_key: os.environ/FIREWORKS_AI_API_KEY}
  - model_name: kimi-k3-stub
    litellm_params: {model: moonshot/kimi-k3, api_key: stub-key, api_base: http://127.0.0.1:29545/v1}
  - model_name: kimi-k3-fw-stub
    litellm_params: {model: fireworks_ai/accounts/fireworks/models/kimi-k3, api_key: stub-key, api_base: http://127.0.0.1:29545/v1}
  - model_name: gpt56-stub
    litellm_params: {model: openai/gpt-5.6, api_key: stub-key, api_base: http://127.0.0.1:29545/v1}
  - model_name: kimi-mixed
    litellm_params: {model: moonshot/kimi-k3, api_key: os.environ/MOONSHOT_API_KEY}
  - model_name: kimi-mixed
    litellm_params: {model: openai/gpt-5.6, api_key: os.environ/OPENAI_API_KEY}

Before (`493bca6`, the merge base)

Case 1: what the proxy advertises

curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:51296/model_group/info

Output, one line per group:

kimi-k3              supported_reasoning_efforts=None
kimi-k3-fireworks    supported_reasoning_efforts=None
kimi-k3-stub         supported_reasoning_efforts=None
kimi-k3-fw-stub      supported_reasoning_efforts=None
gpt56-stub           supported_reasoning_efforts=['none', 'low', 'medium', 'high', 'xhigh']
kimi-mixed           supported_reasoning_efforts=['none', 'low', 'medium', 'high', 'xhigh']

Every kimi-k3 group answers None, which the dashboard reads as "this proxy knows nothing" and falls back to a capability-blind list that leaves out max. kimi-mixed takes the gpt-5.6 set whole because the kimi deployment carries no opinion

Case 2: /v1/chat/completions request path

curl -s -X POST http://127.0.0.1:51296/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"kimi-k3-fw-stub","messages":[{"role":"user","content":"hi"}],"reasoning_effort":"max"}'
200, and the echo upstream received the level unchanged: forwarded reasoning_effort=max

Case 3: /v1/responses request path

curl -s -X POST http://127.0.0.1:51296/v1/responses -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"kimi-k3-fw-stub","input":"hi","reasoning":{"effort":"max"}}'
200, same story: forwarded reasoning_effort=max

Case 4: /v1/messages request path

curl -s -X POST http://127.0.0.1:51296/v1/messages -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"kimi-k3-fw-stub","max_tokens":128,"messages":[{"role":"user","content":"hi"}],"thinking":{"type":"adaptive"},"output_config":{"effort":"max"}}'
200, but the level was silently lowered: forwarded reasoning_effort=high
This surface maps output_config.effort into reasoning_effort only when the body carries "thinking": {"type": "adaptive"}; its degradation chain reads the per-level boolean flags, finds no max flag on the kimi entry and lowers a level the model actually takes

Case 5: real provider calls

curl -s -X POST http://127.0.0.1:51296/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"kimi-k3-fireworks","messages":[{"role":"user","content":"Reply with exactly: ok"}],"reasoning_effort":"max","max_tokens":600}'
200 from the real Fireworks API, content ok, usage 93/114 tokens

Case 6: Admin UI reasoning-effort dropdown

Boot the Admin UI dev server from this commit's ui/litellm-dashboard pointed at this leg's proxy. The dashboard build packaged in _experimental/out predates the capability-aware dropdown, and this PR touches no ui/ files, so the identical dashboard source ran against both legs
Open http://localhost:3518/models-and-endpoints, go to the Auto-Routers tab, click Add Auto Router, expand Detailed Configuration
Put kimi-k3 in the Complex tier and open the Reasoning effort dropdown beside it: it offers Default, none, minimal, low, medium, high, xhigh. No max
Put kimi-mixed in the Reasoning tier: its dropdown offers the gpt-5.6 set, Default, none, low, medium, high, xhigh

After (`e377a04`)

Case 1: what the proxy advertises

Same curl against the after rig: curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:47594/model_group/info

Output:

kimi-k3              supported_reasoning_efforts=['low', 'high', 'max']
kimi-k3-fireworks    supported_reasoning_efforts=['low', 'high', 'max']
kimi-k3-stub         supported_reasoning_efforts=['low', 'high', 'max']
kimi-k3-fw-stub      supported_reasoning_efforts=['low', 'high', 'max']
gpt56-stub           supported_reasoning_efforts=['none', 'low', 'medium', 'high', 'xhigh']
kimi-mixed           supported_reasoning_efforts=['low', 'high']

Each kimi-k3 group now names the three levels Kimi documents and gpt56-stub is byte-identical to before. kimi-mixed narrows to the intersection because a kimi deployment now carries an opinion where it used to carry none: that is the one behavior change a mixed group sees, and it is the intended one

Case 2: /v1/chat/completions request path

Same curl as before
200, unchanged: forwarded reasoning_effort=max

Case 3: /v1/responses request path

Same curl as before
200, unchanged: forwarded reasoning_effort=max

Case 4: /v1/messages request path

Same curl as before
200, and the level now survives: forwarded reasoning_effort=max
The degradation chain consults the declaration first, sees max is a level this entry takes and leaves it alone. Entries without a declaration keep the exact answer the chain gave them before, pinned by a negative-class test

Case 5: real provider calls

Same real Fireworks curl as before: 200, content ok, usage 93/14 tokens
curl -s -X POST http://127.0.0.1:47594/v1/messages -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -d '{"model":"kimi-k3","max_tokens":600,"messages":[{"role":"user","content":"Reply with exactly: ok"}],"thinking":{"type":"adaptive"},"output_config":{"effort":"max"},"allowed_openai_params":["reasoning_effort"]}'
200 from the real Moonshot API, text ok, with the max effort carried through the hatch

Case 6: Admin UI reasoning-effort dropdown

Same dev-server flow against the after proxy, on http://localhost:3517/models-and-endpoints
Same page, same dialog, kimi-k3 in the Complex tier
The Reasoning effort dropdown offers exactly Default, low, high, max
kimi-mixed in the Reasoning tier narrows to Default, low, high

Type

🆕 New Feature

Caveats (if any)

Low

Three of the five declared providers reject an explicit reasoning_effort today
- moonshot, together_ai and azure_ai 400 on chat completions
- Pre-existing at the merge base; only fireworks_ai forwards it
- perplexity appends it when supports_reasoning resolves
- Hatches: allowed_openai_params or drop_params; LIT-6330 tracks real support
- The tip commit's message overstates this; only Fireworks forwards today
Two ways to express one fact now coexist, arbitrated by precedence
- A declaration wins whole; the ~915 flag entries are untouched
- Pinned by a test; flags stay the idiom where they suffice
The seven undocumented entries are an inference, deliberately flagged
- Moonshot, Together and Perplexity document their levels; Fireworks and Azure Foundry do not for K3
- Same K3 weights, so they get the model's own levels
- Azure Foundry still rejects an explicit level (LIT-6330), advertisement-only until then
- Happy to narrow those to flag-thin if a reviewer disagrees
PR feat(model_prices): add databricks kimi k3, databricks glm 5.2 and zai glm 5.3 #38415 adds a databricks kimi-k3 entry and would land without this key

Final Attestation

The tests check the right things, including the edge cases

Note

Medium Risk
Changes shared reasoning-effort resolution and request normalization for any entry with the new key; mixed model groups now intersect to narrower effort sets when Kimi deployments declare levels.

Overview
Adds reasoning_effort_levels on model catalog entries so deployments can advertise an exact set of reasoning_effort values (e.g. Kimi K3’s low / high / max) instead of inferring from per-level supports_* flags, which cannot express “no medium.”

When present, the list wins over the boolean flags in resolve_supported_reasoning_efforts (router / model_group/info) and in normalize_reasoning_effort_value on the Anthropic messages path, so max is no longer silently degraded to high for declared Kimi entries. Schema, ModelInfo, and model-info hydration carry the new field; Kimi K3 variants across Moonshot, Fireworks, Together, Azure AI, and Perplexity are populated in the price map.

^{Reviewed by Cursor Bugbot for commit e377a04. Bugbot is set up for automated code reviews on this repo. Configure here.}

tin-berri · 2026-08-27T07:52:13Z

@greptileai can you reply here with a score and why that score?

tin-berri · 2026-08-27T07:52:14Z

bugbot run

greptile-apps · 2026-08-27T07:53:12Z

Score: 4/5 (high confidence).

The change is narrowly scoped and internally consistent:

The schema, ProviderSpecificModelInfo, model-info hydration, and resolver all support the new key.
litellm/router_utils/reasoning_effort_capability.py correctly gives an explicit list precedence over per-level flags, preserves canonical ordering, distinguishes None from an empty declaration, and safely falls back for malformed declarations.
All ten Kimi K3 map entries are updated in both model-price files.
Tests cover precedence, empty/malformed input, provider-prefixed hydration, the Kimi entries, and mixed-group intersection.
The available Buildkite check passed, and the request forwarding path is unchanged.

I’m not giving 5/5 because the PR documents two remaining semantic gaps: /v1/messages still degrades Kimi K3 max to high, and the Perplexity entry advertises only low/high/max even though that provider documents a wider alias surface. These are documented caveats rather than regressions in this PR, so I consider the change mergeable with high confidence.

greptile-apps · 2026-08-27T07:54:06Z

Greptile Summary

Adds an exact reasoning_effort_levels model-metadata field and gives it precedence over inferred per-level capability flags.

Propagates declared effort levels through model-info hydration, group capability resolution, and Anthropic effort normalization.
Adds Kimi K3 provider metadata, generated schema support, and regression coverage for declarations, precedence, degradation, and mixed-group intersections.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/router_utils/reasoning_effort_capability.py	Resolves explicit effort declarations before legacy flags while preserving fallback behavior for undeclared or malformed values.
litellm/llms/anthropic/experimental_pass_through/utils.py	Uses declared levels when normalizing max, xhigh, and minimal effort values through degradation chains.
litellm/utils.py	Hydrates the new reasoning-effort declaration from model catalog entries.
model_prices_and_context_window.json	Declares provider-specific Kimi K3 reasoning-effort sets.
tests/test_litellm/router_utils/test_reasoning_effort_capability.py	Covers declaration precedence, malformed values, model-info hydration, provider metadata, and mixed-group intersections.
tests/test_litellm/llms/anthropic/experimental_pass_through/test_reasoning_effort_fields.py	Covers declaration-aware effort preservation and degradation without changing undeclared models.

_{Reviews (6): Last reviewed commit: "feat(model_prices): let a map entry decl..." | Re-trigger Greptile}

codecov · 2026-08-27T07:54:58Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit aba9a37. Configure here.}

codspeed-hq · 2026-08-27T08:35:11Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_kimi_k3_reasoning_efforts (e377a04) with litellm_internal_staging (ca9007b)¹}

No successful run was found on litellm_internal_staging (4522549) during the generation of this report, so ca9007b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

tin-berri · 2026-08-27T08:42:29Z

@greptileai both 4/5 reasons are addressed: /v1/messages now honors the declaration, and Perplexity declares its own six levels. Score?

tin-berri · 2026-08-27T08:42:31Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 49655da. Configure here.}

tin-berri · 2026-08-27T08:53:38Z

codecov/patch is red at 55%, but the resolver measures 100% locally and CI logs show the new tests ran. Flagging rather than absorbing.

tin-berri · 2026-08-27T08:56:37Z

Correction: codecov/patch went green once the last shard upload merged. All checks pass; disregard the note above.

tin-berri · 2026-08-27T16:00:47Z

Pushed: trimmed overlong lines and condensed the test prose per the repo comment rule. All within 120 now.

tin-berri · 2026-08-27T16:00:49Z

bugbot run

tin-berri · 2026-08-27T16:00:56Z

@greptileai line lengths and test comments addressed. Score?

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 481f256. Configure here.}

tin-berri · 2026-08-27T19:18:28Z

Fixed a precedence leak: a declaration now wins whole on /v1/messages too, so a flag cannot re-add a level the list omits.

tin-berri · 2026-08-27T19:18:30Z

bugbot run

tin-berri · 2026-08-27T19:18:31Z

@greptileai added declaration-beats-flags precedence on the messages gate, with tests. Score?

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5e02f0a. Configure here.}

tin-berri · 2026-08-27T19:22:27Z

bugbot run

tin-berri · 2026-08-27T19:22:29Z

@greptileai can you reply here with a score and why that score?

greptile-apps · 2026-08-27T19:23:33Z

Score: 5/5 (high confidence).

The previously identified gaps are addressed: normalize_reasoning_effort_value now consults the declaration for /v1/messages, and the Perplexity entry declares its documented six-level surface. The implementation is consistent across the resolver, model-info hydration, schema, catalog entries, and passthrough normalization; declaration precedence, canonical ordering, empty/malformed declarations, provider-prefixed hydration, mixed-group intersection, and unchanged flag-only behavior are covered by focused tests. The request forwarding path remains unchanged, and the latest review found no blocking correctness issues.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0dc08e6. Configure here.}

…t levels Kimi K3 accepts exactly low, high and max, defaults to max, and always thinks. The map could not say that: medium and high have no supports_*_reasoning_effort flag because every other reasoning model takes them, so the ten kimi-k3 entries carried supports_reasoning alone and resolved to unknown. The dashboard then fell back to a capability-blind level list that deliberately omits max, which is why a kimi-k3 tier cannot be set to max thinking today. Add reasoning_effort_levels, an array key in the shape the map already uses for supported_endpoints and supported_modalities. Where present it is read first and wins whole; every other entry keeps answering through the per-level flags, unchanged. It is deliberately a different name from the computed ModelGroupInfo.supported_reasoning_efforts, which stays derived from a group's deployments and is never seeded from one deployment's model_info. The levels are per entry rather than per model, because the deployments differ: Moonshot, Together, Fireworks and Azure Foundry all forward the level unchanged and get the model's own low/high/max, while Perplexity documents a six-value enum it maps down internally and gets that. The /v1/messages degradation chain consults the same declaration, so the level the map advertises is the level that path forwards.

mateo-berri · 2026-08-27T20:01:31Z

@greptileai

mateo-berri · 2026-08-27T20:51:43Z

@greptileai

mateo-berri · 2026-08-27T20:51:45Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e377a04. Configure here.}

mateo-berri

LGTM. Thanks!

mateo-berri · 2026-08-27T22:38:03Z

A mechanical, deterministic script can be made to backfill the nested supports keys for other models. We can keep both for now, defaulting to the nested, then after enough time, remove the old non-nested keys

tin-berri requested a review from mateo-berri as a code owner August 27, 2026 07:51

tin-berri mentioned this pull request Aug 27, 2026

feat(ui): run the Lite preset's medium and complex tiers at their documented efforts #38482

Merged

6 tasks

greptile-apps Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread tests/test_litellm/router_utils/test_reasoning_effort_capability.py

cursor Bot reviewed Aug 27, 2026

View reviewed changes

tin-berri force-pushed the litellm_kimi_k3_reasoning_efforts branch from aba9a37 to 49655da Compare August 27, 2026 08:40

tin-berri changed the title ~~feat(model_prices): declare exact reasoning_effort levels, starting with kimi-k3~~ feat(model_prices): let a map entry declare its exact reasoning_effort levels Aug 27, 2026

cursor Bot reviewed Aug 27, 2026

View reviewed changes

tin-berri mentioned this pull request Aug 27, 2026

fix(anthropic): resolve /v1/messages effort tiers through the capability owner #38492

Merged

6 tasks

tin-berri force-pushed the litellm_kimi_k3_reasoning_efforts branch from 49655da to 481f256 Compare August 27, 2026 16:00

cursor Bot reviewed Aug 27, 2026

View reviewed changes

tin-berri force-pushed the litellm_kimi_k3_reasoning_efforts branch from 481f256 to 5e02f0a Compare August 27, 2026 19:18

tin-berri force-pushed the litellm_kimi_k3_reasoning_efforts branch from 5e02f0a to 0dc08e6 Compare August 27, 2026 19:22

cursor Bot reviewed Aug 27, 2026

View reviewed changes

tin-berri force-pushed the litellm_kimi_k3_reasoning_efforts branch from 0dc08e6 to e377a04 Compare August 27, 2026 19:33

tin-berri enabled auto-merge (squash) August 27, 2026 20:21

cursor Bot reviewed Aug 27, 2026

View reviewed changes

mateo-berri approved these changes Aug 27, 2026

View reviewed changes

tin-berri merged commit 30ff372 into litellm_internal_staging Aug 27, 2026
82 checks passed

tin-berri deleted the litellm_kimi_k3_reasoning_efforts branch August 27, 2026 22:38

Uh oh!

Uh oh!

Conversation

tin-berri commented Aug 27, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Before (493bca6, the merge base)

Case 1: what the proxy advertises

Case 2: /v1/chat/completions request path

Case 3: /v1/responses request path

Case 4: /v1/messages request path

Case 5: real provider calls

Case 6: Admin UI reasoning-effort dropdown

After (e377a04)

Case 1: what the proxy advertises

Case 2: /v1/chat/completions request path

Case 3: /v1/responses request path

Case 4: /v1/messages request path

Case 5: real provider calls

Case 6: Admin UI reasoning-effort dropdown

Type

Caveats (if any)

Low

Final Attestation

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

greptile-apps Bot commented Aug 27, 2026

Uh oh!

greptile-apps Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

Uh oh!

tin-berri commented Aug 27, 2026

tin-berri commented Aug 27, 2026 •

edited by cursor Bot

Loading

Before (`493bca6`, the merge base)

After (`e377a04`)

greptile-apps Bot commented Aug 27, 2026 •

edited

Loading

codecov Bot commented Aug 27, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 27, 2026 •

edited

Loading