feat(nebius): add model pricing metadata - #33185
Conversation
|
Hamidreza Josheghani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Greptile SummaryThis PR adds pricing and capability metadata for six new Nebius models — Qwen3.5-397B-A17B, GLM-5.2, Nemotron-3-Super, Nemotron-3-Ultra, Kimi-K2.7-Code, and Nemotron-3-Nano — to
Confidence Score: 3/5Safe to merge after verifying the supports_reasoning flag and correcting model key casing for the three NVIDIA Nemotron entries. The supports_reasoning flag on Nemotron-3-Super, Nemotron-3-Ultra, and Kimi-K2.7-Code activates reasoning-specific response processing for models that produce standard chat output, which can silently corrupt cost accounting or response handling. The casing inconsistency across the Nemotron keys adds a second independent way for lookups to fail at runtime. model_prices_and_context_window.json — the three new NVIDIA Nemotron entries need casing verified against the live Nebius API, and the supports_reasoning flag should only remain on models confirmed to emit reasoning tokens.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds pricing metadata for 6 new Nebius models; supports_reasoning: true is applied to all entries including standard instruct/code models that likely do not emit reasoning tokens, and the three new NVIDIA Nemotron keys have inconsistent casing that could break model lookups. |
| tests/test_litellm/llms/nebius/test_nebius_chat_transformation.py | Adds a standalone regression test that loads the JSON directly and asserts all six new models' pricing and capability flags; no network calls, path traversal is correct, assertions are appropriately tight. |
Comments Outside Diff (1)
-
model_prices_and_context_window.json, line 28333-28391 (link)supports_reasoningapplied to non-reasoning instruct modelsnebius/nvidia/nemotron-3-super-120b-a12b,nebius/nvidia/Nemotron-3-Ultra-550b-a55b, andnebius/moonshotai/Kimi-K2.7-Codeare listed as standard instruction-following / code-generation models on Nebius. They do not emit chain-of-thought<think>tokens the way DeepSeek-R1 or QwQ-32B do. Settingsupports_reasoning: trueon them causes litellm to activate reasoning-specific response handling for models that will never produce reasoning tokens, which can result in incorrect token accounting or response parsing. Compare with the existingnebius/nvidia/Llama-3.3-Nemotron-Super-49B-v1entry (line 28134), which omits this flag. Please verify against the Nebius API docs that these three models actually produce structured reasoning output before enabling the flag.
Reviews (1): Last reviewed commit: "feat(nebius): add model pricing metadata" | Re-trigger Greptile
| "nebius/nvidia/nemotron-3-super-120b-a12b": { | ||
| "max_tokens": 262144, | ||
| "max_input_tokens": 262144, | ||
| "max_output_tokens": 262144, | ||
| "input_cost_per_token": 3e-07, | ||
| "output_cost_per_token": 9e-07, | ||
| "litellm_provider": "nebius", | ||
| "mode": "chat", | ||
| "supports_function_calling": true, | ||
| "supports_reasoning": true, | ||
| "supports_response_schema": true, | ||
| "source": "https://nebius.com/prices-ai-studio" | ||
| }, | ||
| "nebius/nvidia/Nemotron-3-Ultra-550b-a55b": { | ||
| "max_tokens": 1000000, | ||
| "max_input_tokens": 1000000, | ||
| "max_output_tokens": 1000000, | ||
| "input_cost_per_token": 1e-06, | ||
| "output_cost_per_token": 3e-06, | ||
| "litellm_provider": "nebius", | ||
| "mode": "chat", | ||
| "supports_function_calling": true, | ||
| "supports_reasoning": true, | ||
| "supports_response_schema": true, | ||
| "source": "https://nebius.com/prices-ai-studio" | ||
| }, | ||
| "nebius/moonshotai/Kimi-K2.7-Code": { | ||
| "max_tokens": 262144, | ||
| "max_input_tokens": 262144, | ||
| "max_output_tokens": 262144, | ||
| "input_cost_per_token": 9.5e-07, | ||
| "output_cost_per_token": 4e-06, | ||
| "litellm_provider": "nebius", | ||
| "mode": "chat", | ||
| "supports_function_calling": true, | ||
| "supports_reasoning": true, | ||
| "supports_response_schema": true, | ||
| "source": "https://nebius.com/prices-ai-studio" | ||
| }, | ||
| "nebius/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B": { | ||
| "max_tokens": 262144, | ||
| "max_input_tokens": 262144, | ||
| "max_output_tokens": 262144, | ||
| "input_cost_per_token": 6e-08, | ||
| "output_cost_per_token": 2.4e-07, | ||
| "cache_read_input_token_cost": 6e-09, | ||
| "litellm_provider": "nebius", | ||
| "mode": "chat", | ||
| "supports_function_calling": true, | ||
| "supports_reasoning": true, | ||
| "supports_response_schema": true, | ||
| "source": "https://nebius.com/prices-ai-studio" | ||
| }, | ||
| "nvidia.nemotron-nano-12b-v2": { | ||
| "input_cost_per_token": 2e-07, | ||
| "litellm_provider": "bedrock_converse", |
There was a problem hiding this comment.
Inconsistent casing across the three new NVIDIA Nemotron model keys
The three Nemotron entries use different casing conventions: nemotron-3-super-120b-a12b is all-lowercase, Nemotron-3-Ultra-550b-a55b is title-case with a lowercase b/a suffix, and NVIDIA-Nemotron-3-Nano-30B-A3B uses all-caps "NVIDIA". Because the JSON key must be an exact, case-sensitive match to the model ID that callers pass (and that the Nebius /v1/models endpoint returns), any mismatch will silently fall through to a "model not found" error when cost calculation is attempted. The two existing NVIDIA Nemotron entries (nebius/nvidia/Llama-3.1-Nemotron-Ultra-253B-v1, nebius/nvidia/Llama-3.3-Nemotron-Super-49B-v1) follow consistent title-case. Please verify the exact model IDs against the Nebius API and normalise casing across all three entries.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Verified the entries against models.dev provider metadata and the Nebius Token Factory model-catalog documentation The follow-up corrects Qwen, GLM, Super, and Nano input/output limits and removes reasoning support from Nano The provider metadata reports reasoning support for Super, while the canonical Kimi and Ultra model records also report reasoning support, so those flags remain The map retains the exact provider model IDs rather than normalizing their casing |
|
I audited this existing pricing PR alongside the endpoint/capability repair in #36777 to avoid duplicating its model-catalog scope. The six July records should be refreshed before merge against the current public Token Factory feed: https://tokenfactory.nebius.com/api/public/models_info. The feed now includes newer Kimi K3, DeepSeek V4 Flash, and Nemotron 3.5 Lightning entries and exposes dated pricing/context/capability metadata. Please generate or diff-check these records rather than maintaining another hand-curated subset. #36777 intentionally does not touch |
Relevant issues
Linear ticket
Pre-Submission checklist
tests/test_litellm/directoryScreenshots / Proof of Fix
The cost registry includes the six Nebius model IDs with their configured context limits, token pricing, capability flags, and Nano cache-read pricing
Repository-wide lint and unit checks are not marked complete because this fork lacks its internal staging branch and test collection encounters an existing duplicate test-module name
Type
New Feature
Changes
Adds Nebius pricing metadata for Qwen3.5-397B-A17B, GLM-5.2, Nemotron 3 Super, Nemotron 3 Ultra, Kimi K2.7 Code, and Nemotron 3 Nano
Adds a regression test for the new pricing and metadata records