Skip to content

feat(nebius): add model pricing metadata - #33185

Open
pyguy wants to merge 2 commits into
BerriAI:litellm_oss_daily_2026_07_13from
pyguy:feature/add-nebius-model-pricing
Open

feat(nebius): add model pricing metadata#33185
pyguy wants to merge 2 commits into
BerriAI:litellm_oss_daily_2026_07_13from
pyguy:feature/add-nebius-model-pricing

Conversation

@pyguy

@pyguy pyguy commented Jul 14, 2026

Copy link
Copy Markdown

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added testing in the tests/test_litellm/ directory
  • My PR passes all unit tests
  • My PR scope is isolated to model pricing metadata
  • I have requested and received a Greptile review

Screenshots / Proof of Fix

The cost registry includes the six Nebius model IDs with their configured context limits, token pricing, capability flags, and Nano cache-read pricing

Repository-wide lint and unit checks are not marked complete because this fork lacks its internal staging branch and test collection encounters an existing duplicate test-module name

Type

New Feature

Changes

Adds Nebius pricing metadata for Qwen3.5-397B-A17B, GLM-5.2, Nemotron 3 Super, Nemotron 3 Ultra, Kimi K2.7 Code, and Nemotron 3 Nano

Adds a regression test for the new pricing and metadata records

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Hamidreza Josheghani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds pricing and capability metadata for six new Nebius models — Qwen3.5-397B-A17B, GLM-5.2, Nemotron-3-Super, Nemotron-3-Ultra, Kimi-K2.7-Code, and Nemotron-3-Nano — to model_prices_and_context_window.json, along with a regression test that validates all six entries.

  • All six new entries set supports_reasoning: true, but Nemotron-3-Super, Nemotron-3-Ultra, and Kimi-K2.7-Code are standard instruct/code models that do not produce chain-of-thought reasoning tokens; this flag should only be set for models that emit <think>-style reasoning output.
  • The three new NVIDIA Nemotron keys use inconsistent casing (nemotron-3-super-120b-a12b all-lowercase, Nemotron-3-Ultra-550b-a55b mixed, NVIDIA-Nemotron-3-Nano-30B-A3B with all-caps prefix), which could cause silent model-info lookup failures if callers or the Nebius API use a different casing.

Confidence Score: 3/5

Safe to merge after verifying the supports_reasoning flag and correcting model key casing for the three NVIDIA Nemotron entries.

The supports_reasoning flag on Nemotron-3-Super, Nemotron-3-Ultra, and Kimi-K2.7-Code activates reasoning-specific response processing for models that produce standard chat output, which can silently corrupt cost accounting or response handling. The casing inconsistency across the Nemotron keys adds a second independent way for lookups to fail at runtime.

model_prices_and_context_window.json — the three new NVIDIA Nemotron entries need casing verified against the live Nebius API, and the supports_reasoning flag should only remain on models confirmed to emit reasoning tokens.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds pricing metadata for 6 new Nebius models; supports_reasoning: true is applied to all entries including standard instruct/code models that likely do not emit reasoning tokens, and the three new NVIDIA Nemotron keys have inconsistent casing that could break model lookups.
tests/test_litellm/llms/nebius/test_nebius_chat_transformation.py Adds a standalone regression test that loads the JSON directly and asserts all six new models' pricing and capability flags; no network calls, path traversal is correct, assertions are appropriately tight.

Comments Outside Diff (1)

  1. model_prices_and_context_window.json, line 28333-28391 (link)

    P1 supports_reasoning applied to non-reasoning instruct models

    nebius/nvidia/nemotron-3-super-120b-a12b, nebius/nvidia/Nemotron-3-Ultra-550b-a55b, and nebius/moonshotai/Kimi-K2.7-Code are listed as standard instruction-following / code-generation models on Nebius. They do not emit chain-of-thought <think> tokens the way DeepSeek-R1 or QwQ-32B do. Setting supports_reasoning: true on them causes litellm to activate reasoning-specific response handling for models that will never produce reasoning tokens, which can result in incorrect token accounting or response parsing. Compare with the existing nebius/nvidia/Llama-3.3-Nemotron-Super-49B-v1 entry (line 28134), which omits this flag. Please verify against the Nebius API docs that these three models actually produce structured reasoning output before enabling the flag.

Reviews (1): Last reviewed commit: "feat(nebius): add model pricing metadata" | Re-trigger Greptile

Comment on lines +28333 to 28388
"nebius/nvidia/nemotron-3-super-120b-a12b": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 3e-07,
"output_cost_per_token": 9e-07,
"litellm_provider": "nebius",
"mode": "chat",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_response_schema": true,
"source": "https://nebius.com/prices-ai-studio"
},
"nebius/nvidia/Nemotron-3-Ultra-550b-a55b": {
"max_tokens": 1000000,
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"input_cost_per_token": 1e-06,
"output_cost_per_token": 3e-06,
"litellm_provider": "nebius",
"mode": "chat",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_response_schema": true,
"source": "https://nebius.com/prices-ai-studio"
},
"nebius/moonshotai/Kimi-K2.7-Code": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 9.5e-07,
"output_cost_per_token": 4e-06,
"litellm_provider": "nebius",
"mode": "chat",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_response_schema": true,
"source": "https://nebius.com/prices-ai-studio"
},
"nebius/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 6e-08,
"output_cost_per_token": 2.4e-07,
"cache_read_input_token_cost": 6e-09,
"litellm_provider": "nebius",
"mode": "chat",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_response_schema": true,
"source": "https://nebius.com/prices-ai-studio"
},
"nvidia.nemotron-nano-12b-v2": {
"input_cost_per_token": 2e-07,
"litellm_provider": "bedrock_converse",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Inconsistent casing across the three new NVIDIA Nemotron model keys

The three Nemotron entries use different casing conventions: nemotron-3-super-120b-a12b is all-lowercase, Nemotron-3-Ultra-550b-a55b is title-case with a lowercase b/a suffix, and NVIDIA-Nemotron-3-Nano-30B-A3B uses all-caps "NVIDIA". Because the JSON key must be an exact, case-sensitive match to the model ID that callers pass (and that the Nebius /v1/models endpoint returns), any mismatch will silently fall through to a "model not found" error when cost calculation is attempted. The two existing NVIDIA Nemotron entries (nebius/nvidia/Llama-3.1-Nemotron-Ultra-253B-v1, nebius/nvidia/Llama-3.3-Nemotron-Super-49B-v1) follow consistent title-case. Please verify the exact model IDs against the Nebius API and normalise casing across all three entries.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@pyguy

pyguy commented Jul 14, 2026

Copy link
Copy Markdown
Author

Verified the entries against models.dev provider metadata and the Nebius Token Factory model-catalog documentation

The follow-up corrects Qwen, GLM, Super, and Nano input/output limits and removes reasoning support from Nano

The provider metadata reports reasoning support for Super, while the canonical Kimi and Ultra model records also report reasoning support, so those flags remain

The map retains the exact provider model IDs rather than normalizing their casing

@demianarc

Copy link
Copy Markdown

I audited this existing pricing PR alongside the endpoint/capability repair in #36777 to avoid duplicating its model-catalog scope.

The six July records should be refreshed before merge against the current public Token Factory feed: https://tokenfactory.nebius.com/api/public/models_info. The feed now includes newer Kimi K3, DeepSeek V4 Flash, and Nemotron 3.5 Lightning entries and exposes dated pricing/context/capability metadata. Please generate or diff-check these records rather than maintaining another hand-curated subset.

#36777 intentionally does not touch model_prices_and_context_window.json; it repairs the existing provider endpoint, branding, request tests, and false capability flags. These two PRs are complementary, not competing. Keep direct Token Factory Responses capability false unless a Nebius-specific stateless Responses transformation is implemented and tested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants