Skip to content

fix: correct wandb model prices (off by 100,000x) #23517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gambletan wants to merge 1 commit into from
Closed

fix: correct wandb model prices (off by 100,000x) #23517

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 31 additions & 44 deletions litellm/model_prices_and_context_window_backup.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14527,17 +14527,14 @@
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2-preview": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"input_cost_per_token": 1.5e-07,
"litellm_provider": "vertex_ai",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"source": "https://ai.google.dev/gemini-api/docs/embeddings#multimodal",
"supports_multimodal": true,
"uses_embed_content": true
},
Comment on lines 14529 to 14540

@greptile-apps greptile-apps Bot Mar 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undocumented vertex_ai/gemini-embedding-2-preview changes

This PR fixes wandb prices, but it also silently modifies vertex_ai/gemini-embedding-2-preview in two ways that are not mentioned in the PR description or title:

  1. Removes multimodal pricing fields — input_cost_per_audio_per_second: 0.00016, input_cost_per_image: 0.00012, and input_cost_per_video_per_second: 0.00079 are dropped. Any code path that uses get_model_info to calculate multimodal embedding costs for this model will silently return None/0 instead of the correct cost.

  2. Changes input_cost_per_token from 2e-07 to 1.5e-07 — a ~25% price reduction with no cited justification in the PR (the source URL also changes from the Vertex AI pricing page to the Gemini API docs).

These look like they may be side effects of resolving a duplicate-key situation in the JSON (there were previously two "vertex_ai/gemini-embedding-2-preview" entries). If the intent is to consolidate to the second entry's values, please document this in the PR description and confirm whether removing the multimodal pricing fields is intentional.

Expand All @@ -14554,18 +14551,6 @@
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2-preview": {
"input_cost_per_token": 1.5e-07,
"litellm_provider": "vertex_ai",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://ai.google.dev/gemini-api/docs/embeddings#multimodal",
"supports_multimodal": true,
"uses_embed_content": true
},
"gemini/gemini-embedding-001": {
"input_cost_per_token": 1.5e-07,
"litellm_provider": "gemini",
Expand Down Expand Up @@ -30938,7 +30923,9 @@
"mode": "chat",
"output_cost_per_token": 3.2e-06,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing#glm-models",
"supported_regions": ["global"],
"supported_regions": [
"global"
],
"supports_function_calling": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
Expand Down Expand Up @@ -31438,53 +31425,53 @@
"max_tokens": 131072,
"max_input_tokens": 131072,
"max_output_tokens": 131072,
"input_cost_per_token": 0.015,
"output_cost_per_token": 0.06,
"input_cost_per_token": 1.5e-07,
"output_cost_per_token": 6e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/openai/gpt-oss-20b": {
"max_tokens": 131072,
"max_input_tokens": 131072,
"max_output_tokens": 131072,
"input_cost_per_token": 0.005,
"output_cost_per_token": 0.02,
"input_cost_per_token": 5e-08,
"output_cost_per_token": 2e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/zai-org/GLM-4.5": {
"max_tokens": 131072,
"max_input_tokens": 131072,
"max_output_tokens": 131072,
"input_cost_per_token": 0.055,
"output_cost_per_token": 0.2,
"input_cost_per_token": 5.5e-07,
"output_cost_per_token": 2e-06,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/Qwen/Qwen3-235B-A22B-Instruct-2507": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 0.01,
"output_cost_per_token": 0.01,
"input_cost_per_token": 1e-07,
"output_cost_per_token": 1e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/Qwen/Qwen3-Coder-480B-A35B-Instruct": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 0.1,
"output_cost_per_token": 0.15,
"input_cost_per_token": 1e-06,
"output_cost_per_token": 1.5e-06,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/Qwen/Qwen3-235B-A22B-Thinking-2507": {
"max_tokens": 262144,
"max_input_tokens": 262144,
"max_output_tokens": 262144,
"input_cost_per_token": 0.01,
"output_cost_per_token": 0.01,
"input_cost_per_token": 1e-07,
"output_cost_per_token": 1e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
Expand All @@ -31501,62 +31488,62 @@
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.022,
"output_cost_per_token": 0.022,
"input_cost_per_token": 2.1999999999999998e-07,
"output_cost_per_token": 2.1999999999999998e-07,
Comment on lines +31491 to +31492

@greptile-apps greptile-apps Bot Mar 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Floating-point precision artifacts

Several corrected values contain floating-point representation artifacts rather than clean scientific notation. These arise from computing 0.022 / 100000, 0.275 / 100000, 0.017 / 100000, and 0.035 / 100000 in floating-point arithmetic. While technically equivalent, the noisy representations reduce readability and make future diffs harder to review.

The same issue also appears at:

  • litellm/model_prices_and_context_window_backup.json:31491 – 2.1999999999999998e-07 (Llama-3.1-8B input)
  • litellm/model_prices_and_context_window_backup.json:31492 – 2.1999999999999998e-07 (Llama-3.1-8B output)
  • litellm/model_prices_and_context_window_backup.json:31519 – 2.7500000000000004e-06 (DeepSeek-V3-0324 output)
  • litellm/model_prices_and_context_window_backup.json:31536 – 1.7000000000000001e-07 (Llama-4-Scout input)
  • litellm/model_prices_and_context_window_backup.json:31546 – 3.5000000000000004e-07 (Phi-4-mini output)

These should be rounded to the same precision used in the PR description's table:

Suggested change
"input_cost_per_token": 2.1999999999999998e-07,
"output_cost_per_token": 2.1999999999999998e-07,
"input_cost_per_token": 2.2e-07,
"output_cost_per_token": 2.2e-07,

"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/deepseek-ai/DeepSeek-V3.1": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.055,
"output_cost_per_token": 0.165,
"input_cost_per_token": 5.5e-07,
"output_cost_per_token": 1.65e-06,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/deepseek-ai/DeepSeek-R1-0528": {
"max_tokens": 161000,
"max_input_tokens": 161000,
"max_output_tokens": 161000,
"input_cost_per_token": 0.135,
"output_cost_per_token": 0.54,
"input_cost_per_token": 1.35e-06,
"output_cost_per_token": 5.4e-06,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/deepseek-ai/DeepSeek-V3-0324": {
"max_tokens": 161000,
"max_input_tokens": 161000,
"max_output_tokens": 161000,
"input_cost_per_token": 0.114,
"output_cost_per_token": 0.275,
"input_cost_per_token": 1.14e-06,
"output_cost_per_token": 2.7500000000000004e-06,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/meta-llama/Llama-3.3-70B-Instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.071,
"output_cost_per_token": 0.071,
"input_cost_per_token": 7.1e-07,
"output_cost_per_token": 7.1e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/meta-llama/Llama-4-Scout-17B-16E-Instruct": {
"max_tokens": 64000,
"max_input_tokens": 64000,
"max_output_tokens": 64000,
"input_cost_per_token": 0.017,
"output_cost_per_token": 0.066,
"input_cost_per_token": 1.7000000000000001e-07,
"output_cost_per_token": 6.6e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
"wandb/microsoft/Phi-4-mini-instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.008,
"output_cost_per_token": 0.035,
"input_cost_per_token": 8e-08,
"output_cost_per_token": 3.5000000000000004e-07,
"litellm_provider": "wandb",
"mode": "chat"
},
Expand Down