Skip to content

fix(catalog): align openrouter prices with live openrouter.ai rates - #39591

Closed
astraltrekkin wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
astraltrekkin:cursor/berriai-litellm-39501-3141
Closed

fix(catalog): align openrouter prices with live openrouter.ai rates#39591
astraltrekkin wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
astraltrekkin:cursor/berriai-litellm-39501-3141

Conversation

@astraltrekkin

Copy link
Copy Markdown

TLDR

Problem this solves:

  • 41 OpenRouter catalog prices did not match openrouter.ai
  • Spend, budgets, and alerts used those stale per-token rates
  • 20 of those rows still linked the page that contradicted them

How it solves it:

  • Rewrote the drifted input, output, and cache-read rates
  • Matched GET https://openrouter.ai/api/v1/models headline pricing
  • Left max tokens, vision flags, and the auto-updater alone

User Flow

Before: a developer who bills OpenRouter traffic through LiteLLM records the wrong spend because the catalog still has last quarter's rates

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "openrouter/qwen/qwen3.5-plus-02-15"
  2. The completion succeeds and usage is 1M prompt + 1M completion tokens
  3. They open https://litellm-domain/ui/?page=logs and see that request billed at $2.80 ($0.40 + $2.40 per 1M)
  4. The same check for openrouter/openai/gpt-oss-120b shows $0.98 ($0.18 + $0.80 per 1M)

After: the same traffic is billed at the rates OpenRouter publishes today

  1. They send the same POST https://litellm-domain/v1/chat/completions with "model": "openrouter/qwen/qwen3.5-plus-02-15"
  2. The completion succeeds with the same 1M + 1M usage
  3. https://litellm-domain/ui/?page=logs now shows that request at $1.82 ($0.26 + $1.56 per 1M)
  4. openrouter/openai/gpt-oss-120b now shows $0.207 ($0.037 + $0.17 per 1M)

Relevant issues

#39501

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup: curl -s https://openrouter.ai/api/v1/models -o or.json, then compare every overlapping openrouter/* catalog row's input_cost_per_token / output_cost_per_token / cache_read_input_token_cost to the API pricing.prompt / pricing.completion / pricing.input_cache_read

Before ()

  1. Captured at merge base 27274f65e40fa38870b9e1450ab885e09074644d
  2. GET https://openrouter.ai/api/v1/models returned HTTP 200 with 424 models
  3. Compare against model_prices_and_context_window.json at that commit: comparable=87 mismatch_fields=84 mismatch_entries=41
  4. openrouter/qwen/qwen3.5-plus-02-15 registry $0.40 / $2.40 vs live $0.26 / $1.56 (page https://openrouter.ai/qwen/qwen3.5-plus-02-15 agrees with live)
  5. openrouter/openai/gpt-oss-120b registry $0.18 / $0.80 vs live $0.037 / $0.17 (page https://openrouter.ai/openai/gpt-oss-120b agrees with live)
  6. Control openrouter/anthropic/claude-opus-5 already matched live $5 / $25

After ()

  1. Captured at fe8a7846391127cb071b06c9a1f30551e13283b2
  2. Same live GET https://openrouter.ai/api/v1/models (HTTP 200)
  3. Compare against the updated catalog: comparable=87 mismatch_fields=4 mismatch_entries=2
  4. openrouter/qwen/qwen3.5-plus-02-15 registry $0.26 / $1.56 matches live
  5. openrouter/openai/gpt-oss-120b registry $0.037 / $0.17 matches live
  6. Control openrouter/anthropic/claude-opus-5 still $5 / $25
  7. The two leftover rows are later OpenRouter moves on openrouter/deepseek/deepseek-v4-pro and openrouter/deepseek/deepseek-v4-pro-0813, not the original stale values

Type

🐛 Bug Fix

Caveats (if any)

Medium

  • OpenRouter can reprice at any time, so this is a headline snapshot from the live models API
  • Two DeepSeek rows moved again after this patch and still disagree today
  • Tiered *_above_256k_tokens fields were not part of the reported drift and were left alone

Low

  • The weekly auto-updater was not turned on. It still maps max_tokens to context length and would fail the schema on openrouter/auto

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Update the 41 openrouter/* entries whose input, output, or cache-read
costs disagreed with GET https://openrouter.ai/api/v1/models. Leave
max_tokens, vision flags, and the dormant auto-updater untouched.

Co-authored-by: Noa <rainbowgore@users.noreply.github.com>
@codspeed-hq

codspeed-hq Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing rainbowgore:cursor/berriai-litellm-39501-3141 (80a63f0) with litellm_internal_staging (4b1e24e)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates matching OpenRouter input, output, and cache-read prices in both model catalogs and adds a local regression test for selected rates

  • Keeps the root and bundled backup catalogs byte-aligned
  • Updates 41 OpenRouter model entries
  • Adds value checks for 10 changed models and one unchanged control

Confidence Score: 4/5

The pricing changes appear safe to merge, with non-blocking test-coverage and comment-policy cleanup recommended

Both catalogs remain identical and no incorrect changed price was established, but most updated entries lack value-level regression assertions

Files Needing Attention: tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py

Important Files Changed

Filename Overview
model_prices_and_context_window.json Updates OpenRouter token and cache-read rates while preserving the existing model metadata and catalog schema
litellm/model_prices_and_context_window_backup.json Mirrors the root catalog changes exactly for bundled fallback loading
tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py Adds local root and backup price assertions, but covers only 10 of the 41 changed entries and adds a disallowed explanatory comment

Reviews (1): Last reviewed commit: "fix(catalog): align openrouter prices wi..." | Re-trigger Greptile

Comment on lines +253 to +264
_OPENROUTER_LIVE_COSTS = {
"openrouter/qwen/qwen3.5-plus-02-15": (2.6e-07, 1.56e-06, None),
"openrouter/openai/gpt-oss-120b": (3.7e-08, 1.7e-07, None),
"openrouter/qwen/qwen3-coder-plus": (6.5e-07, 3.25e-06, None),
"openrouter/qwen/qwen3.5-flash-02-23": (6.5e-08, 2.6e-07, None),
"openrouter/qwen/qwen3.5-27b": (1.95e-07, 1.56e-06, None),
"openrouter/gryphe/mythomax-l2-13b": (6e-08, 6e-08, None),
"openrouter/mancer/weaver": (4e-07, 7.5e-07, None),
"openrouter/xiaomi/mimo-v2.5-pro": (4.35e-07, 8.7e-07, 3.6e-09),
"openrouter/moonshotai/kimi-k2.5": (4.5e-07, 2.25e-06, 7e-08),
"openrouter/z-ai/glm-5": (6e-07, 1.92e-06, None),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Price fixture covers ten models

The regression fixture pins only 10 of 41 changed models, so incorrect rates in omitted entries can pass while producing wrong spend

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +250 to +252
# OpenRouter headline rates from GET https://openrouter.ai/api/v1/models.
# These were the catalog values that disagreed with that API (and, for the
# two spotlight models, the public model pages that their source fields cite).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Fixture comment duplicates context

This prose restates the fixture's source and purpose, violating the source-comment policy and adding text that can drift from the test

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Co-authored-by: Noa <rainbowgore@users.noreply.github.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Superseded by rolling registry PR #39388, which applies the OpenRouter price fixes re-derived from the live catalog API plus the regression test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants