Skip to content

refactor(cost-map): remove openrouter rows, part 3 of 5 - #45667

Merged
kerrylu-berri merged 4 commits into
mainfrom
litellm_remove_openrouter_cost_map_3
Oct 9, 2026
Merged

kerrylu-berri merged 4 commits into
mainfrom
litellm_remove_openrouter_cost_map_3

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Summary

Part 3 of 5 of removing the redundant litellm_provider: openrouter rows from the cost map (OpenRouter bills from usage.cost since #45640; see #45665 for why this is split into 5 stacked deletion-only PRs). Stacked on part 2

Deletes the third and fourth contiguous blocks, 139 rows starting at openrouter/minimax/minimax-m3 and including openrouter/typesafe/jev-1.13, from model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json. The *_above_32k_tokens, *_above_128k_tokens and *_above_256k_tokens fields were only used by OpenRouter rows, so ci_cd/generate_model_prices_schema.py drops them from model_prices_and_context_window.schema.json here. No source or test changes

Check against the base branch:

python3 -c "import json,subprocess as s;o=json.loads(s.check_output(['git','show','origin/litellm_remove_openrouter_cost_map_2:model_prices_and_context_window.json']));n=json.load(open('model_prices_and_context_window.json'));r=set(o)-set(n);print(len(r),set(n)-set(o),all(o[k]['litellm_provider']=='openrouter' for k in r),all(o[k]==n[k] for k in n))"
# 139 set() True True

Link to Devin session: https://app.devin.ai/sessions/24b478924abd4f92b269b1556a5fcca6
Open in Devin Desktop: https://app.devin.ai/desktop/session/24b478924abd4f92b269b1556a5fcca6?variant=devin
Requested by: @kerry-berri


Note

Devin errored when opening this Pull Request as kerry-berri.
As a fallback, Devin opened this PR as itself.

kerry and others added 3 commits October 9, 2026 21:09
… google block)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


kerry seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 3/5

[Medium impact] Removes pricing schema fields for cached token tiers.

Preserve model information and pre-response routing prices before merging

Findings

  1. P1 Cheapest routing picks wrong models ▶

Summary

Removes 139 OpenRouter records from both model catalogs and drops ten now-unused price properties from the generated schema

  • The cost maps no longer list these 139 OpenRouter models.

Reviews (1) · Last reviewed commit: "refactor(cost-map): remove openrouter ro..." · Reviewed by Greptile

Comment on lines 65183 to 65185
"together_ai/arcee-ai/trinity-mini": {
"input_cost_per_token": 4.5e-08,
"litellm_provider": "together_ai",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Cheapest routing picks wrong models

For a mixed-provider model group without custom prices, lowest-cost routing reads these records before sending a request. Deleting them makes both token rates default to 5.0, so the router ranks each deleted OpenRouter deployment at 10.0 and can choose a more expensive provider. usage.cost arrives after that choice and cannot fix it. Keep prices available for pre-response routing

Knowledge Base Used: Provider adapters and capabilities

@greptile-apps

greptile-apps Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings could not be posted inline.

  • P1 Model information disappears model_prices_and_context_window.json:65196 ▶

    Deleting openrouter/minimax/minimax-m3 removes its limits and capability flags, not just prices. OpenRouter has no dynamic metadata lookup, and the existing generalizations do not cover MiniMax. get_model_info now raises ModelNotMappedError, while supports_function_calling returns false for a model previously marked as supporting tools. Keep this metadata available. The repository requires an opt-in for incompatible changes

@codecov

codecov Bot commented Oct 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Base automatically changed from litellm_remove_openrouter_cost_map_2 to main October 9, 2026 22:00
@codspeed

codspeed Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

✅ 31 untouched benchmarks


Comparing litellm_remove_openrouter_cost_map_3 (5bdf53e) with main (aa64b07)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (2248fce) during the generation of this report, so aa64b07 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@kerrylu-berri
kerrylu-berri enabled auto-merge (squash) October 9, 2026 22:32
@kerrylu-berri
kerrylu-berri merged commit ed22fd9 into main Oct 9, 2026
137 of 139 checks passed
@kerrylu-berri
kerrylu-berri deleted the litellm_remove_openrouter_cost_map_3 branch October 9, 2026 22:33
mateo-berri added a commit that referenced this pull request Oct 10, 2026
…-reported cost (#45853)

PR #45667 removed openrouter/typesafe/jev-1.13 from the cost map after #45640 moved OpenRouter decisions billing to usage.cost. The OpenAI-format wire test still read the removed row and raised KeyError on CircleCI integration-providers

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
DeoLeung pushed a commit to aiyou178/litellm that referenced this pull request Oct 11, 2026
…-reported cost (BerriAI#45853)

PR BerriAI#45667 removed openrouter/typesafe/jev-1.13 from the cost map after BerriAI#45640 moved OpenRouter decisions billing to usage.cost. The OpenAI-format wire test still read the removed row and raised KeyError on CircleCI integration-providers

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 5894968)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants