Skip to content

fix(cost-map): raise baseten DeepSeek-V4.1-Flash max output to 262144 #43916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kerry-berri merged 1 commit into from
Oct 1, 2026
Merged

fix(cost-map): raise baseten DeepSeek-V4.1-Flash max output to 262144 #43916

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions litellm/model_prices_and_context_window_backup.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77201,8 +77201,8 @@
"input_cost_per_token": 3e-07,
"litellm_provider": "baseten",
"max_input_tokens": 1048576,
"max_output_tokens": 32768,
"max_tokens": 32768,
"max_output_tokens": 262144,
"max_tokens": 262144,
"mode": "chat",
"output_cost_per_token": 1.2e-06,
"source": "https://inference.baseten.co/v1/models",
Expand Down
4 changes: 2 additions & 2 deletions model_prices_and_context_window.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77201,8 +77201,8 @@
"input_cost_per_token": 3e-07,
"litellm_provider": "baseten",
"max_input_tokens": 1048576,
"max_output_tokens": 32768,
"max_tokens": 32768,
"max_output_tokens": 262144,
"max_tokens": 262144,
Comment on lines +77204 to +77205

@greptile-apps greptile-apps Bot Sep 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Requests rejected by token limit

If a deployment has an output-token-per-minute limit below 262144, a request without an explicit max_tokens now reserves the full 262144 tokens before it runs. The pre-call check rejects the request even if it would produce a short response. For example, a deployment limited to 100000 output tokens per minute previously admitted such requests but now rejects them all. The backup catalog mirrors this change.

Knowledge Base Used: Provider adapters and capabilities

@greptile-apps greptile-apps Bot Sep 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s fair. This row should reflect Baseten’s current maxOutput, and the router’s OTPM reservation behavior is separate from the catalog correction. I’m withdrawing the finding for this PR; any change to avoid reserving the full catalog maximum belongs in the router’s rate-limit logic, with its own tests, rather than in this metadata update.

"mode": "chat",
"output_cost_per_token": 1.2e-06,
"source": "https://inference.baseten.co/v1/models",
Expand Down
Loading