Skip to content

fix(cost-map): add tool calling and reasoning flags, correct max output for nebius DeepSeek-V4.1-Flash - #43588

Merged
kerry-berri merged 1 commit into
BerriAI:mainfrom
Flexomatic81:fix/nebius-deepseek-v4.1-flash-capabilities
Sep 29, 2026
Merged

kerry-berri merged 1 commit into
BerriAI:mainfrom
Flexomatic81:fix/nebius-deepseek-v4.1-flash-capabilities

Conversation

@Flexomatic81

Copy link
Copy Markdown

TLDR

Problem this solves:

  • nebius/deepseek-ai/DeepSeek-V4.1-Flash row lacks tool calling and reasoning flags
  • Its max output tokens equal the 1M context, which is wrong
  • Proxies expose the row through /model/info; clients hide tools and reasoning

How it solves it:

  • Sets supports_function_calling and supports_reasoning to true
  • Sets max_output_tokens and max_tokens to 384000
  • Verified live against Nebius Token Factory through a LiteLLM proxy

User Flow

Before: a proxy admin who added the Nebius DeepSeek V4.1 Flash deployment sees no capabilities for it, so their client app disables tools, reasoning and shows a 1M output limit

  1. They add the model at https://litellm-domain/ui/?page=models with provider nebius and model deepseek-ai/DeepSeek-V4.1-Flash
  2. They call GET https://litellm-domain/model/info and get supports_function_calling: null, supports_reasoning: null, max_output_tokens: 1048576
  3. Their client app reads that and offers the model without tools or reasoning, while the model actually supports both

After: the same call reports the real capabilities and output limit

  1. They add the model at https://litellm-domain/ui/?page=models the same way
  2. They call GET https://litellm-domain/model/info and get supports_function_calling: true, supports_reasoning: true, max_output_tokens: 384000
  3. Their client app enables tools and reasoning for the model

Relevant issues

The row was introduced in #42951 from the Nebius catalog page, which renders "Tool calling N/A" and only the context size, so the flags were left out and the output limit was copied from the context.

Pre-Submission checklist

  • I have added meaningful tests (data-only change; covered by the cost map guard, which validates schema and backup sync)
  • The handful of test files covering my change pass locally: python ci_cd/cost_map_guard.py --base <merge-base> --head <tip> --head-ref <branch> passes
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Setup: LiteLLM proxy with the Nebius deployment deepseek-ai/DeepSeek-V4.1-Flash (custom_llm_provider: nebius, api_base: https://api.tokenfactory.nebius.com/v1/). Live calls below went through that proxy to Nebius Token Factory on 2026-09-28 (real spend, no mocks).

Before (6f5ad78)

Row in the cost map

  1. python -c "import json; e=json.load(open('model_prices_and_context_window.json'))['nebius/deepseek-ai/DeepSeek-V4.1-Flash']; print({k: e.get(k) for k in ('max_output_tokens','supports_function_calling','supports_reasoning','supports_vision')})"
  2. Output: {'max_output_tokens': 1048576, 'supports_function_calling': None, 'supports_reasoning': None, 'supports_vision': True}

Tool call through the proxy

  1. POST /v1/chat/completions with tools: [get_weather(city)], tool_choice: auto, user message "Wie ist das Wetter in Berlin? Nutze das Werkzeug."
  2. Response: finish_reason: "tool_calls", tool_calls: [{"name": "get_weather", "arguments": "{\"city\": \"Berlin\"}"}] — the provider supports tools, the map says nothing

Reasoning through the proxy

  1. POST /v1/chat/completions with reasoning_effort: "medium", user message "Was ist 17 * 23? Denke Schritt für Schritt und antworte dann nur mit der Zahl."
  2. Response: message.reasoning_content with 211 characters, usage.completion_tokens_details.reasoning_tokens: 57, content: "391" — the provider emits reasoning, the map says nothing

Output limit through the proxy

  1. POST /v1/chat/completions with max_tokens: 384000, user message "Antworte nur mit: ok"
  2. Response: HTTP 200 — 384000 is accepted; the Hugging Face model card states max_tokens: ≥ 256K, DeepSeek's own API and OpenRouter list 384K for the same model

After (f6c33a0)

Row in the cost map

  1. Same command as before
  2. Output: {'max_output_tokens': 384000, 'supports_function_calling': True, 'supports_reasoning': True, 'supports_vision': True}

Tool call through the proxy

  1. Same request
  2. Same response (finish_reason: "tool_calls", get_weather with {"city": "Berlin"}); the map now matches

Reasoning through the proxy

  1. Same request
  2. Same response (reasoning_content present, reasoning_tokens: 57); the map now matches

Output limit through the proxy

  1. Same request
  2. HTTP 200; the map now states 384000

Vision was also checked (a 64×64 blue PNG as image_url, answer "Blau"); supports_vision: true was already correct and is unchanged.

Type

🐛 Bug Fix

Caveats (if any)

Low

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…ut for nebius DeepSeek-V4.1-Flash

The row added in BerriAI#42951 was built from the Nebius catalog page, which shows
"Tool calling N/A" and only the context size. Verified live against Nebius
Token Factory through a LiteLLM proxy: the model returns tool_calls, emits
reasoning_content with reasoning_tokens in usage, reads images and accepts
max_tokens=384000.

- supports_function_calling: true
- supports_reasoning: true
- max_output_tokens / max_tokens: 1048576 -> 384000
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Updates model capability flags and token limits for one LLM.

The PR appears safe to merge; no actionable issue was established.

Summary

Updates the Nebius DeepSeek-V4.1-Flash cost-map entry to advertise tool calling and reasoning, and changes its output-token limit to 384,000. The same metadata change is mirrored in the backup map.

Reviews (1) · Last reviewed commit: "fix(cost-map): add tool calling and reas..."

@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing Flexomatic81:fix/nebius-deepseek-v4.1-flash-capabilities (f6c33a0) with main (6f5ad78)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (0f96d09) during the generation of this report, so 6f5ad78 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@kerry-berri
kerry-berri merged commit 7b2cbf6 into BerriAI:main Sep 29, 2026
95 of 97 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants