fix(fireworks_ai): resolve tool_choice and reasoning support for short model names

[devin-ai-integration]

TLDR

Problem this solves:

• fireworks_ai/deepseek-v4-pro-0813 rejects tool_choice with UnsupportedParamsError
• cost map only lists it under the accounts/fireworks/models/ prefix
• the tool_choice and reasoning gates did an exact lookup, so the short name missed

How it solves it:

• gate tool_choice and reasoning params on _get_model_cost_capability_exact first, which already tries the fireworks_ai/accounts/fireworks/models/<name> key
• fall back to the generic supports_tool_choice / supports_reasoning resolvers so their bare-key and provider-info fallbacks keep working (a sweep of all 318 Fireworks cost-map keys shows 16 models gain tool_choice, none lose anything)

User Flow

Before: a developer using opencode against the gateway gets an error on every tool call turn

1. Proxy admin configures model: fireworks_ai/deepseek-v4-pro-0813 in model_list
2. opencode sends POST https://litellm-domain/v1/chat/completions with tools and "tool_choice": "auto"
3. The gateway returns 400 litellm.UnsupportedParamsError: fireworks_ai does not support parameters: ['tool_choice'] and the fallback chain fails too

After: the same request reaches Fireworks and returns a normal completion

1. Proxy admin configures model: fireworks_ai/deepseek-v4-pro-0813 in model_list
2. opencode sends POST https://litellm-domain/v1/chat/completions with tools and "tool_choice": "auto"
3. The gateway returns 200 with the model's tool call or text response

Relevant issues

Reported in an internal Slack thread by a teammate running opencode against the gateway

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

QA'd with the real end-user client: opencode 1.18.29 TUI, driven interactively in tmux, against a live proxy booted with --num_workers 2 (no database) making real Fireworks calls. Same scratch project, same opencode.json (only the port differs), same prompt on both legs; the only difference is the commit the proxy was booted from

Proxy config:

model_list:
  - model_name: deepseek-v4-pro-0813
    litellm_params:
      model: fireworks_ai/deepseek-v4-pro-0813
      api_key: os.environ/FIREWORKS_AI_API_KEY
general_settings:
  master_key: sk-1234

Proxy: .venv/bin/python -m litellm.proxy.proxy_cli --config config.yaml --port <port> --num_workers 2

opencode.json in the scratch project (README.md, hello.py, opencode.json):

{"$schema":"https://opencode.ai/config.json",
 "provider":{"litellm":{"npm":"@ai-sdk/openai-compatible","name":"litellm",
   "options":{"baseURL":"http://localhost:<port>/v1","apiKey":"sk-1234"},
   "models":{"deepseek-v4-pro-0813":{"name":"deepseek-v4-pro-0813"}}}},
 "model":"litellm/deepseek-v4-pro-0813","permission":"allow"}

Before (2e73400, merge base)

1. tmux new -s qa, cd into the scratch project, run opencode
2. Type List the files in this directory and read README.md, press Enter
3. opencode shows the gateway error instead of running the tool call (tmux capture-pane):

┃  List the files in this directory and read README.md
┃
┃  litellm.UnsupportedParamsError: fireworks_ai does not support parameters: ['tool_choice'], for
┃  model=deepseek-v4-pro-0813. To drop these, set `litellm.drop_params=True` or for proxy:
┃
┃  `litellm_settings:
┃   drop_params: true`
┃  .
┃   If you want to use these params dynamically send allowed_openai_params=['tool_choice'] in your
┃  request.. Received Model Group=deepseek-v4-pro-0813
┃  Available Model Group Fallbacks=None
   ▣  Build · deepseek-v4-pro-0813

4. Proxy access log for the turn: "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (the tool-call request) followed by one 200 OK for opencode's session-title request, which carries no tools

After (dad1b13)

1. tmux new -s qa, cd into the same scratch project, run opencode
2. Type List the files in this directory and read README.md, press Enter
3. opencode runs the bash tool and the read tool, then answers (tmux capture-pane):

┃  List the files in this directory and read README.md
┃
┃  $ ls -la
┃
┃  total 24
┃  drwxr-xr-x@ 6 mateo  wheel  192 Sep  4 17:59 .
┃  drwxr-xr-x@ 9 mateo  wheel  288 Sep  4 17:59 ..
┃  drwxr-xr-x@ 9 mateo  wheel  288 Sep  4 17:56 .git
┃  -rw-r--r--@ 1 mateo  wheel   43 Sep  4 17:56 hello.py
┃  -rw-r--r--@ 1 mateo  wheel  329 Sep  4 17:59 opencode.json
┃  -rw-r--r--@ 1 mateo  wheel   86 Sep  4 17:56 README.md
┃
   → Read README.md
   Files:
   - hello.py
   - opencode.json
   - README.md
   README: "QA scratch project (fix). Contains one helper script hello.py that prints a greeting."
   ▣  Build · deepseek-v4-pro-0813 · 17.3s

4. Proxy access log for the turn: four "POST /v1/chat/completions HTTP/1.1" 200 OK, no UnsupportedParamsError

Other unified endpoints (same two proxies, curl)

The gate lives in the shared param validation, so the same short name was driven through the other two unified endpoints and a streaming chat request on both proxies, H being -H "Authorization: Bearer sk-1234" -H "content-type: application/json"

curl -s -w "%{http_code}\n" http://localhost:$PORT/v1/messages "${H[@]}" -d '{"model":"deepseek-v4-pro-0813","max_tokens":300,"messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],"tools":[{"name":"get_weather","description":"Get the weather for a city","input_schema":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}],"tool_choice":{"type":"auto"}}'
curl -s -w "%{http_code}\n" http://localhost:$PORT/v1/responses "${H[@]}" -d '{"model":"deepseek-v4-pro-0813","input":"What is the weather in Paris? Use the tool.","tools":[{"type":"function","name":"get_weather","description":"Get the weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}],"tool_choice":"auto","max_output_tokens":300}'
curl -s -N http://localhost:$PORT/v1/chat/completions "${H[@]}" -d '{"model":"deepseek-v4-pro-0813","messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get the weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"tool_choice":"auto","stream":true,"max_tokens":300}'

Before (2e73400, port 23505), each request:

/v1/messages        400  {"error":{"message":"litellm.UnsupportedParamsError: fireworks_ai does not support parameters: ['tool_choice'], for model=deepseek-v4-pro-0813. ...
/v1/responses       400  {"error":{"message":"litellm.UnsupportedParamsError: fireworks_ai does not support parameters: ['tool_choice'], for model=deepseek-v4-pro-0813. ...
/v1/chat stream     0 data chunks, body is the same UnsupportedParamsError

After (dad1b13, port 26389), each request:

/v1/messages        200  content types ["thinking","text","tool_use"], tool_use name get_weather
/v1/responses       200  output types ["reasoning","message","function_call"], function_call name get_weather
/v1/chat stream     26 data chunks, 6 carrying tool_calls deltas for get_weather

Observed alongside the fix:

• Before leg: tool-free title request already got 200 (left alone)

Type

🐛 Bug Fix

Caveats (if any)

Low

• models absent from the cost map still get no tool_choice, same as before
• _translate_chat_template_kwargs still uses the generic supports_reasoning alone; it does not hit the short-name miss for this model, left as is
• the generic litellm.utils.supports_tool_choice() and litellm.get_model_info() still miss Fireworks short names (/model/info shows supports_tool_choice: null for the short name on both trees); only the param gate resolves them here. Pre-existing and left alone, since fixing it means changing get_provider_info output for every Fireworks model, which is more surface than this fix should carry
• three ci/circleci jobs are red on this PR only because the branch predates staging commits dbf8fe0 and a5a7867 (test: repair four chronically failing CI tests and its review follow-up): litellm_router_unit_testing fails test_router_index_management.py::test_no_linear_scans_in_router (the flagged router.py loops exist at the merge base 2e73400 and staging allowlisted them), proxy_store_model_in_db_tests fails test_openai_error_handling.py::test_missing_model_parameter_curl[chat] (staging relaxed its assertion to HTTP 400 plus a non-empty message), and build_and_test fails tests/test_models.py::test_model_group_info_e2e (staging rewrote the wildcard assertion) plus the Rust harness test_should_derive_ocr_mapping_status_from_live_tests (staging skips it). Staging pipeline 89088 fails all of them the same way and 89097, 89104, and 89115 pass; this PR touches none of those files
• the three red non-required GitHub Actions jobs are staging-wide and unrelated to Fireworks: responses-caching-types (3.10, 3.11) fails tests/test_litellm/caching/test_redis_cache.py::test_event_loop_stall_timeout_burst_keeps_breaker_closed (a stall-timing flake) and proxy-infra (3.10) fails test_http_parsing_utils.py::TestNumericFormFields::test_qualifiers_and_optionality_are_unwrapped, tracked in fix(proxy): strip every TypedDict qualifier before numeric form-field detection #39780; all 33 required checks pass

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
• /live-pr-risk passed at dad1b13 (format-only delta from 2f1da03): dependency sweep over all 318 Fireworks cost-map keys and 3 name shapes, 16 gained tool_choice, 0 regressions; Fireworks translation and local_testing suites pass; live A/B above

Link to Devin session: https://app.devin.ai/sessions/2fccaa5a6fbd40f9b38b0433ecfc6b2e
Open in Devin Desktop: https://app.devin.ai/desktop/session/2fccaa5a6fbd40f9b38b0433ecfc6b2e?variant=devin
Requested by: @krrish-berri-2

---

---

Note

Low Risk
Narrows param validation for Fireworks only by fixing cost-map lookup; behavior change is allowing params that were incorrectly blocked, with tests and no reported capability regressions.

Overview
Fixes false UnsupportedParamsError when callers use short Fireworks model ids (e.g. fireworks_ai/deepseek-v4-pro-0813) while capability flags in the cost map only exist under the accounts/fireworks/models/ key.

get_supported_openai_params now treats tool_choice and reasoning-related params (reasoning_effort, reasoning_history, thinking) as supported when _get_model_cost_capability_exact finds the flag on an alternate cost-map key, or when the existing supports_tool_choice / supports_reasoning helpers still say yes—so short names inherit the prefixed entry without dropping prior fallbacks.

Adds regression tests for the short-name case and for generic reasoning fallback on models like glm-5p3-flash.

^{Reviewed by Cursor Bugbot for commit dad1b13. Bugbot is set up for automated code reviews on this repo. Configure here.}

/live-pr-risk at dad1b13

Breaking: none. Backward incompatible: the get_supported_openai_params list for 16 Fireworks short names (deepseek-v3p1, deepseek-v3p1-terminus, deepseek-v3p2, deepseek-v4-pro-0813, firefunction-v2, glm-4p5, glm-4p5-air, glm-4p6, glm-5p3, glm-5p3-flash, inkling, kimi-k2-instruct, kimi-k2-instruct-0905, kimi-k2-thinking, llama-v3p1-405b-instruct, mixtral-8x22b-instruct-hf) grows by tool_choice; a sweep over all 317 fireworks_ai/ cost-map keys and 575 name shapes shows no key losing a param and no reasoning-param delta on any shape. Regression risk: none found; the only dependents are the provider dispatch in get_supported_openai_params.py, map_openai_params, and get_optional_params validation, all driven live on both trees through /v1/chat/completions (tool_choice required, named, streaming), /v1/messages, /v1/responses, /utils/supported_openai_params, plus reasoning_effort and thinking (identical 200s on both sides). _get_model_cost_capability_exact already gated supports_vision the same way, no subclasses override FireworksAIConfig, and the other supports_tool_choice( callers (azure_ai, bedrock converse, gpt_5) are untouched. Fireworks tests/llm_translation/ and tests/local_testing/ hits pass on the head tree. Merge-ref drift (141 staging commits, 5 touching Fireworks off-peak pricing): re-ran S2 (chat tool_choice required), S6 (/v1/messages), S7 (/v1/responses), and S8 (streaming chat) on a proxy booted from origin/litellm_internal_staging 59d42d3 merged with dad1b13 (76537482cf, 2 workers): all 200 with a get_weather tool call, 24 stream chunks with 6 tool_calls deltas, and /utils/supported_openai_params lists tool_choice

• dad1b13 passes /live-pr-risk

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fireworks_tool_choice_short_name (dad1b13) with litellm_internal_staging (2e73400)}

[greptile-apps]

Greptile Summary

Fixes Fireworks capability detection for short model names by combining normalized exact cost-map lookup with existing generic capability resolution. Adds focused regression coverage for short-name tool-choice and reasoning support, plus generic reasoning fallback preservation

Confidence Score: 5/5

The PR appears safe to merge, with focused capability lookup changes and regression coverage for both corrected and preserved behavior

No actionable correctness, security, compatibility, or repository-rule issue remains in the reviewed changes

Important Files Changed

Filename	Overview
litellm/llms/fireworks_ai/chat/transformation.py	Resolves tool-choice and reasoning capabilities through exact normalized Fireworks metadata while retaining existing generic fallbacks
tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py	Adds focused regression tests for short Fireworks model names and generic reasoning fallback behavior

_{Reviews (2): Last reviewed commit: "style(fireworks_ai): format capability f..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[devin-ai-integration]

🟡 Custom reasoning parameters get rejected

Custom model names matching known reasoning models now fail _get_model_cost_capability_exact. Their reasoning parameters are rejected or silently dropped instead of reaching Fireworks.

Prompt for agents

Restore reasoning capability handling for custom and suffixed Fireworks model names while preserving short-name resolution. In litellm/llms/fireworks_ai/chat/transformation.py, the old supports_reasoning call could obtain supports_reasoning from FireworksAIConfig.get_provider_info, which uses _get_model_cost_capability's hyphen-boundary fallback. The replacement uses only _get_model_cost_capability_exact, so names such as accounts/myorg/models/custom-glm-5p1 no longer advertise reasoning parameters. Choose a lookup that resolves default-account short names without removing the prior fuzzy reasoning support, and add a regression test for a custom or suffixed reasoning model.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit dad1b13. Configure here.}

[mateo-berri]

LGTM