Skip to content

fix(fireworks_ai): restore supports_vision on minimax-m3 in the cost map - #41699

Merged
mateo-berri merged 1 commit into
mainfrom
litellm_fireworks_minimax_m3_supports_vision
Sep 18, 2026
Merged

mateo-berri merged 1 commit into
mainfrom
litellm_fireworks_minimax_m3_supports_vision

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

User Flow

Before: a developer sending an image to Fireworks minimax-m3 through the proxy gets a 400 saying the model cannot take images, while Fireworks itself answers the same request

  1. They send POST https://litellm-domain/v1/chat/completions with model fireworks-minimax-m3 and one user message holding a text block plus an image_url block
  2. The proxy answers HTTP 400: litellm.BadRequestError: Fireworks AI model accounts/fireworks/models/minimax-m3 does not support image inputs. Use a Fireworks vision model or remove image_url content blocks.
  3. GET https://litellm-domain/model/info lists that deployment with supports_vision: false
  4. The same request sent straight to POST https://api.fireworks.ai/inference/v1/chat/completions returns HTTP 200 and describes the image

After: the same image request goes through the proxy and comes back with the image described

  1. They send POST https://litellm-domain/v1/chat/completions with model fireworks-minimax-m3 and one user message holding a text block plus an image_url block
  2. The proxy answers HTTP 200 with the image described, and usage.prompt_tokens includes the image tokens
  3. GET https://litellm-domain/model/info lists that deployment with supports_vision: true
  4. The same request sent straight to POST https://api.fireworks.ai/inference/v1/chat/completions returns HTTP 200 and describes the image

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for every checkpoint below: a fresh worktree at that commit, its own venv, and a proxy booted with --num_workers 2. Every image request goes to Fireworks for real and costs money. The four checkpoints bracket both the regression (the price sync merge) and this fix, so the flag flip is visible from both sides

mat570-proxy-config.yaml:

model_list:
  - model_name: fireworks-minimax-m3
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/minimax-m3
      api_key: os.environ/FIREWORKS_AI_API_KEY
general_settings:
  master_key: sk-mat570-qa

.env holds only FIREWORKS_AI_API_KEY=<key> and LITELLM_LOCAL_MODEL_COST_MAP=True, so each proxy reads the cost map from its own checkout instead of fetching main's copy at boot

Boot, from the worktree root: set -a && source .env && set +a && python litellm/proxy/proxy_cli.py --config mat570-proxy-config.yaml --port <port> --num_workers 2

chat.json:

{"model":"fireworks-minimax-m3","max_tokens":400,"messages":[{"role":"user","content":[{"type":"text","text":"What animal is in this image? Answer with one word."},{"type":"image_url","image_url":{"url":"https://httpbin.org/image/png"}}]}]}

messages.json:

{"model":"fireworks-minimax-m3","max_tokens":400,"messages":[{"role":"user","content":[{"type":"text","text":"What animal is in this image? Answer with one word."},{"type":"image","source":{"type":"url","url":"https://httpbin.org/image/png"}}]}]}

responses.json:

{"model":"fireworks-minimax-m3","max_output_tokens":400,"input":[{"role":"user","content":[{"type":"input_text","text":"What animal is in this image? Answer with one word."},{"type":"input_image","image_url":"https://httpbin.org/image/png"}]}]}

Checkpoint table (the Before and After blocks below are the merge base and the tip rows in full):

Checkpoint Commit Expected POST /v1/chat/completions POST /v1/messages POST /v1/responses GET /model/info
main before #41570 merged decbb96382 200 on every image request, supports_vision: true PASS 200, Pig, prompt_tokens 155 PASS 200, Pig, input_tokens 1 PASS 200, Pig, input_tokens 155 PASS 200, supports_vision: true
#41570 merge commit (the price sync) 87650bfe67 400 on chat and messages, supports_vision: false; responses never hit the gate FAIL 400, does not support image inputs FAIL 400, does not support image inputs PASS 200, Pig, input_tokens 155 FAIL 200, supports_vision: false
this PR's merge base on main e5aa10a1ea 400 on chat and messages, supports_vision: false; responses never hit the gate FAIL 400, does not support image inputs FAIL 400, does not support image inputs PASS 200, Pig, input_tokens 155 FAIL 200, supports_vision: false
this PR's tip 99b83a2d52 200 on every image request, supports_vision: true PASS 200, Pig, prompt_tokens 155 PASS 200, Pig, input_tokens 155 PASS 200, Pig, input_tokens 155 PASS 200, supports_vision: true

Before (e5aa10a)

POST /v1/chat/completions with an image_url block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:58535/v1/chat/completions -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @chat.json

  2. Output (long fields elided, everything else verbatim):

    {"error":{"message":"litellm.BadRequestError: Fireworks AI model accounts/fireworks/models/minimax-m3 does not support image inputs. Use a Fireworks vision model or remove image_url content blocks.. Received Model Group=fireworks-minimax-m3\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"400"}}
    http=400
    

POST /v1/messages with an Anthropic image block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:58535/v1/messages -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @messages.json

  2. Output (long fields elided, everything else verbatim):

    {"error":{"type":"invalid_request_error","message":"litellm.BadRequestError: Fireworks AI model accounts/fireworks/models/minimax-m3 does not support image inputs. Use a Fireworks vision model or remove image_url content blocks.. Received Model Group=fireworks-minimax-m3\nAvailable Model Group Fallbacks=None"}}
    http=400
    

POST /v1/responses with an input_image block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:58535/v1/responses -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @responses.json

  2. Output (long fields elided, everything else verbatim):

    {"id":"[elided]","object":"response","model":"fireworks-minimax-m3","status":"completed","output":[{"type":"reasoning","summary":"[elided]"},{"type":"message","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Pig"}]}],"usage":{"input_tokens":155,"output_tokens":30,"total_tokens":185}}
    http=200
    

GET /model/info reports supports_vision

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:58535/model/info -H "Authorization: Bearer sk-mat570-qa"

  2. Output (long fields elided, everything else verbatim):

    {"data":[{"model_name":"fireworks-minimax-m3","litellm_params":{"model":"fireworks_ai/accounts/fireworks/models/minimax-m3"},"model_info":{"supports_vision":false,"litellm_provider":"fireworks_ai","other_fields":"[elided]"}}]}
    http=200
    

After (99b83a2)

POST /v1/chat/completions with an image_url block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:21664/v1/chat/completions -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @chat.json

  2. Output (long fields elided, everything else verbatim):

    {"id":"chatcmpl-78e18a7cadc2440dae7170e199302031","model":"fireworks-minimax-m3","choices":[{"finish_reason":"stop","message":{"role":"assistant","content":"Pig","reasoning_content":"[elided]"}}],"usage":{"prompt_tokens":155,"completion_tokens":51,"total_tokens":206}}
    http=200
    

POST /v1/messages with an Anthropic image block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:21664/v1/messages -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @messages.json

  2. Output (long fields elided, everything else verbatim):

    {"id":"chatcmpl-ec8e4ba2e6194174a6a1da743fadf446","type":"message","role":"assistant","model":"fireworks-minimax-m3","content":[{"type":"thinking","thinking":"[elided]"},{"type":"text","text":"Pig"}],"stop_reason":"end_turn","usage":{"input_tokens":155,"output_tokens":54}}
    http=200
    

POST /v1/responses with an input_image block

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:21664/v1/responses -H "Authorization: Bearer sk-mat570-qa" -H "Content-Type: application/json" -d @responses.json

  2. Output (long fields elided, everything else verbatim):

    {"id":"[elided]","object":"response","model":"fireworks-minimax-m3","status":"completed","output":[{"type":"reasoning","summary":"[elided]"},{"type":"message","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Pig"}]}],"usage":{"input_tokens":155,"output_tokens":38,"total_tokens":193}}
    http=200
    

GET /model/info reports supports_vision

  1. curl -s -w '\nhttp=%{http_code}\n' http://127.0.0.1:21664/model/info -H "Authorization: Bearer sk-mat570-qa"

  2. Output (long fields elided, everything else verbatim):

    {"data":[{"model_name":"fireworks-minimax-m3","litellm_params":{"model":"fireworks_ai/accounts/fireworks/models/minimax-m3"},"model_info":{"supports_vision":true,"litellm_provider":"fireworks_ai","other_fields":"[elided]"}}]}
    http=200
    

Observations from the run, none introduced by this PR:

  • Fireworks image gate skips /v1/responses; PR leaves it alone

Type

🐛 Bug Fix

Caveats (if any)

Medium

  • The next Fireworks price sync re-applies supports_vision: false from the litellm-providers store
    • That store keeps the first enrichment it saw, and the sync re-publishes every stored row each tick
    • A follow-up PR in litellm-providers fixes the Fireworks source and lets a stated value refresh the row
    • Until it lands, the pins restored here turn that sync PR red instead of main

Low

  • The Fireworks fail-closed gate turns any wrong flag into a user-facing 400
  • Fireworks' catalog still omits image input for minimax-m3; the flag follows the served behavior
  • The gate skips /v1/responses, so that endpoint never returned the 400
  • The same sync (chore(prices): sync Together AI prices: 6 models, 6 deprecated [sync failed: Google Gemini] #41570) rounded 13 wandb max_input_tokens values; out of scope here
  • Proxies without LITELLM_LOCAL_MODEL_COST_MAP fetch main's map at boot and pick this up on merge

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

  • 99b83a2 passes /live-pr-risk

@devin-ai-integration

devin-ai-integration Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge because its model metadata, alias handling, backup map, and regression expectations remain aligned

Summary

Restores vision capability metadata for both Fireworks MiniMax M3 aliases and strengthens regression coverage

  • Updates the primary and backup model maps consistently
  • Verifies both aliases through public and provider-specific capability lookups
  • Requires every Fireworks fixture to assert its vision capability

Reviews (1) · Last reviewed commit: "fix(fireworks_ai): restore supports_visi..."

@codspeed

codspeed Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_fireworks_minimax_m3_supports_vision (99b83a2) with main (deb9d8a)

Open in CodSpeed

@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 99b83a2. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit 02a20fe into main Sep 18, 2026
137 of 141 checks passed
@mateo-berri
mateo-berri deleted the litellm_fireworks_minimax_m3_supports_vision branch September 18, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant