Skip to content

test(fireworks_ai): stop pinning vision support on minimax-m3 - #41627

Merged
kerry-berri merged 1 commit into
mainfrom
litellm_fireworks_minimax_m3_vision_tests
Sep 17, 2026
Merged

kerry-berri merged 1 commit into
mainfrom
litellm_fireworks_minimax_m3_vision_tests

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • Stop asserting a vision value for the minimax-m3 catalog entry
  • Move the vision helper tests onto llama-v3p2-11b-vision-instruct

User Flow

Test-only change, no user-visible behavior

Relevant issues

Unblocks #41570

Affected release

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Raw provider evidence, fetched today with a cache buster from GET https://api.fireworks.ai/v1/serverless/models:

accounts/fireworks/models/minimax-m3
  input_modalities:  ["text"]
  output_modalities: ["text"]

The sync writes supports_vision from input_modalities, so the catalog value is what the provider publishes. The tests pinned the opposite

The changed tests pass against both the main catalog and the #41570 catalog:

python3 -m pytest tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py tests/test_litellm/test_utils.py -q -k "fireworks or vision or transform_messages"
120 passed, 378 deselected

Type

✅ Test

Caveats (if any)

Low

  • The minimax-m3 entry no longer pins vision either way; the other Fireworks rows still do

Link to Devin session: https://app.devin.ai/sessions/70717d80beaa4b07b193697a69f1e6a7
Open in Devin Desktop: https://app.devin.ai/desktop/session/70717d80beaa4b07b193697a69f1e6a7?variant=devin


Note

Low Risk
Test-only updates to Fireworks model metadata and vision helper coverage; no runtime or production behavior changes.

Overview
Test-only change so Fireworks catalog tests match provider-published supports_vision instead of hardcoding vision on minimax-m3.

The shared Fireworks model-cost parametrization now uses None for minimax-m3’s expected vision flag and only asserts supports_vision when that expectation is set. Vision behavior tests in the Fireworks chat transformation suite were retargeted from minimax-m3 to llama-v3p2-11b-vision-instruct (including _transform_messages_helper image cases).

This unblocks the price/catalog sync in PR #41570, where the Fireworks API reports minimax-m3 as text-only.

Reviewed by Cursor Bugbot for commit 6d20e68. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 17, 2026 18:58
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6d20e68. Configure here.

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge because it only removes an unstable provider-owned expectation while retaining relevant capability and transformation coverage

Summary

This test-only PR stops requiring minimax-m3 to support vision and moves Fireworks image transformation coverage to a cataloged vision model

  • Allows the minimax-m3 catalog capability to follow synchronized provider metadata
  • Uses llama-v3p2-11b-vision-instruct for positive vision and image pass-through checks
  • Preserves shorthand Fireworks capability coverage elsewhere

Reviews (1) · Last reviewed commit: "test(fireworks_ai): stop pinning vision ..."

@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant