Skip to content

feat: add MiniMax video and image generation providers - #31

Merged
blackflame007 merged 5 commits into
litellm_internal_stagingfrom
litellm_minimax_models
Jul 31, 2026
Merged

feat: add MiniMax video and image generation providers#31
blackflame007 merged 5 commits into
litellm_internal_stagingfrom
litellm_minimax_models

Conversation

@blackflame007

@blackflame007 blackflame007 commented Jul 31, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

The platform has no way to generate video or images through MiniMax. Upstream litellm carries MiniMax chat, messages and TTS only, so MiniMax-H3 (the current 2K multimodal video model), the Hailuo 2.3 family and image-01 are unreachable through /v1/videos and /v1/images/generations, and none of them can be priced or COGS-tracked

How it solves it:

Adds a MinimaxVideoConfig that speaks both MiniMax video APIs behind one provider: minimax/MiniMax-H3 uses the v2 multimodal content API (POST /v2/video_generation with a content array of text, first/last frame and reference media items, polled at GET /v2/query/video_generation/{task_id}), while minimax/MiniMax-Hailuo-2.3 and minimax/MiniMax-Hailuo-2.3-Fast use the legacy v1 task API (POST /v1/video_generation, polled at GET /v1/query/video_generation?task_id=, where success yields a file_id that must be exchanged at GET /v1/files/retrieve for a time-limited download_url). The API family is encoded in the returned video id's model slot at create time, the same way kling encodes its text2video/image2video kind, so status and content lookups rebuild the right URL family from the id alone. A MinimaxImageGenerationConfig adds minimax/image-01 on the synchronous POST /v1/image_generation API. All four models are priced in both cost maps and image cost routing gains a minimax branch reading output_cost_per_image, so spend rows come out non-zero

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

All calls below ran against a local proxy on this branch (python litellm/proxy/proxy_cli.py --config <dev config> --port 4000 --detailed_debug with LITELLM_LOCAL_MODEL_COST_MAP=True and the real stg MINIMAX_API_KEY, models minimax-image-01 -> minimax/image-01, minimax-h3 -> minimax/MiniMax-H3, minimax-hailuo-2.3-fast -> minimax/MiniMax-Hailuo-2.3-Fast), hitting the real MiniMax API and spending real money

image-01, with the response cost header proving the new cost route:

$ curl -s -X POST http://localhost:4000/v1/images/generations -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "minimax-image-01", "prompt": "a lighthouse on a rocky coast at golden hour, cinematic", "n": 1, "size": "1280x720"}' -D -
...
x-litellm-response-cost: 0.0035
...
"data": [{"b64_json": null, "revised_prompt": null, "url": "http://hailuo-image-algeng-data-us.oss-us-east-1.aliyuncs.com/image_inference_output%2F...jpeg?..."}]
$ file mm-image.jpeg
mm-image.jpeg: JPEG image data ... 1280x720, components 3

Hailuo 2.3 Fast, 6s image-to-video from that lighthouse frame, exercising input_reference mapping, the v1 create, the polling status map and the two-hop file retrieve download:

$ curl -s -X POST http://localhost:4000/v1/videos -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "minimax-hailuo-2.3-fast", "prompt": "Waves crash against the rocks as the lighthouse beam sweeps across the twilight sky, camera slowly pushing in", "seconds": 6, "input_reference": "<image url from the previous call>"}' -D -
...
x-litellm-response-cost: 0.192
{"id": "video_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOm1pbmltYXg7bW9kZW...", "object": "video", "status": "queued", "seconds": "6", "model": "minimax-hailuo-2.3-fast", ...}

$ for i in $(seq 1 36); do curl -s "http://localhost:4000/v1/videos/$VID" -H "Authorization: Bearer sk-1234" | python3 -c "import json,sys; print(json.load(sys.stdin)['status'])"; sleep 15; done
queued queued queued in_progress in_progress in_progress completed

$ curl -s "http://localhost:4000/v1/videos/$VID/content" -H "Authorization: Bearer sk-1234" -o out.mp4 && file out.mp4 && ffprobe -v error -show_entries format=duration:stream=codec_name,width,height -of default=noprint_wrappers=1 out.mp4
out.mp4: ISO Media, MP4 Base Media v1 [ISO 14496-12:2003]
codec_name=h264
width=1364
height=768
duration=5.875000

The $0.192 cost header is 6s at the mapped $0.032/s rate, and the downloaded bytes are a real 768P h264 MP4, which proves the v1 path end to end including the file_id -> /v1/files/retrieve -> download_url hop that only exists on this API family

MiniMax-H3 could not be exercised to completion because the account's billing mode does not include it: the live API answers {"type":"error","error":{"type":"bad_request_error","message":"invalid params, TokenPlan or Credit does not currently support MiniMax-H3 series models (2013)","http_code":"400"}}. The same key generated the image and Hailuo video above, and the stg and prod minimax_api_key secrets hold the identical key, so this is an account limitation (H3 requires pay-as-you-go balance) rather than a request-shape problem; the error comes back after auth and model routing accept the request. Once the account has PAYG balance, the H3 curl is the same /v1/videos call with "model": "minimax-h3" and the unit tests pin the v2 request body to the shapes in MiniMax's published OpenAPI spec

Type

🆕 New Feature

Changes

litellm/llms/minimax/videos/transformation.py implements MinimaxVideoConfig. Parameter mapping honors what nolgia-api sends: seconds becomes the integer duration, input_reference (or its image_url mirror, string URL or uploaded file coerced to a base64 data URI) becomes the first frame, end_image_url the last frame, image_urls/audio_urls become v2 reference media, aspect_ratio/size become ratio, and resolution passes through uppercased. Unknown params the MiniMax API would reject with invalid params (2013) (negative_prompt, seed, generate_audio, bitrate_mode, duration_seconds) are dropped, with extra_body kept as the explicit escape hatch. v2 ratio rules follow the API contract: text-to-video defaults to 16:9 because adaptive is rejected there, first/last-frame requests omit ratio because the API forces adaptive, and reference requests send it only when explicitly given. Mixing frame conditioning with reference media, or sending a last frame without a first frame, raises BadRequestError up front instead of burning a paid API call. Reference videos (video_urls) are rejected the same way: MiniMax bills usage.input_seconds for a reference clip on top of the generated output seconds, the clip's length is unknown when the create call is charged, and the status poll that reports the final billed total is never cost-tracked, so accepting them would silently undercharge by up to the 15s reference allowance. Reference images and audio bill no input seconds and stay supported. v1 status strings (Preparing/Queueing/Processing/Success/Fail) and v2 statuses (queued/running/succeeded/failed/cancelled/expired) both map onto the standard queued/in_progress/completed/failed set, unknown states fall toward in_progress, and v2 billing usage lands in usage.duration_seconds from usage.total_seconds (falling back to task.duration) since input reference video seconds bill at the same per-second rate. v1 responses carry a base_resp envelope whose non-zero status_code is an error even on HTTP 200 and is raised with a mapped HTTP status; v2 errors are OpenAI-style bodies whose error.message is extracted for clean surfacing

litellm/llms/minimax/image_generation/transformation.py implements MinimaxImageGenerationConfig: OpenAI size maps onto the API's aspect_ratio enum, n and response_format pass through, and the native aspect_ratio, width, height, seed, prompt_optimizer and subject_reference fields are honored. The response transform handles both image_urls and image_base64 payloads and treats non-zero base_resp.status_code as an error

litellm/llms/minimax/cost_calculator.py plus a minimax branch in route_image_generation_cost_calculator price images from output_cost_per_image, mirroring the xai calculator, because the default image cost path only reads input_cost_per_image/input_cost_per_pixel and would raise. Video pricing needs no provider code; the shared output_cost_per_video_per_second path picks up the new map entries

Registry wiring: ProviderConfigManager.get_provider_video_config and get_provider_image_generation_config gain MINIMAX branches, images/main.py adds MINIMAX to the llm_http_handler allowlist, provider_endpoints_support.json marks video_generations and image_generations, constants.py adds MINIMAX_MEDIA_DEFAULT_API_BASE (excluded in test_env_keys.py as a fork-private var), and both cost maps gain byte-identical minimax/MiniMax-H3 ($0.13/s), minimax/MiniMax-Hailuo-2.3 ($0.056/s), minimax/MiniMax-Hailuo-2.3-Fast ($0.032/s) and minimax/image-01 ($0.0035/image) entries sourced from the published pay-as-you-go pricing

ruff.toml adds TID251 to lint.external next to C901 so RUF100 does not strip noqa directives that the strict gate relies on, which the file's own comment already prescribes for strict-gate rules

Tests: 60 new tests across tests/test_litellm/llms/minimax/{videos,image_generation} and test_minimax_cost_calculator.py cover bearer auth and the missing-key error, URL building for both API families, the full param-mapping matrix including the drop rules, the mutual-exclusion and lone-last-frame errors, the data-URI coercion of file inputs, both create body shapes, id encode/decode roundtrips carrying the provider and model, both status maps parametrized, failure surfacing with error codes, the v2 billed-seconds usage, the v1 two-hop file retrieve with auth header propagation against fixture clients, registry dispatch for both configs, and image/video pricing resolved from the real cost map

Follow-up commit: image generation now maps the platform's image_url param onto MiniMax subject_reference ([{type: character, image_file: url}]) so character references reach image-01; an explicitly passed subject_reference wins over image_url. Covered by two new transformation tests

QA runbook

  1. MINIMAX_API_KEY=<key> LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config <config declaring minimax/image-01, minimax/MiniMax-H3, minimax/MiniMax-Hailuo-2.3-Fast> --detailed_debug
  2. POST /v1/images/generations with model=minimax-image-01, any prompt and size=1280x720; confirm a JPEG URL comes back and the response carries x-litellm-response-cost: 0.0035
  3. POST /v1/videos with model=minimax-hailuo-2.3-fast, a prompt, seconds: 6 and an input_reference image URL; confirm status: queued, x-litellm-response-cost: 0.192 and an opaque video_... id
  4. GET /v1/videos/{id} every 15s until completed (about 2 minutes), then GET /v1/videos/{id}/content and ffprobe the bytes; expect 768P h264 MP4 of about 6s
  5. Once the MiniMax account has pay-as-you-go balance, repeat step 3 with model=minimax-h3 and seconds: 4 to exercise the v2 API ($0.52)

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

View with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is enabled.

Adds minimax/MiniMax-H3 (v2 multimodal content API), minimax/MiniMax-Hailuo-2.3
and minimax/MiniMax-Hailuo-2.3-Fast (legacy v1 task API with the two-hop
files/retrieve download), and minimax/image-01 (synchronous image API) behind
the existing /v1/videos and /v1/images/generations surfaces. Registers the
provider in the video and image config registries, the images allowlist, and
the image cost calculator routing, prices all four models in both cost maps,
and marks the endpoints in provider_endpoints_support.json
@blacksmith-sh

blacksmith-sh Bot commented Jul 31, 2026

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
strict-rule totals exceed their limit (base 3cfb72478928c1f848fc2762e7ada30b96a5ea28):/
strict-rule totals exceed their limit (base 3cfb72478928c1f848fc2762e7ada30b96a5ea28):
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

blackflame007 and others added 2 commits July 31, 2026 08:46
…inator

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5522a4a199

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

status="queued",
model=model,
seconds=str(raw_duration) if raw_duration is not None else None,
usage=_duration_usage(_safe_float(raw_duration)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Charge H3 reference jobs from total billed seconds

For H3 jobs containing reference_videos or other billed reference media, the chargeable create response records only the requested output duration, even though the completed task can report a larger usage.total_seconds that includes reference input. The later status transform exposes that total, but video_retrieve is not in cost_calculator.py's _VIDEO_CALL_TYPES, so it never corrects the initial charge; these jobs therefore undercharge credits and spend tracking. Defer the charge or otherwise account for the provider's final billed total.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Comment on lines +147 to +150
extra_body = video_create_optional_params.get("extra_body")
params = {
**video_create_optional_params,
**(extra_body if isinstance(extra_body, dict) else EMPTY_MAP),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent normalized extra_body aliases from leaking

When MiniMax aliases are supplied through the supported extra_body mechanism, this method normalizes them, but VideoGenerationRequestUtils.get_optional_params_video_generation() subsequently executes mapped_params.update(extra_body). That restores the original aliases alongside the normalized fields—for example, extra_body={"seconds": "4"} produces both duration and top-level seconds, while aspect_ratio or image_urls leak beside ratio or generated content—and the H3 request forwards those unknown fields to an API that rejects them. Remove consumed aliases after the shared merge or avoid translating extra_body before it.

Useful? React with 👍 / 👎.

Comment on lines +142 to +143
response_data = self._parse_json(raw_response)
self._raise_for_minimax_error(raw_response, response_data)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve HTTP status errors from image generation

When the MiniMax endpoint or an intermediary returns a non-2xx response without the v1 base_resp envelope, such as a JSON 401/429 or an HTML 502, the shared image handler does not call raise_for_status() and this transform instead reports a parsing or “no images” ValueError. That discards the real HTTP status, causing incorrect exception mapping and retry behavior; check raw_response.is_success and raise the provider error with the response status before parsing the success payload.

Useful? React with 👍 / 👎.

Comment on lines +103 to +105
size = non_default_params.get("size")
aspect_ratio = (
_SIZE_TO_ASPECT_RATIO.get(size, size.replace("x", ":")) if isinstance(size, str) and size else None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Normalize square image sizes before forwarding

For supported OpenAI square sizes that are absent from the lookup, notably 256x256 and 512x512, this fallback emits aspect_ratio values such as 256:256 instead of the MiniMax enum 1:1. Consequently otherwise portable image requests are rejected upstream even though this config advertises support for the OpenAI size parameter; reduce dimension pairs to a supported ratio or add the standard square sizes to the mapping.

Useful? React with 👍 / 👎.

Comment thread litellm/llms/minimax/common_utils.py Outdated
headers: Mapping[str, Any],
api_key: str | None,
) -> dict: # mutable-ok: validate_environment contracts return dict
final_api_key = api_key or litellm.api_key or get_secret_str("MINIMAX_API_KEY")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prefer the MiniMax-specific key over the global key

When an application has both a global litellm.api_key for another provider and MINIMAX_API_KEY configured, media requests select the unrelated global key and receive authentication failures. MiniMax chat and messages resolve the provider-specific environment key before the global fallback, so image and video calls behave inconsistently in mixed-provider processes; preserve an explicitly passed key first, then prefer MINIMAX_API_KEY, and use litellm.api_key only as the final fallback.

Useful? React with 👍 / 👎.

Comment on lines +509 to +510
api_base = str(raw_response.request.url).split("/v1/")[0]
url = f"{api_base}/v1/files/retrieve?file_id={quote(str(file_id), safe='')}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain custom base paths for legacy file retrieval

For a path-prefixed custom media base containing /v1/, such as https://gateway.example/v1/minimax, splitting on the first occurrence strips the proxy prefix when constructing the second-hop file-retrieval URL. Creation and polling can therefore succeed through the configured gateway while content download unexpectedly calls https://gateway.example/v1/files/retrieve; derive the base from the final provider-path occurrence or carry the configured base through directly.

Useful? React with 👍 / 👎.

"minimax/image-01": {
"litellm_provider": "minimax",
"mode": "image_generation",
"output_cost_per_image": 0.0035,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Load fork-private pricing in the default runtime

These new prices are only present in the repository and bundled backup maps, while normal startup successfully fetches the upstream remote cost map and returns it wholesale; the local map is selected only when LITELLM_LOCAL_MODEL_COST_MAP=True or the remote fetch fails validation. Because these are fork-private models, a healthy default production startup omits all four entries, so get_model_info() cannot supply the image rate and video spend is missing despite the tests and QA forcing local-map mode. Merge fork entries into the fetched map or configure stage/prod to use the bundled map before relying on this billing behavior.

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fork-wide deployment concern rather than a MiniMax defect: the cost-map loader returns the upstream remote map unless LITELLM_LOCAL_MODEL_COST_MAP=True, so every fork-private entry (including the merged grok-imagine pricing from #29) is affected equally. Fixing it needs an infra stage/prod env change or a new fork-overlay mechanism in the loader, both out of scope for this provider PR.

blackflame007 and others added 2 commits July 31, 2026 08:55
…t be undercharged

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…s leakage and keep custom base paths

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@blackflame007
blackflame007 merged commit cd47c86 into litellm_internal_staging Jul 31, 2026
50 of 52 checks passed
blackflame007 added a commit that referenced this pull request Aug 3, 2026
Base moved four commits ahead (MiniMax video/image providers #31, the blind-catch
narrowing #33, and the two Gemini Veo route fixes #34/#35). Everything auto-merged
except tests/test_litellm/interactions/test_openapi_compliance.py, where both sides
had independently loosened the same Content-discriminator assertion after Google's
spec dropped the keyword: base via #31, upstream via BerriAI#35161. Kept upstream's
version, which is a strict superset (it accepts a discriminator mapping *or* a
per-variant `type` const/1-item enum, asserts the values are distinct, and pins
TextContent to "text") and matches the `_declared_type_value` helper already in
the file.

Budget ceilings: LIT002 (27511 -> 27678) and TRY004 (98 -> 100) were the only two
rules over limit on the merged tree with a count above the base, so the gates would
have failed. The merge adds no net-new violations: every file's LIT002 count in the
merged tree equals one of its two parents, so the overage is purely the union of
ceilings both sides had ratcheted down independently since the branch point. Raised
those two to the merged tree's actual counts; every other ratchet is untouched.

Also regenerated model_prices_and_context_window.schema.json, which the new upstream
sync check flagged as stale (missing `output_cost_per_audio`) already before this
merge.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant