feat: add MiniMax video and image generation providers - #31
Conversation
Adds minimax/MiniMax-H3 (v2 multimodal content API), minimax/MiniMax-Hailuo-2.3 and minimax/MiniMax-Hailuo-2.3-Fast (legacy v1 task API with the two-hop files/retrieve download), and minimax/image-01 (synchronous image API) behind the existing /v1/videos and /v1/images/generations surfaces. Registers the provider in the video and image config registries, the images allowlist, and the image cost calculator routing, prices all four models in both cost maps, and marks the endpoints in provider_endpoints_support.json
|
Found 1 test failure on Blacksmith runners: Failure
|
…inator Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5522a4a199
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| status="queued", | ||
| model=model, | ||
| seconds=str(raw_duration) if raw_duration is not None else None, | ||
| usage=_duration_usage(_safe_float(raw_duration)), |
There was a problem hiding this comment.
Charge H3 reference jobs from total billed seconds
For H3 jobs containing reference_videos or other billed reference media, the chargeable create response records only the requested output duration, even though the completed task can report a larger usage.total_seconds that includes reference input. The later status transform exposes that total, but video_retrieve is not in cost_calculator.py's _VIDEO_CALL_TYPES, so it never corrects the initial charge; these jobs therefore undercharge credits and spend tracking. Defer the charge or otherwise account for the provider's final billed total.
AGENTS.md reference: AGENTS.md:L15-L15
Useful? React with 👍 / 👎.
| extra_body = video_create_optional_params.get("extra_body") | ||
| params = { | ||
| **video_create_optional_params, | ||
| **(extra_body if isinstance(extra_body, dict) else EMPTY_MAP), |
There was a problem hiding this comment.
Prevent normalized extra_body aliases from leaking
When MiniMax aliases are supplied through the supported extra_body mechanism, this method normalizes them, but VideoGenerationRequestUtils.get_optional_params_video_generation() subsequently executes mapped_params.update(extra_body). That restores the original aliases alongside the normalized fields—for example, extra_body={"seconds": "4"} produces both duration and top-level seconds, while aspect_ratio or image_urls leak beside ratio or generated content—and the H3 request forwards those unknown fields to an API that rejects them. Remove consumed aliases after the shared merge or avoid translating extra_body before it.
Useful? React with 👍 / 👎.
| response_data = self._parse_json(raw_response) | ||
| self._raise_for_minimax_error(raw_response, response_data) |
There was a problem hiding this comment.
Preserve HTTP status errors from image generation
When the MiniMax endpoint or an intermediary returns a non-2xx response without the v1 base_resp envelope, such as a JSON 401/429 or an HTML 502, the shared image handler does not call raise_for_status() and this transform instead reports a parsing or “no images” ValueError. That discards the real HTTP status, causing incorrect exception mapping and retry behavior; check raw_response.is_success and raise the provider error with the response status before parsing the success payload.
Useful? React with 👍 / 👎.
| size = non_default_params.get("size") | ||
| aspect_ratio = ( | ||
| _SIZE_TO_ASPECT_RATIO.get(size, size.replace("x", ":")) if isinstance(size, str) and size else None |
There was a problem hiding this comment.
Normalize square image sizes before forwarding
For supported OpenAI square sizes that are absent from the lookup, notably 256x256 and 512x512, this fallback emits aspect_ratio values such as 256:256 instead of the MiniMax enum 1:1. Consequently otherwise portable image requests are rejected upstream even though this config advertises support for the OpenAI size parameter; reduce dimension pairs to a supported ratio or add the standard square sizes to the mapping.
Useful? React with 👍 / 👎.
| headers: Mapping[str, Any], | ||
| api_key: str | None, | ||
| ) -> dict: # mutable-ok: validate_environment contracts return dict | ||
| final_api_key = api_key or litellm.api_key or get_secret_str("MINIMAX_API_KEY") |
There was a problem hiding this comment.
Prefer the MiniMax-specific key over the global key
When an application has both a global litellm.api_key for another provider and MINIMAX_API_KEY configured, media requests select the unrelated global key and receive authentication failures. MiniMax chat and messages resolve the provider-specific environment key before the global fallback, so image and video calls behave inconsistently in mixed-provider processes; preserve an explicitly passed key first, then prefer MINIMAX_API_KEY, and use litellm.api_key only as the final fallback.
Useful? React with 👍 / 👎.
| api_base = str(raw_response.request.url).split("/v1/")[0] | ||
| url = f"{api_base}/v1/files/retrieve?file_id={quote(str(file_id), safe='')}" |
There was a problem hiding this comment.
Retain custom base paths for legacy file retrieval
For a path-prefixed custom media base containing /v1/, such as https://gateway.example/v1/minimax, splitting on the first occurrence strips the proxy prefix when constructing the second-hop file-retrieval URL. Creation and polling can therefore succeed through the configured gateway while content download unexpectedly calls https://gateway.example/v1/files/retrieve; derive the base from the final provider-path occurrence or carry the configured base through directly.
Useful? React with 👍 / 👎.
| "minimax/image-01": { | ||
| "litellm_provider": "minimax", | ||
| "mode": "image_generation", | ||
| "output_cost_per_image": 0.0035, |
There was a problem hiding this comment.
Load fork-private pricing in the default runtime
These new prices are only present in the repository and bundled backup maps, while normal startup successfully fetches the upstream remote cost map and returns it wholesale; the local map is selected only when LITELLM_LOCAL_MODEL_COST_MAP=True or the remote fetch fails validation. Because these are fork-private models, a healthy default production startup omits all four entries, so get_model_info() cannot supply the image rate and video spend is missing despite the tests and QA forcing local-map mode. Merge fork entries into the fetched map or configure stage/prod to use the bundled map before relying on this billing behavior.
AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fork-wide deployment concern rather than a MiniMax defect: the cost-map loader returns the upstream remote map unless LITELLM_LOCAL_MODEL_COST_MAP=True, so every fork-private entry (including the merged grok-imagine pricing from #29) is affected equally. Fixing it needs an infra stage/prod env change or a new fork-overlay mechanism in the loader, both out of scope for this provider PR.
…t be undercharged Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…s leakage and keep custom base paths Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
cd47c86
into
litellm_internal_staging
Base moved four commits ahead (MiniMax video/image providers #31, the blind-catch narrowing #33, and the two Gemini Veo route fixes #34/#35). Everything auto-merged except tests/test_litellm/interactions/test_openapi_compliance.py, where both sides had independently loosened the same Content-discriminator assertion after Google's spec dropped the keyword: base via #31, upstream via BerriAI#35161. Kept upstream's version, which is a strict superset (it accepts a discriminator mapping *or* a per-variant `type` const/1-item enum, asserts the values are distinct, and pins TextContent to "text") and matches the `_declared_type_value` helper already in the file. Budget ceilings: LIT002 (27511 -> 27678) and TRY004 (98 -> 100) were the only two rules over limit on the merged tree with a count above the base, so the gates would have failed. The merge adds no net-new violations: every file's LIT002 count in the merged tree equals one of its two parents, so the overage is purely the union of ceilings both sides had ratcheted down independently since the branch point. Raised those two to the merged tree's actual counts; every other ratchet is untouched. Also regenerated model_prices_and_context_window.schema.json, which the new upstream sync check flagged as stale (missing `output_cost_per_audio`) already before this merge. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light.png)
TLDR
Problem this solves:
The platform has no way to generate video or images through MiniMax. Upstream litellm carries MiniMax chat, messages and TTS only, so MiniMax-H3 (the current 2K multimodal video model), the Hailuo 2.3 family and image-01 are unreachable through
/v1/videosand/v1/images/generations, and none of them can be priced or COGS-trackedHow it solves it:
Adds a
MinimaxVideoConfigthat speaks both MiniMax video APIs behind one provider:minimax/MiniMax-H3uses the v2 multimodal content API (POST /v2/video_generationwith a content array of text, first/last frame and reference media items, polled atGET /v2/query/video_generation/{task_id}), whileminimax/MiniMax-Hailuo-2.3andminimax/MiniMax-Hailuo-2.3-Fastuse the legacy v1 task API (POST /v1/video_generation, polled atGET /v1/query/video_generation?task_id=, where success yields afile_idthat must be exchanged atGET /v1/files/retrievefor a time-limiteddownload_url). The API family is encoded in the returned video id's model slot at create time, the same way kling encodes its text2video/image2video kind, so status and content lookups rebuild the right URL family from the id alone. AMinimaxImageGenerationConfigaddsminimax/image-01on the synchronousPOST /v1/image_generationAPI. All four models are priced in both cost maps and image cost routing gains a minimax branch readingoutput_cost_per_image, so spend rows come out non-zeroRelevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
All calls below ran against a local proxy on this branch (
python litellm/proxy/proxy_cli.py --config <dev config> --port 4000 --detailed_debugwithLITELLM_LOCAL_MODEL_COST_MAP=Trueand the real stgMINIMAX_API_KEY, modelsminimax-image-01 -> minimax/image-01,minimax-h3 -> minimax/MiniMax-H3,minimax-hailuo-2.3-fast -> minimax/MiniMax-Hailuo-2.3-Fast), hitting the real MiniMax API and spending real moneyimage-01, with the response cost header proving the new cost route:
Hailuo 2.3 Fast, 6s image-to-video from that lighthouse frame, exercising
input_referencemapping, the v1 create, the polling status map and the two-hop file retrieve download:The $0.192 cost header is 6s at the mapped $0.032/s rate, and the downloaded bytes are a real 768P h264 MP4, which proves the v1 path end to end including the
file_id -> /v1/files/retrieve -> download_urlhop that only exists on this API familyMiniMax-H3 could not be exercised to completion because the account's billing mode does not include it: the live API answers
{"type":"error","error":{"type":"bad_request_error","message":"invalid params, TokenPlan or Credit does not currently support MiniMax-H3 series models (2013)","http_code":"400"}}. The same key generated the image and Hailuo video above, and the stg and prodminimax_api_keysecrets hold the identical key, so this is an account limitation (H3 requires pay-as-you-go balance) rather than a request-shape problem; the error comes back after auth and model routing accept the request. Once the account has PAYG balance, the H3 curl is the same/v1/videoscall with"model": "minimax-h3"and the unit tests pin the v2 request body to the shapes in MiniMax's published OpenAPI specType
🆕 New Feature
Changes
litellm/llms/minimax/videos/transformation.pyimplementsMinimaxVideoConfig. Parameter mapping honors what nolgia-api sends:secondsbecomes the integerduration,input_reference(or itsimage_urlmirror, string URL or uploaded file coerced to a base64 data URI) becomes the first frame,end_image_urlthe last frame,image_urls/audio_urlsbecome v2 reference media,aspect_ratio/sizebecomeratio, andresolutionpasses through uppercased. Unknown params the MiniMax API would reject withinvalid params (2013)(negative_prompt,seed,generate_audio,bitrate_mode,duration_seconds) are dropped, withextra_bodykept as the explicit escape hatch. v2 ratio rules follow the API contract: text-to-video defaults to16:9because adaptive is rejected there, first/last-frame requests omitratiobecause the API forces adaptive, and reference requests send it only when explicitly given. Mixing frame conditioning with reference media, or sending a last frame without a first frame, raisesBadRequestErrorup front instead of burning a paid API call. Reference videos (video_urls) are rejected the same way: MiniMax billsusage.input_secondsfor a reference clip on top of the generated output seconds, the clip's length is unknown when the create call is charged, and the status poll that reports the final billed total is never cost-tracked, so accepting them would silently undercharge by up to the 15s reference allowance. Reference images and audio bill no input seconds and stay supported. v1 status strings (Preparing/Queueing/Processing/Success/Fail) and v2 statuses (queued/running/succeeded/failed/cancelled/expired) both map onto the standard queued/in_progress/completed/failed set, unknown states fall toward in_progress, and v2 billing usage lands inusage.duration_secondsfromusage.total_seconds(falling back totask.duration) since input reference video seconds bill at the same per-second rate. v1 responses carry abase_respenvelope whose non-zerostatus_codeis an error even on HTTP 200 and is raised with a mapped HTTP status; v2 errors are OpenAI-style bodies whoseerror.messageis extracted for clean surfacinglitellm/llms/minimax/image_generation/transformation.pyimplementsMinimaxImageGenerationConfig: OpenAIsizemaps onto the API'saspect_ratioenum,nandresponse_formatpass through, and the nativeaspect_ratio,width,height,seed,prompt_optimizerandsubject_referencefields are honored. The response transform handles bothimage_urlsandimage_base64payloads and treats non-zerobase_resp.status_codeas an errorlitellm/llms/minimax/cost_calculator.pyplus a minimax branch inroute_image_generation_cost_calculatorprice images fromoutput_cost_per_image, mirroring the xai calculator, because the default image cost path only readsinput_cost_per_image/input_cost_per_pixeland would raise. Video pricing needs no provider code; the sharedoutput_cost_per_video_per_secondpath picks up the new map entriesRegistry wiring:
ProviderConfigManager.get_provider_video_configandget_provider_image_generation_configgain MINIMAX branches,images/main.pyadds MINIMAX to the llm_http_handler allowlist,provider_endpoints_support.jsonmarksvideo_generationsandimage_generations,constants.pyaddsMINIMAX_MEDIA_DEFAULT_API_BASE(excluded intest_env_keys.pyas a fork-private var), and both cost maps gain byte-identicalminimax/MiniMax-H3($0.13/s),minimax/MiniMax-Hailuo-2.3($0.056/s),minimax/MiniMax-Hailuo-2.3-Fast($0.032/s) andminimax/image-01($0.0035/image) entries sourced from the published pay-as-you-go pricingruff.tomladdsTID251tolint.externalnext toC901so RUF100 does not strip noqa directives that the strict gate relies on, which the file's own comment already prescribes for strict-gate rulesTests: 60 new tests across
tests/test_litellm/llms/minimax/{videos,image_generation}andtest_minimax_cost_calculator.pycover bearer auth and the missing-key error, URL building for both API families, the full param-mapping matrix including the drop rules, the mutual-exclusion and lone-last-frame errors, the data-URI coercion of file inputs, both create body shapes, id encode/decode roundtrips carrying the provider and model, both status maps parametrized, failure surfacing with error codes, the v2 billed-seconds usage, the v1 two-hop file retrieve with auth header propagation against fixture clients, registry dispatch for both configs, and image/video pricing resolved from the real cost mapFollow-up commit: image generation now maps the platform's
image_urlparam onto MiniMaxsubject_reference([{type: character, image_file: url}]) so character references reach image-01; an explicitly passedsubject_referencewins overimage_url. Covered by two new transformation testsQA runbook
MINIMAX_API_KEY=<key> LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config <config declaring minimax/image-01, minimax/MiniMax-H3, minimax/MiniMax-Hailuo-2.3-Fast> --detailed_debug/v1/images/generationswithmodel=minimax-image-01, any prompt andsize=1280x720; confirm a JPEG URL comes back and the response carriesx-litellm-response-cost: 0.0035/v1/videoswithmodel=minimax-hailuo-2.3-fast, a prompt,seconds: 6and aninput_referenceimage URL; confirmstatus: queued,x-litellm-response-cost: 0.192and an opaquevideo_...id/v1/videos/{id}every 15s untilcompleted(about 2 minutes), then GET/v1/videos/{id}/contentandffprobethe bytes; expect 768P h264 MP4 of about 6smodel=minimax-h3andseconds: 4to exercise the v2 API ($0.52)Final Attestation
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is enabled.