Add MiniMax-M3 modelopt nvfp4 support - #46380
Merged
Merged
Conversation
Signed-off-by: Xin Li <xinli@nvidia.com>
xinli-sw
requested review from
mgoin,
pavanimajety,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256 and
zyongye
as code owners
June 22, 2026 15:00
xinli-sw
marked this pull request as draft
June 22, 2026 15:01
xinli-sw
marked this pull request as ready for review
June 24, 2026 05:25
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
@xinli-sw The NVFP4 ckpt is released at https://huggingface.co/nvidia/MiniMax-M3-NVFP4 |
Contributor
|
@mgoin @pavanimajety Please help review this one. |
Contributor
Author
|
this currently is rebased on vllm-project:minimax-m3-perf , we could target main instead or wait a bit until we merge that branch to main in a few days |
|
I am getting this FROM vllm/vllm-openai:nightly
WORKDIR /app
# Install git + CrowdStrike deps, then CentML vLLM fork
RUN apt-get update && \
dpkg --configure -a && \
apt-get -f install -y && \
apt-get install -y git curl sudo libnl-3-200 libnl-genl-3-200 && \
rm -rf /var/lib/apt/lists/*
RUN pip install setuptools_rust && \
VLLM_USE_PRECOMPILED=1 pip install "git+https://github.com/CentML/vllm.git@minimax_m3_nvfp4"
COPY start_with_ray.sh /app/start_with_ray.sh
ENV PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
ENV FLASHINFER_DISABLE_VERSION_CHECK=1
EXPOSE 8000
EXPOSE 8265
ENTRYPOINT ["/app/start_with_ray.sh"]
# Overriden during runtime with the arguments mentioned in https://huggingface.co/nvidia/MiniMax-M3-NVFP4
CMD ["models/qwen-a3b", "--served-model-name", "qwen-a3b", "--max-model-len", "200000", "--dtype", "auto", "--gpu-memory-utilization", "0.95"]
|
khluu
added a commit
to vllm-project/perf-eval
that referenced
this pull request
Sep 1, 2026
MiniMax-M3 NVFP4 support merged into vLLM in June (vllm-project/vllm#46380) and Kimi K3 is registered in vLLM main (KimiK3ForConditionalGeneration, including the dspark_mla draft), so current nightlies serve both models. These three workloads were the only ones in the repo setting a dedicated vllm.image (and, briefly, pin_image). Every other workload omits image and defaults to the nightly under test. Drop the image field from minimax_m3_b200, minimax_m3_h200, and kimi_k3_mi355x so they follow the same nightly-default convention; the VLLM_IMAGE / VLLM_COMMIT override applies as it does for all other workloads. Keep the pin_image resolver mechanism as an escape hatch, but update the README so it no longer cites kimi-k3 / minimax-m3 as models that need it. This commit was authored with assistance from Kimi Code CLI. Co-Authored-By: Kimi <noreply@moonshot.cn>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.