[ModelOpt] Unify MoE methods onto QuantSpec + ModelOptMoEMethod - #55997
Open
ima-helikoptaaa wants to merge 6 commits into
Open
ima-helikoptaaa wants to merge 6 commits into
ima-helikoptaaa wants to merge 6 commits into
Conversation
Unify homogeneous and mixed-precision MoE construction behind one helper, matching build_linear_method, and drop the unused FusedMoEMethodCls pointer. Co-authored-by: Cursor <cursoragent@cursor.com>
…thod. FP8-family MoE now goes through resolve() like linears; NVFP4 and MXFP8 stay on their per-format classes. Co-authored-by: Cursor <cursoragent@cursor.com>
Route NVFP4 / W4A16 through resolve() like FP8 and delete ModelOptNvFp4FusedMoE. MXFP8 stays on its own class. Co-authored-by: Cursor <cursoragent@cursor.com>
All ModelOpt MoE algos now go through resolve() and one method class; delete ModelOptMxFp8FusedMoE. Co-authored-by: Cursor <cursoragent@cursor.com>
Move FP8, NVFP4, and MXFP8 expert create/process/quant_config onto per-weight-key schemes so ModelOptMoEMethod stays a thin dispatcher. Refuse linear-only PcPt/PB_WO at the factory and cover mixed MoE dispatch. Co-authored-by: Cursor <cursoragent@cursor.com>
After rebasing onto vllm-project#55513, mixed FP8_PB_WO experts are vLLM block-FP8, not a skip. Only PcPt mixed MoE stays unquantized. Co-authored-by: Cursor <cursoragent@cursor.com>
ima-helikoptaaa
requested review from
AndreasKaratzas,
WoosukKwon,
mgoin,
pavanimajety,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256 and
zyongye
as code owners
September 9, 2026 03:37
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
Documentation preview: https://vllm--55997.org.readthedocs.build/en/55997/ |
mergify
Bot
added
documentation
1 task
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Unify ModelOpt MoE the same way #49381 unified linear methods.
One factory (
build_moe_method) and one method class (ModelOptMoEMethod).resolve()still builds the QuantSpec. MoE looks up a weight-key scheme inMOE_SCHEME_FOR(not linearSCHEME_FOR).Folds
ModelOptFp8MoEMethod,ModelOptNvFp4FusedMoE, andModelOptMxFp8FusedMoEintoModelOptMoEMethod. Config classes stay split.Deepseek V4 NVFP4 experts go through the same factory.
Mixed block-FP8 MoE (
FP8_PB_WO/FP8_BLOCK_SCALES) continues to useFp8MoEMethod.Related: #55339
Test Plan
Unit:
pytest tests/quantization/test_modelopt.py -k "not checkpoint_setup"Parity, same method as #49381 (
VLLM_BATCH_INVARIANT=1,--enforce-eager,old vs new weight/kernel hash and prefill logit delta) on every ModelOpt MoE
format this PR serves:
Also:
moe_quant_algo=NVFP4)GSM8K settings match #49381: 1319 Q, 5-shot, T=0, 512 tok.
Test Result
Unit tests added or updated in
tests/quantization/test_modelopt.py.Parity hashes, logit deltas, and GSM8K for the four MoE formats are still
pending. Results will be filled into the table above.
(Optional) Documentation
No new model. Existing design docs now point at
ModelOptMoEMethod.Not duplicating existing work
This is the MoE half of the ModelOpt QuantKey rewrite. #49381 already landed
the linear side and left MoE on the old per-format classes. This PR replaces
those classes. Mixed block-FP8 MoE from #55513 stays on
Fp8MoEMethod.AI assistance
AI assistance (Cursor) was used to draft and edit the change. I reviewed
every changed line and own the behavior. This is not a pure-agent PR.