[MoE Refactor] Migrate MoeWNA16Method quantization method over to using the new MK oracle scheme. - #44120
Merged
Conversation
…ion oracle Converts the WNA16 Marlin MoE method to the new oracle-based kernel abstraction pattern, matching the structure of CompressedTensorsW8A8Fp8MoEMethod. Key changes: - Add oracle/wna16.py with select_wna16_moe_backend(), make_wna16_moe_quant_config(), and make_wna16_moe_kernel() following the same pattern as oracle/fp8.py - Add experts/trtllm_mxint4_moe.py with TrtLlmMxint4ExpertsMonolithic wrapping flashinfer_trtllm_mxint4_moe() as a FusedMoEExpertsMonolithic subclass - Update CompressedTensorsWNA16MarlinMoEMethod.__init__ to call select_wna16_moe_backend() instead of inline backend selection - Update process_weights_after_loading to build moe_quant_config and moe_kernel for 4-bit paths (both Flashinfer and Marlin); non-4-bit Marlin retains the direct fused_marlin_moe() fallback in apply() - Remove GPTQMarlinState enum and select_gemm_impl() override (base class raises ValueError; moe_kernel is set so the old path is never reached for 4-bit) - apply() and apply_monolithic() delegate to moe_kernel when available Co-authored-by: Claude https://claude.ai/code/session_01FLcv14ytXvwBUCsKZqpze1 Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Bill Nell <bnell@redhat.com>
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
mgoin
approved these changes
Jul 22, 2026
1 task
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This PR is an fixed + updated version of #42647
Migrate MoeWNA16Method quantization method over to using the new MK oracle scheme.
Additional fixes
Also fixes broken quantization import when humming module is not installed. This was broken prior to #42647
cc @bedeks @yzong-rh
Test Plan
Test Result
pytest tests pass.
lm-eval results:
Qwen baseline
Qwen PR
Mixtral baseline
Mixtral PR
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.