[Refactor][Ops] Refactor MoE mlp and quantization methods by earthmanylf · Pull Request #13773 · vllm-project/vllm-ascend

earthmanylf · 2026-08-07T07:02:53Z

What this PR does / why we need it?

Come from #13318 and #13220
Right now, moe_mlp has too many branches about different quantization method and different activations. In this PR, we split moe_mlp's methods into differnet quantization method.

Does this PR introduce any user-facing change?

No. This is an internal refactor of the fused MoE runtime contract and its test coverage.

How was this patch tested?

The following models are tested and can be curled successfully:
A3:
float: qwen30b-a3b
w8a8: qwen30b-a3b-w8a8
w4a8: kimi2.6-w4a8
megamoe, dynamic eplb: DeepSeek-V4-Flash-w8a8-mtp
A5:
w8a8mxfp: step3.7-w8a8mxfp
w4a8mxfp: qwen3-30b-w4a8mxfp
w4a4mxfp: glm5.1-w4a4mxfp

vLLM main: vllm-project/vllm@ba07e4a

gemini-code-assist · 2026-08-07T07:03:03Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the fused MoE MLP runtime to improve maintainability and extensibility. By abstracting the activation and quantization logic into separate components, the codebase removes redundant branching and allows for cleaner support of diverse quantization methods. This change is internal and does not impact user-facing APIs.

Highlights

Refactoring MoE MLP Logic: Decoupled MoE MLP operations from specific quantization methods by introducing a modular activation strategy, reducing branch complexity in the runtime.
New Activation Abstraction: Added vllm_ascend/ops/fused_moe/moe_activation.py to handle MLP application logic, allowing for cleaner integration of different quantization schemes.
Quantization Support: Implemented _quant_hidden_states in the base quantization class and added specific GMM (Grouped MatMul) application methods to w8a8_dynamic.py to support flexible activation and weight handling.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-08-07T07:03:59Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-08-07T07:04:07Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist

Code Review

Suggested PR Title:

[Ops][Feature] Refactor MoE activation and quantization methods for Ascend

Suggested PR Summary:

### What this PR does / why we need it?
This pull request refactors the MoE (Mixture of Experts) activation handling by introducing a modular `MoeMlpActivation` class and `get_moe_activation_method` helper. It also adds dynamic quantization helper methods (`_quant_hidden_states`, `apply_gmm1`, `apply_gmm1_act_quant`, and `apply_gmm2`) to the Ascend quantization schemes.

However, several critical issues were identified in the implementation:
- In `moe_comm_method.py`, `MoEFusedExpertsInput` (the class) is incorrectly used instead of the instance `fused_experts_input` to access the activation attribute.
- In `w8a8_dynamic.py`, helper functions `_require_single_tensor_for_swiglu_quant` and `cumsum_group_list` are used but not imported.
- Multiple bugs exist in `apply_gmm1`, `apply_gmm1_act_quant`, and `apply_gmm2` within `w8a8_dynamic.py`, including undefined variables (`hidden_states`), typos (`hidden_statesm`), missing return statements, and incorrect attribute access (`weight` instead of `weights`).
- The return type annotation for `apply_mlp` in `moe_activation.py` is incorrect as it returns a tuple instead of a single `Tensor`.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
No testing details were provided. It is highly recommended to fix the runtime NameErrors and attribute errors before testing.

gemini-code-assist · 2026-08-07T07:04:33Z

            token_dispatch_output=token_dispatch_output,
            use_fusion_ops=self.use_fusion_ops,
        )
+        act_method = get_moe_activation_method(MoEFusedExpertsInput.activation)


MoEFusedExpertsInput is the class name, not the instance. Accessing MoEFusedExpertsInput.activation will fail at runtime. It should be changed to fused_experts_input.activation.

Suggested change

act_method = get_moe_activation_method(MoEFusedExpertsInput.activation)

act_method = get_moe_activation_method(fused_experts_input.activation)

gemini-code-assist · 2026-08-07T07:04:33Z

+from vllm_ascend.device.device_op import DeviceOperator
 from vllm_ascend.distributed.parallel_state import get_mc2_group
-from vllm_ascend.ops.fused_moe.moe_runtime_args import build_fused_experts_input
+from vllm_ascend.ops.fused_moe.moe_runtime_args import MoEMlpComputeInput, build_fused_experts_input


The helper functions _require_single_tensor_for_swiglu_quant and cumsum_group_list are used in apply_gmm1_act_quant but are not imported, which will cause a NameError at runtime. They should be imported from vllm_ascend.ops.fused_moe.moe_mlp.

Suggested change

from vllm_ascend.device.device_op import DeviceOperator

from vllm_ascend.distributed.parallel_state import get_mc2_group

from vllm_ascend.ops.fused_moe.moe_runtime_args import build_fused_experts_input

from vllm_ascend.ops.fused_moe.moe_runtime_args import MoEMlpComputeInput, build_fused_experts_input

from vllm_ascend.device.device_op import DeviceOperator

from vllm_ascend.distributed.parallel_state import get_mc2_group

from vllm_ascend.ops.fused_moe.moe_mlp import (

_require_single_tensor_for_swiglu_quant,

cumsum_group_list,

)

from vllm_ascend.ops.fused_moe.moe_runtime_args import MoEMlpComputeInput, build_fused_experts_input

gemini-code-assist · 2026-08-07T07:04:33Z

+    def apply_gmm1(self, mlp_compute_input: MoEMlpComputeInput):
+        hidden_states = torch_npu.npu_grouped_matmul(
+            x=[hidden_states],
+            weight=mlp_compute_input.weight.w1
+            if isinstance(mlp_compute_input.weight.w1, list)
+            else [mlp_compute_input.weight.w1],
+            scale=[mlp_compute_input.weight.w1_scale[0].to(mlp_compute_input.weight.w2_scale[0].dtype)]
+            if isinstance(mlp_compute_input.weight.w1_scale, list)
+            else [mlp_compute_input.weight.w1_scale],
+            bias=None,
+            per_token_scale=[mlp_compute_input.weight.pertoken_scale],
+            split_item=2,
+            group_type=0,
+            group_list=mlp_compute_input.group_list,
+            output_dtype=torch.bfloat16,
+        )[0]


The apply_gmm1 method has several critical issues:

hidden_states is referenced in x=[hidden_states] before it is defined. It should be mlp_compute_input.hidden_states.

mlp_compute_input.weight is used instead of mlp_compute_input.weights.

mlp_compute_input.weight.pertoken_scale is used, but MoEMlpComputeInput does not have a weight attribute or a pertoken_scale attribute under weights. It should use mlp_compute_input.dynamic_scale.

The method does not return hidden_states.

Suggested change

def apply_gmm1(self, mlp_compute_input: MoEMlpComputeInput):

hidden_states = torch_npu.npu_grouped_matmul(

x=[hidden_states],

weight=mlp_compute_input.weight.w1

if isinstance(mlp_compute_input.weight.w1, list)

else [mlp_compute_input.weight.w1],

scale=[mlp_compute_input.weight.w1_scale[0].to(mlp_compute_input.weight.w2_scale[0].dtype)]

if isinstance(mlp_compute_input.weight.w1_scale, list)

else [mlp_compute_input.weight.w1_scale],

bias=None,

per_token_scale=[mlp_compute_input.weight.pertoken_scale],

split_item=2,

group_type=0,

group_list=mlp_compute_input.group_list,

output_dtype=torch.bfloat16,

)[0]

def apply_gmm1(self, mlp_compute_input: MoEMlpComputeInput) -> torch.Tensor:

hidden_states = torch_npu.npu_grouped_matmul(

x=[mlp_compute_input.hidden_states],

weight=mlp_compute_input.weights.w1

if isinstance(mlp_compute_input.weights.w1, list)

else [mlp_compute_input.weights.w1],

scale=[mlp_compute_input.weights.w1_scale[0].to(mlp_compute_input.weights.w2_scale[0].dtype)]

if isinstance(mlp_compute_input.weights.w1_scale, list)

else [mlp_compute_input.weights.w1_scale],

bias=None,

per_token_scale=[mlp_compute_input.dynamic_scale],

split_item=2,

group_type=0,

group_list=mlp_compute_input.group_list,

output_dtype=torch.bfloat16,

)[0]

return hidden_states

gemini-code-assist · 2026-08-07T07:04:33Z

+    def apply_gmm1_act_quant(self, mlp_compute_input: MoEMlpComputeInput):
+        hidden_states, pertoken_scale = self._quant_hidden_states(
+            mlp_compute_input.hidden_statesm, mlp_compute_input.dynamic_scale
+        )
+        hidden_states, swiglu_out_scale, _ = DeviceOperator.npu_grouped_matmul_swiglu_quant(
+            x=hidden_states,
+            weight=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weight.w1, name="w1"),
+            group_list=cumsum_group_list(mlp_compute_input.group_list, mlp_compute_input.group_list_type, 0),
+            weight_scale=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weight.w1_scale, name="w1_scale"),
+            x_scale=pertoken_scale,
+            bias=None,
+            use_mxfp_quant=False,
+            act_quant_type=torch.int8,
+            swiglu_limit=mlp_compute_input.swiglu_limit,
+        )
+
+        return hidden_states, swiglu_out_scale


This method has two critical bugs:

mlp_compute_input.hidden_statesm has a typo and should be mlp_compute_input.hidden_states.

mlp_compute_input.weight is used instead of mlp_compute_input.weights.

Suggested change

def apply_gmm1_act_quant(self, mlp_compute_input: MoEMlpComputeInput):

hidden_states, pertoken_scale = self._quant_hidden_states(

mlp_compute_input.hidden_statesm, mlp_compute_input.dynamic_scale

)

hidden_states, swiglu_out_scale, _ = DeviceOperator.npu_grouped_matmul_swiglu_quant(

x=hidden_states,

weight=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weight.w1, name="w1"),

group_list=cumsum_group_list(mlp_compute_input.group_list, mlp_compute_input.group_list_type, 0),

weight_scale=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weight.w1_scale, name="w1_scale"),

x_scale=pertoken_scale,

bias=None,

use_mxfp_quant=False,

act_quant_type=torch.int8,

swiglu_limit=mlp_compute_input.swiglu_limit,

)

return hidden_states, swiglu_out_scale

def apply_gmm1_act_quant(self, mlp_compute_input: MoEMlpComputeInput):

hidden_states, pertoken_scale = self._quant_hidden_states(

mlp_compute_input.hidden_states, mlp_compute_input.dynamic_scale

)

hidden_states, swiglu_out_scale, _ = DeviceOperator.npu_grouped_matmul_swiglu_quant(

x=hidden_states,

weight=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weights.w1, name="w1"),

group_list=cumsum_group_list(mlp_compute_input.group_list, mlp_compute_input.group_list_type, 0),

weight_scale=_require_single_tensor_for_swiglu_quant(mlp_compute_input.weights.w1_scale, name="w1_scale"),

x_scale=pertoken_scale,

bias=None,

use_mxfp_quant=False,

act_quant_type=torch.int8,

swiglu_limit=mlp_compute_input.swiglu_limit,

)

return hidden_states, swiglu_out_scale

gemini-code-assist · 2026-08-07T07:04:33Z

+    def apply_gmm2(self, mlp_compute_input: MoEMlpComputeInput, hidden_states, act_out_scale):
+        hidden_states = DeviceOperator.npu_grouped_matmul_gmm2(
+            hidden_states=hidden_states,
+            weight=mlp_compute_input.weight.w2,
+            weight_scale=mlp_compute_input.weight.w2_scale,
+            per_token_scale=act_out_scale,
+            group_list=mlp_compute_input.group_list,
+            group_list_type=mlp_compute_input.group_list_type,
+            input_dtype=None,
+            act_quant_type=torch.int8,
+            weight_quant_type=None,
+            scale_type=None,
+            per_token_scale_type=None,
+            use_bf16=True,
+            use_mxfp_quant=False,
+            bias=None,
+            fallback_output_dtype=mlp_compute_input.weight.w2_scale[0].dtype
+            if isinstance(mlp_compute_input.weight.w2_scale, list)
+            else mlp_compute_input.weight.w2_scale.dtype,
+        )
+        return hidden_states


mlp_compute_input.weight is used instead of mlp_compute_input.weights.

def apply_gmm2(self, mlp_compute_input: MoEMlpComputeInput, hidden_states, act_out_scale): hidden_states = DeviceOperator.npu_grouped_matmul_gmm2( hidden_states=hidden_states, weight=mlp_compute_input.weights.w2, weight_scale=mlp_compute_input.weights.w2_scale, per_token_scale=act_out_scale, group_list=mlp_compute_input.group_list, group_list_type=mlp_compute_input.group_list_type, input_dtype=None, act_quant_type=torch.int8, weight_quant_type=None, scale_type=None, per_token_scale_type=None, use_bf16=True, use_mxfp_quant=False, bias=None, fallback_output_dtype=mlp_compute_input.weights.w2_scale[0].dtype if isinstance(mlp_compute_input.weights.w2_scale, list) else mlp_compute_input.weights.w2_scale.dtype, ) return hidden_states

gemini-code-assist · 2026-08-07T07:04:33Z

+
+class MoeMlpActivation:
+    @staticmethod
+    def apply_mlp(mlp_compute_input: MoEMlpComputeInput, quant_method) -> torch.Tensor:


The return type annotation -> torch.Tensor is incorrect because the method returns a tuple of (hidden_states, before_gmm2_evt). It should be updated to tuple[torch.Tensor, torch.npu.Event].

Suggested change

def apply_mlp(mlp_compute_input: MoEMlpComputeInput, quant_method) -> torch.Tensor:

@staticmethod

def apply_mlp(mlp_compute_input: MoEMlpComputeInput, quant_method) -> tuple[torch.Tensor, torch.npu.Event]:

github-actions · 2026-08-08T06:57:52Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

zhangc10 · 2026-08-10T09:22:15Z

+    )
+
+
+def _gmm_swiglu_quant_fusion_enabled(use_mxfp_quant, fusion, dynamic_eplb, activation=None):


This function can be splitted. Only fusion and dynamic_eplb is useful. use_mxfp_quant and activation is useless after the refactoring.

zhangc10 · 2026-08-10T09:23:18Z

    return int(num_experts - global_redundant_expert_num - num_shared_experts)
+
+
+def _custom_gmm_swiglu_enabled(fusion, dynamic_eplb, activation=None):


This function can be splitted. Activation is useless after the refactoring.

zhangc10 · 2026-08-10T10:55:11Z

+
+    # ---- MLP gmm hooks (activation orchestrated by MoeActionMethod) ----
+
+    def get_moe_weights(self, layer):


This function seems not consistent with the original code in AscendUnquantizedFusedMoEMethod's apply method

zhangc10 · 2026-08-10T11:03:31Z

        )
-        return final_hidden_states
+
+    def _get_mlp_weights(self, layer):


_get_mlp_weights processes both w1 and w2, but only one of them is used. Either in apply_gmm1 or apply_gmm2. Why not split it into two functions, one for w1, another for w2.

zhangc10 · 2026-08-10T11:15:10Z

+        w1_bias = layer.w13_bias if self.moe.has_bias else None
+        gate_up_out = torch_npu.npu_grouped_matmul(
+            x=[hidden_states],
+            weight=w1 if isinstance(w1, list) else [w1],


In the original unquant_apply_mlp, w1 is not a list. Maybe it's redundant to add the isinstance(w1, list)

zhangc10 · 2026-08-11T02:21:03Z

+                QuantType.W8A8MXFP,
+                QuantType.W4A4MXFP,
+                QuantType.W4A8MXFP,
+                QuantType.W4A16MXFP,


W4A16MXFP is not needed here.

zhangc10 · 2026-08-11T02:28:41Z

+
+    def apply_gmm1(self, mlp_compute_input: MoEMlpComputeInput):
+        # A16: activations stay unquantized; gmm1 dequantizes with antiquant.
+        hidden_states, _ = self._quant_hidden_states(mlp_compute_input.hidden_states, mlp_compute_input.dynamic_scale)


For specific quant types (e.g., here W4A16), it's better not to use the general _quant_hidden_states method. As you know for sure, hidden_states stay unquantized.

zhangc10 · 2026-08-11T02:42:15Z

    supports_eplb = False
    quant_type: QuantType = QuantType.W4A16MXFP
+    act_quant_type: torch.dtype | None = None
+    fused_activations = frozenset({"silu"})


Consider an empty set for W4A16_MXFP4 (i.e., fused_activations = frozenset()). It has a similar logic as W4A16, the the following apply_gmm1_act_quant is not needed.

zhangc10 · 2026-08-11T03:16:15Z

+            output_dtype = w2_scale[0].dtype if isinstance(w2_scale, list) else w2_scale.dtype
+        return group_list, group_list_type, output_dtype
+
+    def get_moe_weights(self, layer):


It's better to use a more clear method name. get_moe_weights is a little confused. B.T.W, the final return branch seems for the case of non-fused mc2 (may be redundant)

zhangc10 · 2026-08-11T03:20:55Z

+        activation = mlp_compute_input.activation
+        act_name = getattr(activation, "value", activation)
+
+        if self.is_per_channel_weight and enable_custom_op() and act_name != "swigluoai_uninterleave":


It seems that act_name != "swigluoai_uninterleave" is not needed here.

zhangc10 · 2026-08-11T03:24:37Z

+                group_list_type=group_list_type,
+                swiglu_limit=mlp_compute_input.swiglu_limit,
+            )
+        elif _custom_gmm_swiglu_enabled(mlp_compute_input.fusion, mlp_compute_input.dynamic_eplb, activation):


This branch is not needed for W4A8. In the original code, when use_w4a8_per_channel_gmm_swiglu is True, it only enter the first branch, which calls torch.ops._C_ascend.grouped_matmul_swiglu_quant_v2.

zhangc10 · 2026-08-11T03:25:47Z

+            )
+        elif _gmm_swiglu_quant_fusion_enabled(
+            False, mlp_compute_input.fusion, mlp_compute_input.dynamic_eplb, activation
+        ):


Same as above, this branch is not needed for W4A8

zhangc10 · 2026-08-11T03:32:07Z

+        dispose_tensor(mlp_compute_input.hidden_states)
+        return hidden_states, swiglu_out_scale
+
+    def _soft_gmm1(self, mlp_compute_input: MoEMlpComputeInput, hidden_states: torch.Tensor, pertoken_scale):


The method name _soft_gmm1 seems confused. What does soft mean?

zhangc10 · 2026-08-11T03:37:43Z

+            scale=w2_scale,
+            bias=bias2,
+            per_token_scale=[act_out_scale],
+            split_item=2,


Can use torch_npu.npu_grouped_matmul directly.

github-actions · 2026-08-21T14:41:51Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-08-28T01:39:20Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-09-01T09:26:24Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-09-01T09:26:25Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-09-01T15:44:05Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2026-09-03T03:44:05Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…methods - Remove moe_mlp.py; distribute quant_apply_mlp/unquant_apply_mlp into per-quant-method gmm hooks (apply_gmm1/apply_gmm1_act_quant/apply_act_quant/apply_gmm2) under quantization/methods - Introduce MoeActionMethod per activation family (silu default, gelu, gelu_tanh, swiglustep, swigluoai_uninterleave) orchestrating the MLP stage with the quant method injected at runtime - Merge the redundant MC2/non-MC2 fused gmm+swiglu+quant branches; keep the int32+npu_dequant_swiglu_quant path for swigluoai_uninterleave - Extract A5 MXFP kernel branches from device_op.py into the respective FusedMoEMethod implementations - Thread layer through MoEFusedExpertsInput/MoEMlpComputeInput so apply() no longer passes weight tensors; FUSED_MC2 builds weights via quant_method.get_fused_mc2_weights(layer) - Keep the 310P legacy path untouched - Align MoE MLP UTs with the functional orchestration and the get_fused_mc2_weights rename: drop the orphan TestSwigluOaiDynamicMxQuant, mock DeviceOperator.clipped_swiglu for swigluoai_uninterleave, and track fused_w1/w2_scale_bias in the fused-MC2 weight test Signed-off-by: earthmanylf <yulinfeng2@huawei.com>

earthmanylf · 2026-09-04T02:32:55Z

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

E2E

github-actions Bot added the merge-conflicts label Aug 7, 2026

github-actions Bot added module:ops module:quantization labels Aug 7, 2026

gemini-code-assist Bot reviewed Aug 7, 2026

View reviewed changes

earthmanylf force-pushed the refactor_moe branch from 6f22bc8 to f5ad9ea Compare August 7, 2026 07:10

github-actions Bot removed the merge-conflicts label Aug 7, 2026

earthmanylf force-pushed the refactor_moe branch from f5ad9ea to dba6efc Compare August 7, 2026 09:09

earthmanylf changed the title ~~[Refactor][Ops] moe_mlp refactor~~ [Refactor][Ops] Refactor MoE mlp and quantization methods Aug 7, 2026

github-actions Bot added the module:tests label Aug 7, 2026

earthmanylf force-pushed the refactor_moe branch from dba6efc to 1edc01c Compare August 7, 2026 09:33

github-actions Bot added the merge-conflicts label Aug 8, 2026

kunpengW-code mentioned this pull request Aug 10, 2026

[RFC]: Quantization Module Refactoring #13318

Open

zhangc10 reviewed Aug 10, 2026

View reviewed changes

zhangc10 reviewed Aug 11, 2026

View reviewed changes

github-actions Bot added the merge-conflicts label Aug 21, 2026

earthmanylf force-pushed the refactor_moe branch from f9303cd to 01697b3 Compare August 22, 2026 06:47

github-actions Bot added merge-conflicts and removed merge-conflicts labels Aug 22, 2026

earthmanylf force-pushed the refactor_moe branch from 01697b3 to c759197 Compare August 31, 2026 12:52

github-actions Bot removed the merge-conflicts label Aug 31, 2026

earthmanylf force-pushed the refactor_moe branch 4 times, most recently from e3eb9a7 to 403532c Compare September 1, 2026 03:41

github-actions Bot added the merge-conflicts label Sep 1, 2026

earthmanylf force-pushed the refactor_moe branch from 403532c to 27ef236 Compare September 1, 2026 14:46

github-actions Bot added merge-conflicts and removed merge-conflicts labels Sep 1, 2026

earthmanylf force-pushed the refactor_moe branch from 27ef236 to f11e5c4 Compare September 3, 2026 03:19

github-actions Bot added merge-conflicts and removed merge-conflicts labels Sep 3, 2026

earthmanylf force-pushed the refactor_moe branch from f11e5c4 to ff41910 Compare September 3, 2026 13:20

kunpengW-code approved these changes Sep 4, 2026

View reviewed changes

This was referenced Sep 5, 2026

[Bugfix][MoE] Fix w8a8fp8_dynamic moe quant path #15853

Merged

[Revert] Revert refactor MoE mlp and quantization methods #15862

Closed

sweetpig97 mentioned this pull request Sep 10, 2026

[Refactor][310P] Refactor MoE mlp and quantization methods For 310P #14496

Merged

	act_method = get_moe_activation_method(MoEFusedExpertsInput.activation)
	act_method = get_moe_activation_method(fused_experts_input.activation)

	def apply_mlp(mlp_compute_input: MoEMlpComputeInput, quant_method) -> torch.Tensor:
	@staticmethod
	def apply_mlp(mlp_compute_input: MoEMlpComputeInput, quant_method) -> tuple[torch.Tensor, torch.npu.Event]:

		)


		def _gmm_swiglu_quant_fusion_enabled(use_mxfp_quant, fusion, dynamic_eplb, activation=None):

		return int(num_experts - global_redundant_expert_num - num_shared_experts)


		def _custom_gmm_swiglu_enabled(fusion, dynamic_eplb, activation=None):


		# ---- MLP gmm hooks (activation orchestrated by MoeActionMethod) ----

		def get_moe_weights(self, layer):

Conversation

earthmanylf commented Aug 7, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

gemini-code-assist Bot commented Aug 7, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

earthmanylf commented Aug 7, 2026 •

edited by github-actions Bot

Loading

earthmanylf commented Sep 4, 2026 •

edited by vllm-ascend-ci

Loading