Skip to content

adapt to minimax_m2 #5624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wangxiyuan merged 4 commits into from
Jan 10, 2026
Merged

adapt to minimax_m2 #5624

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
097ddea
adapt to minimax_m2
Feng-xiaosuo Jan 6, 2026
2e2698f
Merge branch 'main' into main
Feng-xiaosuo Jan 6, 2026
dd37063
fix: correct lint error
Feng-xiaosuo Jan 7, 2026
866dfa6
fix: correct lint error
Feng-xiaosuo Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions vllm_ascend/quantization/quant_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
from .utils import get_quant_method



@register_quantization_config(ASCEND_QUANTIZATION_METHOD)
class AscendQuantConfig(QuantizationConfig):
"""Config class for Ascend
Expand Down Expand Up @@ -117,6 +118,18 @@ def get_quant_method(self, layer: torch.nn.Module,
prefix: str) -> Optional["QuantizeMethodBase"]:
vllm_config = get_current_vllm_config()
model_type = vllm_config.model_config.hf_config.model_type

if model_type in ["minimax", "minimax_m2"]:
prefix = prefix.replace("mlp", "block_sparse_moe")

#To adapt to minimax, modify the prefix of the model layer name
parts = prefix.split('.')
if "experts" in parts and len(parts) > 2:
exp_idx = parts.index("experts")
if exp_idx + 1 < len(parts) and parts[exp_idx + 1].isdigit():
parts = parts[:exp_idx + 1]

@gemini-code-assist gemini-code-assist Bot Jan 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This line appears to have an off-by-one error in the slice index. By slicing up to exp_idx + 1, the expert index is stripped from the prefix. For example, ...experts.1.w1 becomes ...experts. The downstream logic then incorrectly assumes expert 0 due to the hardcoded packed_modules_model_mapping.

This will cause incorrect quantization behavior for any expert with an index other than 0. This is a critical bug for models with multiple experts.

A more robust solution is to slice up to exp_idx + 2 to preserve the expert index in the prefix (e.g., ...experts.1). This would mean the quantization configuration should be defined per-expert (e.g., ...experts.1.weight), which is a cleaner design. This change would also make the experts entry in packed_modules_model_mapping for minimax_m2 unnecessary.

Suggested change
parts = parts[:exp_idx + 1]
parts = parts[:exp_idx + 2]

prefix = ".".join(parts)

if model_type in packed_modules_model_mapping:
self.packed_modules_mapping = packed_modules_model_mapping[
model_type]
Expand Down Expand Up @@ -304,6 +317,14 @@ def get_scaled_act_names(self) -> List[str]:
["experts.0.gate_proj", "experts.0.up_proj", "experts.0.down_proj"],
"fused_qkv_a_proj": ["q_a_proj", "kv_a_proj_with_mqa"]
},
"minimax_m2": {
"qkv_proj": [
"q_proj",
"k_proj",
"v_proj",
],
"experts": ["experts.0.w1", "experts.0.w2", "experts.0.w3"]
}
}


Expand Down
Loading