✨ [llm][npu][quant] Add W8A8 MXFP8 quantization support for Qwen3 Dense on Ascend NPU

[TallMessiWu]

Summary

This PR adds W8A8 MXFP8 (Microscaling FP8) quantization support for Qwen3 / Qwen3.5 dense LLM models on Ascend NPU. It closes part of the NPU quantization gap tracked in issue #21584.

Prerequisites (both merged into main): #20922 (Diffusion MXFP8) | #22338 (Diffusion MXFP4). After the latest sync with upstream main, this PR no longer touches any multimodal_gen/... files — all diffusion changes are accepted from main.

Hardware requirement: Ascend A5 series or newer. npu_dynamic_mx_quant is not available on A2 / A3.

Two modes are supported:

Online quantization (--quantization mxfp8)

• Reuses the existing Fp8Config path (triggered by --quantization mxfp8, use_mxfp8=True).
• Adds an NPU bypass in Fp8Config.get_min_capability() (returns 0, skipping CUDA capability checks that are meaningless on NPU) — kept alongside the new upstream _is_musa branch.
• Adds an NPU dispatch in Fp8Config.get_quant_method() → NPUMXFP8LinearMethod (new class in srt/hardware_backend/npu/quantization/linear_method_npu.py).
• At load time, FP16 / BF16 weights are quantized online to MXFP8 via npu_dynamic_mx_quant and pre-transposed to [in, out] layout. At inference, activations are quantized per-token and the matmul is executed by npu_quant_matmul with group_sizes=[1, 1, 32] (block_size = 32).

Offline quantization (msmodelslim pre-quantized weights)

• Adds ModelSlimMXFP8Scheme (srt/layers/quantization/modelslim/schemes/modelslim_mxfp8.py) for loading weights pre-quantized by msmodelslim (float8_e4m3fn weights + uint8 scale in float8_e8m0fnu encoding).
• Registered in ModelSlimConfig.get_linear_scheme() as ("W8A8_MXFP8", ModelSlimMXFP8Scheme) (table-driven dispatch introduced upstream); the scheme's __init__ accepts (and ignores) the standard quant_config / prefix kwargs to match the dispatch signature.

Bug fix

• srt/layers/rotary_embedding/base.py: wrap fused_rope_qk_mqa import in try/except, falling back to None if the kernel is absent. Without this, a missing kernel in sgl_kernel_npu causes the whole module import to fail; ModelRegistry then silently skips the model and falls back to HF Transformers without quantization awareness, producing garbled output with FP8 weights interpreted as BF16.

Key NPU APIs used

API	Purpose
torch_npu.npu_dynamic_mx_quant(x, dst_type=torch_npu.float8_e4m3fn)	Dynamic MXFP8 quantization of activations / weights
torch_npu.npu_quant_matmul(..., group_sizes=[1, 1, 32])	MXFP8 quantized matmul (block_size = 32)
torch_npu.float8_e4m3fn / torch_npu.float8_e8m0fnu	FP8 weight dtype / scale factor dtype

Files Changed

File	Change
srt/hardware_backend/npu/quantization/linear_method_npu.py	Add NPUMXFP8LinearMethod (online MXFP8 weight quantization + inference)
srt/layers/quantization/fp8.py	NPU bypass in get_min_capability(); dispatch to NPUMXFP8LinearMethod on NPU + use_mxfp8=True path
srt/layers/quantization/modelslim/modelslim.py	Register ("W8A8_MXFP8", ModelSlimMXFP8Scheme) in get_linear_scheme()
srt/layers/quantization/modelslim/schemes/__init__.py	Export ModelSlimMXFP8Scheme; # isort: off block keeps the base-class import first to avoid circular dependency
srt/layers/quantization/modelslim/schemes/modelslim_mxfp8.py	New — offline MXFP8 (ModelSlimMXFP8Scheme) for msmodelslim pre-quantized weights
srt/layers/rotary_embedding/base.py	Guard fused_rope_qk_mqa import + None fallback

Final diff against upstream/main: 6 files, +245 / −2.

Implementation Notes

.contiguous() on transpose: online vs offline

The two paths differ intentionally:

• Online (NPUMXFP8LinearMethod): calls .contiguous() after transpose. Safe because the quantized weight is freshly allocated by npu_dynamic_mx_quant; there are no pre-existing block-scale mappings tied to the original memory layout.
• Offline (ModelSlimMXFP8Scheme): does not call .contiguous() after transpose, using .data assignment to preserve the non-contiguous transpose view. npu_quant_matmul reads strides correctly; calling .contiguous() would physically reorder the pre-quantized weight data and break the block-scale mapping, producing garbled output.

This pattern matches the vllm-ascend reference implementation.

Performance Comparison Report

Performance numbers are not yet available. This section will be filled in once benchmark runs on Ascend hardware are complete.

Metric	Baseline	Offline (ModelSlim MXFP8)	Online (--quantization mxfp8)
E2E Latency	TBD	TBD	TBD

Related Issues

Closes part of #21584 (MXFP8 / MXFP4 support on Ascend NPU for Qwen3 dense LLM).

Builds on the NPU quantization infrastructure introduced by the now-merged #20922 (Diffusion MXFP8) and #22338 (Diffusion MXFP4).

---

CI States

Latest PR Test (Base): ❌ Run #26176245090
Latest PR Test (Extra): ❌ Run #26176244908

[gemini-code-assist]

Code Review

This pull request adds MXFP8 quantization support for Diffusion and LLM models on Ascend NPU, implementing new schemes like ModelSlimMXFP8Scheme and NPUMXFP8LinearMethod, and introducing a --quantization CLI flag. It also refactors the wan_repack tool and improves modelslim configuration discovery. Feedback suggests optimizing the multimodal implementation by pre-transposing weights and scales during loading to reduce forward-pass overhead and using robust fallback definitions for the float8_e8m0fnu data type to ensure environment compatibility.