[qianlihuang]

Purpose

• Add use_ue8m0 support to silu_and_mul_per_block_quant.

In addition to #42497 This patch adds the missing flag so the fused path can be used in both modes without changing scale semantics.

Test Plan

pytest tests/kernels/core/test_fused_silu_mul_block_quant.py

python3 benchmarks/fused_kernels/silu_mul_block_quant_benchmark.py --use-ue8m0

vllm serve /model/minimax-m2.7/ \
  -tp 4 \
  --tool-call-parser minimax_m2 \
  --reasoning-parser minimax_m2  \
  --enable-auto-tool-choice \
  --trust-remote-code

python3 tests/evals/gsm8k/gsm8k_eval.py

Test Result

UE8M0 scales: True

100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [03:20<00:00,  2.51s/it]

================================================================================
FINAL COMPARISON - ALL RESULTS
================================================================================
[----------------------------------------------- silu-mul-block-quant-ue8m0 ----------------------------------------------]
                                                     |  unfused_groupwise_fp8_ue8m0_impl  |  fused_groupwise_fp8_ue8m0_impl
1 threads: ----------------------------------------------------------------------------------------------------------------
      N 16 x D 1024 x DT torch.float16 x GS 64       |                66.3                |               28.4             
      N 16 x D 1024 x DT torch.float16 x GS 128      |                67.0                |               28.3             
      N 16 x D 1024 x DT torch.bfloat16 x GS 64      |                67.2                |               28.3             
      N 16 x D 1024 x DT torch.bfloat16 x GS 128     |                66.2                |               28.5             
      N 16 x D 2048 x DT torch.float16 x GS 64       |                67.3                |               28.3             
      N 16 x D 2048 x DT torch.float16 x GS 128      |                66.8                |               28.5             
      N 16 x D 2048 x DT torch.bfloat16 x GS 64      |                67.2                |               28.7             
      N 16 x D 2048 x DT torch.bfloat16 x GS 128     |                67.6                |               28.5             
      N 16 x D 4096 x DT torch.float16 x GS 64       |                65.9                |               28.4             
      N 16 x D 4096 x DT torch.float16 x GS 128      |                66.5                |               28.2             
      N 16 x D 4096 x DT torch.bfloat16 x GS 64      |                67.0                |               28.6             
      N 16 x D 4096 x DT torch.bfloat16 x GS 128     |                66.5                |               28.4             
      N 16 x D 5120 x DT torch.float16 x GS 64       |                67.0                |               28.3             
      N 16 x D 5120 x DT torch.float16 x GS 128      |                66.4                |               28.4             
      N 16 x D 5120 x DT torch.bfloat16 x GS 64      |                66.8                |               28.4             
      N 16 x D 5120 x DT torch.bfloat16 x GS 128     |                66.9                |               28.4             
      N 16 x D 14336 x DT torch.float16 x GS 64      |                66.4                |               28.6             
      N 16 x D 14336 x DT torch.float16 x GS 128     |                66.9                |               28.4             
      N 16 x D 14336 x DT torch.bfloat16 x GS 64     |                68.0                |               28.2             
      N 16 x D 14336 x DT torch.bfloat16 x GS 128    |                67.2                |               28.5             
      N 128 x D 1024 x DT torch.float16 x GS 64      |                66.2                |               28.3             
      N 128 x D 1024 x DT torch.float16 x GS 128     |                66.0                |               28.3             
      N 128 x D 1024 x DT torch.bfloat16 x GS 64     |                67.1                |               28.5             
      N 128 x D 1024 x DT torch.bfloat16 x GS 128    |                66.9                |               28.5             
      N 128 x D 2048 x DT torch.float16 x GS 64      |                66.9                |               28.2             
      N 128 x D 2048 x DT torch.float16 x GS 128     |                66.2                |               28.4             
      N 128 x D 2048 x DT torch.bfloat16 x GS 64     |                66.3                |               28.3             
      N 128 x D 2048 x DT torch.bfloat16 x GS 128    |                66.9                |               28.6             
      N 128 x D 4096 x DT torch.float16 x GS 64      |                67.0                |               28.6             
      N 128 x D 4096 x DT torch.float16 x GS 128     |                65.9                |               28.3             
      N 128 x D 4096 x DT torch.bfloat16 x GS 64     |                66.9                |               28.6             
      N 128 x D 4096 x DT torch.bfloat16 x GS 128    |                66.7                |               28.5             
      N 128 x D 5120 x DT torch.float16 x GS 64      |                66.7                |               28.3             
      N 128 x D 5120 x DT torch.float16 x GS 128     |                67.0                |               28.3             
      N 128 x D 5120 x DT torch.bfloat16 x GS 64     |                66.7                |               28.4             
      N 128 x D 5120 x DT torch.bfloat16 x GS 128    |                66.2                |               28.5             
      N 128 x D 14336 x DT torch.float16 x GS 64     |                67.1                |               28.4             
      N 128 x D 14336 x DT torch.float16 x GS 128    |                66.7                |               28.5             
      N 128 x D 14336 x DT torch.bfloat16 x GS 64    |                67.6                |               28.5             
      N 128 x D 14336 x DT torch.bfloat16 x GS 128   |                67.4                |               28.5             
      N 512 x D 1024 x DT torch.float16 x GS 64      |                66.5                |               28.7             
      N 512 x D 1024 x DT torch.float16 x GS 128     |                65.9                |               28.6             
      N 512 x D 1024 x DT torch.bfloat16 x GS 64     |                67.8                |               28.4             
      N 512 x D 1024 x DT torch.bfloat16 x GS 128    |                66.9                |               28.5             
      N 512 x D 2048 x DT torch.float16 x GS 64      |                67.3                |               28.6             
      N 512 x D 2048 x DT torch.float16 x GS 128     |                67.9                |               28.2             
      N 512 x D 2048 x DT torch.bfloat16 x GS 64     |                67.5                |               28.6             
      N 512 x D 2048 x DT torch.bfloat16 x GS 128    |                67.4                |               28.7             
      N 512 x D 4096 x DT torch.float16 x GS 64      |                67.4                |               28.5             
      N 512 x D 4096 x DT torch.float16 x GS 128     |                67.3                |               28.7             
      N 512 x D 4096 x DT torch.bfloat16 x GS 64     |                67.2                |               28.8             
      N 512 x D 4096 x DT torch.bfloat16 x GS 128    |                66.8                |               28.6             
      N 512 x D 5120 x DT torch.float16 x GS 64      |                66.9                |               28.3             
      N 512 x D 5120 x DT torch.float16 x GS 128     |                66.7                |               28.5             
      N 512 x D 5120 x DT torch.bfloat16 x GS 64     |                67.4                |               28.3             
      N 512 x D 5120 x DT torch.bfloat16 x GS 128    |                67.3                |               28.5             
      N 512 x D 14336 x DT torch.float16 x GS 64     |                95.8                |               72.7             
      N 512 x D 14336 x DT torch.float16 x GS 128    |                74.3                |               43.6             
      N 512 x D 14336 x DT torch.bfloat16 x GS 64    |                79.1                |               72.7             
      N 512 x D 14336 x DT torch.bfloat16 x GS 128   |                67.5                |               43.3             
      N 2048 x D 1024 x DT torch.float16 x GS 64     |                66.6                |               28.7             
      N 2048 x D 1024 x DT torch.float16 x GS 128    |                67.7                |               28.6             
      N 2048 x D 1024 x DT torch.bfloat16 x GS 64    |                67.5                |               28.6             
      N 2048 x D 1024 x DT torch.bfloat16 x GS 128   |                67.8                |               28.5             
      N 2048 x D 2048 x DT torch.float16 x GS 64     |                67.1                |               42.6             
      N 2048 x D 2048 x DT torch.float16 x GS 128    |                67.7                |               28.6             
      N 2048 x D 2048 x DT torch.bfloat16 x GS 64    |                67.5                |               42.6             
      N 2048 x D 2048 x DT torch.bfloat16 x GS 128   |                67.2                |               28.4             
      N 2048 x D 4096 x DT torch.float16 x GS 64     |               109.7                |               82.8             
      N 2048 x D 4096 x DT torch.float16 x GS 128    |                84.7                |               51.6             
      N 2048 x D 4096 x DT torch.bfloat16 x GS 64    |                89.9                |               82.8             
      N 2048 x D 4096 x DT torch.bfloat16 x GS 128   |                74.6                |               51.5             
      N 2048 x D 5120 x DT torch.float16 x GS 64     |               133.6                |              102.6             
      N 2048 x D 5120 x DT torch.float16 x GS 128    |               102.0                |               64.3             
      N 2048 x D 5120 x DT torch.bfloat16 x GS 64    |               109.7                |              102.6             
      N 2048 x D 5120 x DT torch.bfloat16 x GS 128   |                90.4                |               64.2             
      N 2048 x D 14336 x DT torch.float16 x GS 64    |               353.6                |              279.8             
      N 2048 x D 14336 x DT torch.float16 x GS 128   |               265.4                |              173.4             
      N 2048 x D 14336 x DT torch.bfloat16 x GS 64   |               288.3                |              279.8             
      N 2048 x D 14336 x DT torch.bfloat16 x GS 128  |               234.0                |              172.7             

Times are in microseconds (us).

Results:
Accuracy: 0.925
Invalid responses: 0.000
Total latency: 40.881 s
Questions per second: 32.265
Total output tokens: 130873
Output tokens per second: 3201.337

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[Copilot]

Pull request overview

Removes an E8M0-related guard that prevented the fused SiLU+Mul + FP8 block quant CUDA kernel from being used in Triton MoE experts, ensuring the fused path can run under the default DeepGEMM E8M0 configuration.

Changes:

• Dropped the is_deep_gemm_e8m0_used() import and the corresponding runtime condition in the fused-kernel eligibility check.
• Allows ops.silu_and_mul_per_block_quant to be selected based solely on activation/quant/block-shape/LoRA constraints.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gemini-code-assist]

Code Review

This pull request removes the is_deep_gemm_e8m0_used check and its corresponding import from the Triton MoE layer, simplifying the conditional logic in the apply method for FP8 W8A8 quantization. I have no feedback to provide as there were no review comments to evaluate.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @qianlihuang.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork