[puririshi98]

Summary

Three small, opt-in additions for ModelOpt MXFP8 serving on Blackwell (sm_100), all inert by default — with none of the env vars set, behaviour is identical to current main:

1. VLLM_MODELOPT_EXTRA_EXCLUDE_MODULES — append comma-separated fnmatch wildcard patterns to the exclude_modules list parsed from the checkpoint, so operators can keep specific Linear/MoE modules in BF16 (e.g. small projections where per-call quantize overhead dominates) without re-exporting the checkpoint. Patterns flow through the existing is_layer_excluded() path. (vllm/model_executor/layers/quantization/modelopt.py)

2. VLLM_MXFP8_BF16_FALLBACK_SMALL_M — at decode-time batch sizes (M < 128), FlashInfer's CUTLASS mm_mxfp8 pads M up to a 128-row tile and discards the rest, wasting most of the GEMM. When enabled, process_weights_after_loading caches a BF16-dequantized copy of the weight (via the existing dequant_mxfp8_to_bf16) and apply_weights does a plain BF16 matmul for M < 128. Roughly doubles this layer's weight memory, hence opt-in. (vllm/model_executor/kernels/linear/mxfp8/flashinfer.py)

3. VLLM_MXFP8_TRITON_SMALLM — a dedicated Triton MXFP8 GEMM that handles M < 128 natively without the 128-row tile padding. Caches the un-swizzled [N, K/32] weight scale for row-major indexing; takes precedence over the BF16 fallback when both are set. (new vllm/model_executor/kernels/linear/mxfp8/triton_smallm.py + wiring)

All three env vars are registered in vllm/envs.py. The change is purely additive (+397 lines, 6 files, no deletions).

Not a duplicate

Searched open PRs (modelopt exclude_modules, mxfp8 small M, mxfp8 triton, EXTRA_EXCLUDE_MODULES). No open PR adds an env-based exclude-modules override, a small-M BF16 fallback, or a small-M Triton GEMM for the FlashInfer CUTLASS MXFP8 linear kernel. Nearest neighbours are distinct: #45875 tunes tile sizes for the MXFP8 MTP path; the ROCm / MiniMax-M3 MXFP8 PRs target AMD gfx, not the sm_100 CUTLASS linear path.

Test plan

Adds one allclose correctness test per numerical path under tests/kernels/quantization/:

• test_mxfp8_smallm_gemm.py — Triton kernel vs a dequantize-then-matmul reference, M ∈ {1, 16, 17, 32, 64, 127}, N/K ∈ {128, 256} (CUDA-gated).
• test_mxfp8_bf16_fallback.py — small-M BF16 fallback vs reference (sm_100-gated).

pytest tests/kernels/quantization/test_mxfp8_smallm_gemm.py \
       tests/kernels/quantization/test_mxfp8_bf16_fallback.py -v

The allclose equivalences these tests assert were validated on an sm_100 (Blackwell) device — 28/28 pass across both paths (M ∈ {1,16,17,32,64,127}, N/K ∈ {128,256}, rtol/atol 3e-2). The committed pytest files encode the same checks; please run them in CI / on a Blackwell node before merge.

Based on work reviewed by @TomerBN-Nvidia

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mgoin]

This seems silly to me. Why can you just add a swap ab case to the cutlass kernel for small M?

[puririshi98]

Closing this. After digging into the actual code path (prompted by @mgoin's swap-A/B question), the small-M optimization this PR was adding isn't needed in vLLM. - vLLM main never pads M, the padding I was working around only ever existed in an specific fork