Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121

[meena-at-work]

Summary

Adds two new FlashInfer b12x backends targeting SM120/SM121 GPUs (DGX Spark GB10, RTX Pro 6000 Blackwell):

1. b12x fused MoE backend ()

Uses FlashInfer's cute_dsl_fused_moe_nvfp4 kernel (flashinfer-ai/flashinfer#3066) to accelerate NVFP4 MoE on SM120/SM121. The kernel fuses token dispatch, W1 GEMM, SwiGLU, and W2 GEMM into a single call; BF16 hidden states are passed directly as activation quantization is fused internally.

• experts/flashinfer_cutedsl_moe.py — new FlashInferCuteDSLSM12xExperts class; process_weights_after_loading normalises FP4 block scales and resets a2_gscale to 1.0 (SM12x uses dynamic per-block quantisation for FC2 input; the calibrated modelopt value causes saturation)
• vllm/config/kernel.py — adds flashinfer_cutedsl_sm12x to MoEBackend Literal
• vllm/utils/flashinfer.py — adds has_flashinfer_cutedsl_sm12x_moe() capability check
• oracle/nvfp4.py — oracle auto-selects the SM12x backend on SM120-family GPUs
• flashinfer_fp4_moe.py — weight-prep path handles the SM12x backend
• tests/kernels/moe/test_cutedsl_sm12x_moe.py — 24 unit tests (skipped on non-SM12x hardware)

2. b12x FP4 dense GEMM backend (flashinfer-b12x)

Integrates FlashInfer PR flashinfer-ai/flashinfer#3051 b12x dense GEMM backend into the NVFP4 linear layer path. b12x uses CuTe DSL warp-level MMA with adaptive tile sizing to improve SM utilization on small-M decode shapes.

• has_flashinfer_b12x_gemm() — availability check via Sm120BlockScaledDenseGemmKernel
• FlashInferB12xNvFp4LinearKernel — new NvFp4LinearKernel subclass; auto-selected on SM120/SM121, falls back to FLASHINFER_CUTLASS when unavailable
• Adds flashinfer-b12x to VLLM_NVFP4_GEMM_BACKEND valid choices
• b12x test cases in test_flashinfer_nvfp4_scaled_mm.py

Hardware requirements

• SM120 or SM121 GPU (DGX Spark GB10 / RTX Pro 6000 Blackwell)
• FlashInfer with PRs feat: Add backend="b12x" for mm_fp4 on SM120 flashinfer-ai/flashinfer#3051 and feat: Add b12x CuTe DSL fused MoE for SM120 flashinfer-ai/flashinfer#3066

Test plan

• pytest tests/kernels/moe/test_cutedsl_sm12x_moe.py on SM120/SM121 hardware (24/24 passing)
• pytest tests/kernels/quantization/test_flashinfer_nvfp4_scaled_mm.py -k b12x on SM121 hardware (36/36 passing)
• Full-model throughput on DGX Spark (SM121, Qwen3-30B-A3B-NVFP4):
  • b12x GEMM vs flashinfer-cutlass: +1.8% (1P), +6.0% (8P)
• Non-SM12x hardware: oracle does not select these backends (falls through to existing path)

[gemini-code-assist]

Code Review

This pull request introduces support for the FlashInfer CuteDSL fused MoE kernel on SM12x architectures, adding the FlashInferCuteDSLSM12xExperts class and necessary backend configurations. Feedback identifies several issues: in-place modification of a2_gscale causes permanent side effects on model parameters, and performing MMA layout conversions during every forward pass adds unnecessary overhead. Additionally, suggestions were made to ensure correct data types for activation scale placeholders and routing weights when interfacing with the FlashInfer kernel.

[gemini-code-assist]

Modifying self.a2_gscale in-place using fill_(1.0) has a permanent side effect on the model's quantization parameters in memory. If the model is subsequently used with a different MoE backend (e.g., during profiling, testing, or if the backend is changed via configuration), the results will be incorrect because the original calibrated scales have been destroyed. It is safer to pass a tensor of ones directly to the kernel in the apply method or create a local copy if modification is necessary.

[gemini-code-assist]

Converting scale factors to MMA layout on every forward pass introduces unnecessary overhead. Since the dimensions m (intermediate size) and k (hidden dimension) are fixed for the layer, this conversion should be performed once during weight initialization or in process_weights_after_loading. Even if the conversion is a zero-copy view, it is more efficient to pre-compute these views and store them as attributes of the expert class to avoid redundant metadata creation and function calls in the hot path.

[gemini-code-assist]

The x_sf_placeholder should use the correct data type expected by the FlashInfer kernel. The existing FlashInferCuteDSLExperts class (line 157) uses float8_e4m3fn for activation scales. Using hidden_states.new_zeros(1) will create a bfloat16 tensor, which may cause a type mismatch or incorrect behavior in the kernel even if the values are ignored.

Suggested change

	else hidden_states.new_zeros(1)
	else hidden_states.new_zeros(1, dtype=torch.float8_e4m3fn)

[gemini-code-assist]

The token_final_scales argument should be converted to float32 before being passed to the FlashInfer kernel. The existing FlashInferCuteDSLExperts class (line 166) explicitly performs this conversion, and FlashInfer fused MoE kernels typically expect routing weights in float32 for accumulation precision. Passing topk_weights directly (which is bfloat16) is likely to cause type errors or precision issues.

Suggested change

	token_final_scales=topk_weights,
	token_final_scales=topk_weights.float(),