[Lucaskabela]

Purpose

Fixes https://github.com/pytorch/pytorch/actions/runs/24457152504/job/71753061820#step:23:5866 by skipping the failing tests

On B200, 8 test_fusion_rmsnorm_quant cases fail in the torch vllm joint test. These tests are bf16 × DeepGemm/FlashInferDeepGemmDynamic kernels × kernel_groupshape5/7.

This is because QuantFP8 takes the packed UE8M0 path (per_token_group_fp8_quant_packed, int32-packed scales), which the rms+quant fusion pattern doesn't match failing our test assert

So, we skip these tests, leaving a todo to implement proper fusion as it would be nontrivial here

Test Plan

On B200, run:

pytest tests/compile/passes/test_fusion.py::test_fusion_rmsnorm_quant

Test Result

===================== 104 passed, 56 skipped, 16 warnings in 490.78s (0:08:10) ======================

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[Lucaskabela]

cc @zou3519 @huydhn

[gemini-code-assist]

Code Review

This pull request introduces support for scale_ue8m0 in the rms_norm_per_block_quant kernel and its associated fusion passes, adding a specific scale adjustment logic during quantization. Feedback suggests refactoring the duplicated scale adjustment logic in layernorm_utils.cuh into a shared helper function to improve maintainability and replacing the magic number 1e-10f with a named constant.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ElizaWszola]

How does it affect performance? Would it make sense to benchmark packed vs. fused op to show that this improves the runtime?

[ProExpertProg]

I agree with Eliza, we'd be affecting the default path here so we need to make sure this maintains/improves perf. I think the proper fusion fix here is slightly more involved and I'd be more than okay just disabling this unit test on B200 for now

[Lucaskabela]

That is fair - here is the timing result:

N	D	packed unfused (µs)	fused UE8M0 (µs)	winner	Δ
1	4096	38.1	30.0	fused	−21%
1	7168	38.1	30.0	fused	−21%
128	4096	39.2	30.5	fused	−22%
128	7168	39.2	31.0	fused	−21%
1024	4096	38.5	30.5	fused	−21%
1024	7168	38.9	34.1	fused	−12%
2048	4096	38.8	34.8	fused	−10%
2048	7168	46.9	55.4	packed	+18%
3072	4096	42.6	49.3	packed	+16%
3072	7168	71.8	84.3	packed	+17%
4096	4096	53.4	65.7	packed	+23%
4096	7168	98.7	112.4	packed	+14%
8192	4096	114.5	130.4	packed	+14%
8192	7168	195.7	218.7	packed	+12%

Since it is a regresssion above N = 1024, I think leaving a todo and disabling is good for now

[Lucaskabela]

cc @ProExpertProg @ElizaWszola ready for another look

[ProExpertProg]

Can you post the old version as a draft somewhere? Probably still a great start for the proper fix

[Lucaskabela]

Can you post the old version as a draft somewhere? Probably still a great start for the proper fix

#40650 :)

[Lucaskabela]

@ProExpertProg can you also mark this PR ready to run testing?