Skip to content

[CI] Fix broken compile tests due to unsupported SiluMul+Nvfp4Quant fusion#23973

Merged
vllm-bot merged 3 commits intovllm-project:mainfrom
sarckk:fix-fusion-pass-test
Aug 30, 2025
Merged

[CI] Fix broken compile tests due to unsupported SiluMul+Nvfp4Quant fusion#23973
vllm-bot merged 3 commits intovllm-project:mainfrom
sarckk:fix-fusion-pass-test

Conversation

@sarckk
Copy link
Collaborator

@sarckk sarckk commented Aug 30, 2025

Purpose

Fix failing tests compile/test_pass_manager.py::test_pass_manager_uuid and compile/test_full_graph.py::test_custom_compile_config.

The error is:

[2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)   File "/usr/local/lib/python3.12/dist-packages/vllm/compilation/activation_quant_fusion.py", line 174, in __init__
--
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)     pattern_silu_mul_nvfp4 = SiluMulNvfp4QuantPattern()
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)   File "/usr/local/lib/python3.12/dist-packages/vllm/compilation/activation_quant_fusion.py", line 116, in __init__
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)     super().__init__(kNvfp4Quant)
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)   File "/usr/local/lib/python3.12/dist-packages/vllm/compilation/activation_quant_fusion.py", line 55, in __init__
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)     assert self.quant_key in FUSED_OPS, \
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176) torch._dynamo.exc.BackendCompilerFailed: backend='<vllm.compilation.backends.VllmBackend object at 0x7f27e72f50a0>' raised:
  | [2025-08-29T20:36:33Z] (EngineCore_0 pid=14176) AssertionError: unsupported fusion scheme QuantKey(u8,scale(f8e4m3fn,dynamic,GroupShape(row=1, col=16)),scale2(f32,static,per_tensor),symmetric)

This can happen when hasattr(torch.ops._C, "silu_and_mul_nvfp4_quant") is false. Only enable SiluMul+Nvfp4Quant fusion added in #23671 if supported.

Test Plan

pytest tests/compile/test_pass_manager.py::test_pass_manager_uuid
pytest tests/compile/test_full_graph.py::test_custom_compile_config

Test Result

All tests passed locally, check passes in CI as well.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly fixes a crash that occurs when the SiluMul+Nvfp4Quant fusion is attempted on a platform where it's not supported. The change adds a check to ensure the fusion is only registered if the required silu_and_mul_nvfp4_quant op is available. The fix is sound. I have added one comment regarding code duplication which could pose a maintenance risk.

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
@ywang96 ywang96 added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 30, 2025
@ywang96 ywang96 enabled auto-merge (squash) August 30, 2025 04:09
@vllm-bot vllm-bot merged commit 9748c51 into vllm-project:main Aug 30, 2025
37 of 39 checks passed
eicherseiji pushed a commit to eicherseiji/vllm that referenced this pull request Sep 9, 2025
…usion (vllm-project#23973)

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
…usion (vllm-project#23973)

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants