[Bugfix][Hardware][AMD] Fix FP8 dtype in silu_mul quantization

[c0de128]

Summary

Fix hardcoded torch.float8_e4m3fn dtype in silu_mul_per_token_group_quant_fp8_colmajor() that causes incorrect dtype and accuracy issues on ROCm platforms using torch.float8_e4m3fnuz.

Problem

The function in vllm/model_executor/layers/quantization/utils/fp8_utils.py was:

1. Hardcoding torch.float8_e4m3fn dtype for the output tensor (line 629)
2. Using default finfo.min/max values from torch.float8_e4m3fn (lines 640-642)

On ROCm platforms that use torch.float8_e4m3fnuz:

• The output tensor has the wrong dtype
• The fp8 min/max values (240.0) cause accuracy issues - the correct value is 224.0 for fnuz

Solution

Apply the same pattern already used in per_token_group_quant_fp8() in the same file (lines 766-770):

fp8_dtype = current_platform.fp8_dtype()
finfo = torch.finfo(fp8_dtype)
fp8_min = -224.0 if current_platform.is_fp8_fnuz() else finfo.min
fp8_max = 224.0 if current_platform.is_fp8_fnuz() else finfo.max

Test Plan

This is a consistency fix that aligns the function with the existing ROCm-aware pattern used elsewhere in the same file. The fix ensures:

• Correct fp8 dtype is used based on platform
• Correct fp8 min/max values are used for fnuz dtype on ROCm

Related

This is similar to the pattern established for other FP8 quantization functions in this file that already handle ROCm fnuz correctly.

[gemini-code-assist]

Code Review

This pull request addresses a bug in silu_mul_per_token_group_quant_fp8_colmajor where a hardcoded torch.float8_e4m3fn dtype and its associated min/max values were used, causing accuracy issues on ROCm platforms that expect torch.float8_e4m3fnuz. The changes correctly use current_platform.fp8_dtype() to determine the appropriate float8 data type and apply platform-specific min/max values for quantization, aligning the function's behavior with per_token_group_quant_fp8 and ensuring correctness on ROCm. The changes are correct and well-implemented.

[c0de128]

@hongxiayang @jithunnair-amd This is ready for review and addresses critical FP8 dtype handling for ROCm on the new Strix Halo architecture.

[yewentao256]

LGTM, thanks for the work!

[c0de128]

Hi maintainers,

This PR has been approved by @yewentao256. The CI failures appear to be known flaky tests (e.g., async-engine-inputs-utils-worker-config-test timeout) that are unrelated to this FP8 dtype fix.

Would it be possible to trigger a merge or a final re-run of the failing jobs? The fix itself is straightforward - using current_platform.fp8_dtype() instead of hardcoded torch.float8_e4m3fn.

Thank you!

[c0de128]

@vllm-bot rerun ci

The async-engine-inputs-utils-worker-config-test failure appears unrelated to this PR (FP8 dtype fix). This change only affects ROCm FP8 quantization, not async engine configuration. Requesting CI rerun.