[Bugfix] Fix fp8 DeepGemm compilation issues

[ElizaWszola]

is_deep_gemm_e8m0_used() and current_platform.is_device_capability() are not compatible with Dynamo, causing failed compilations. This PR intends to fix this problem.

Testing

Run inference on Qwen/Qwen3-30B-A3B-FP8 (one of the models affected) with VLLM_USE_DEEP_GEMM=1.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[gemini-code-assist]

Code Review

This pull request aims to fix compilation issues for fp8 DeepGemm by refactoring how device capabilities are checked. While the changes in vllm/utils/deep_gemm.py are correct, a critical bug has been introduced in vllm/model_executor/layers/quantization/utils/fp8_utils.py. A line of code was moved out of an else block, which will cause incorrect quantization behavior. This needs to be addressed.

[yewentao256]

Why is_deepgemm_e8m0 could not be used? If it is because of the @cache, I am thinking we should have something like #29038 instead of hardcode for is_blackwell.

[ElizaWszola]

@yewentao256 It's because of torch._dynamo.exc.Unsupported: can't handle functions not implemented in python - this is the error I had also run into a lot originally when unwrapping W8A8BlockFp8LinearOp, it's why so many variables that rely on knowing the architecture are assigned in its constructor

[yewentao256]

@yewentao256 It's because of torch._dynamo.exc.Unsupported: can't handle functions not implemented in python - this is the error I had also run into a lot originally when unwrapping W8A8BlockFp8LinearOp, it's why so many variables that rely on knowing the architecture are assigned in its constructor

Got it, please refactor the class DeepGemmQuantScaleFMT, make it cache the architecture value in initialization then.
There are a lot of places using this class and may have the similar issue.

[ElizaWszola]

@yewentao256 Isn't from_oracle() a static method?

Alternatively, before cleaning up this PR, I had implemented this kind of changes: aea97d1 but this felt a bit superfluous to me

[yewentao256]

@yewentao256 Isn't from_oracle() a static method?

Alternatively, before cleaning up this PR, I had implemented this kind of changes: aea97d1 but this felt a bit superfluous to me

I think we don't actually need the static, just refactoring it to normal func. @varun-sundar-rabindranath also CC

[jhaotingc]

#26197

Shouldn't e8m0 also compatible with hopper?

[jhaotingc]

cc @yewentao256 , thanks!

[mgoin]

IIUC this is specifically for the case where e8m0 scales need to be packed, which is a Blackwell only case

[jhaotingc]

Thank you @mgoin for the explanation!

[yewentao256]

OK Let's land this first since it is a blocker for CI, I can have follow up PR for refactoring the class later.