fix: DeepSeek-V3.2 DeepGEMM RuntimeError

[KeeProMise]

Purpose

see: #30206

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request addresses a RuntimeError in DeepSeek-V3.2 DeepGEMM by introducing a crucial alignment check for the input K dimension. The changes correctly extend the should_use_deepgemm_for_fp8_linear function to include input_k_dim, ensuring that DeepGEMM is only utilized when the input tensor's K dimension is properly aligned. This is particularly important for tensor parallelism scenarios, where input_size_per_partition might not always meet DeepGEMM's alignment requirements. The addition of clear comments and an updated docstring enhances code clarity and maintainability. Overall, this is a well-implemented fix that improves the robustness and correctness of the DeepGEMM integration.

[mergify]

Hi @KeeProMise, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[robertgshaw2-redhat]

Couple notes:

• please add a clear PR description and before + after
• shouldn't we be able to detect this at startup time rather than runtime if this is based on the hidden dim?
• why is this just issue occuring now?

[coval3nte]

why is this just issue occuring now?

official recipe advises TP=8.
I tried building vLLM against multiple commit in last few days and always got the same error.

Do you have a working version and the commit built against?

[yewentao256]

Thanks for the work! Prefer this fix #30267

[KeeProMise]

why is this just issue occuring now?

official recipe advises TP=8. I tried building vLLM against multiple commit in last few days and always got the same error.

Do you have a working version and the commit built against?

How about trying the latest version?

[IKACE]

I applied the fix on vllm 0.12.0 but still got the same error, do you know if there are other ways to resolve this?

[yewentao256]

I applied the fix on vllm 0.12.0 but still got the same error, do you know if there are other ways to resolve this?

Please use the main branch, there is another fix for this merged to main.