Skip to content

[Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel - #49467

Merged
vllm-bot merged 2 commits into
vllm-project:mainfrom
neuralmagic:deepgemm-fallback-warmup
Jul 22, 2026
Merged

[Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel#49467
vllm-bot merged 2 commits into
vllm-project:mainfrom
neuralmagic:deepgemm-fallback-warmup

Conversation

@mgoin

@mgoin mgoin commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Purpose

Fix an issue where Hopper + FP8 models regress by up to 20% after https://github.com/vllm-project/vllm/pull/41652/changes#diff-5ee97c25141448b7fd638e73ebcca7d3eead061d22b538e7983f4cdd81ac8f69 landed due to skipping deepgemm warmup

Test Plan

Test Result

vllm serve Qwen/Qwen3-0.6B-FP8
...
Selected FlashInferFp8DeepGEMMDynamicBlockScaledKernel for Fp8LinearMethod
...
DeepGEMM warmup: 100%|██████████| 948/948 [03:23<00:00, 4.65it/s]
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@mgoin
Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScale…
798b3b2 
…dKernel

Signed-off-by: mgoin <mgoin64@gmail.com>
claude[bot]
claude Bot reviewed Jul 22, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Jul 22, 2026
@mgoin mgoin added ready ONLY add when PR is ready to merge/full CI is needed deepseek Related to DeepSeek models nvidia labels Jul 22, 2026
@mgoin mgoin added this to the v0.26.0 cherry picks milestone Jul 22, 2026
@github-project-automation github-project-automation Bot moved this to Ready in NVIDIA Jul 22, 2026
@vllm-bot
vllm-bot merged commit 917fdb5 into vllm-project:main Jul 22, 2026
77 of 79 checks passed
@github-project-automation github-project-automation Bot moved this from Ready to Done in NVIDIA Jul 22, 2026
khluu pushed a commit that referenced this pull request Jul 23, 2026
@mgoin @khluu
[Bugfix] Fix DeepGEMM warmup when using `FlashInferFp8DeepGEMMDynamic…
8b30569 
…BlockScaledKernel` (#49467)

Signed-off-by: mgoin <mgoin64@gmail.com>
(cherry picked from commit 917fdb5)
@fank fank mentioned this pull request Jul 31, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@tlrmchlsmth tlrmchlsmth tlrmchlsmth approved these changes

Assignees

No one assigned

Labels

bug Something isn't working deepseek Related to DeepSeek models nvidia ready ONLY add when PR is ready to merge/full CI is needed

Projects

NVIDIA
Status: Done

Milestone

v0.26.0 cherry picks

Development

Successfully merging this pull request may close these issues.

3 participants

@mgoin @tlrmchlsmth @vllm-bot