Skip to content

[Bugfix] Fix DSV3.2 NVFP4#33932

Merged
robertgshaw2-redhat merged 2 commits intovllm-project:mainfrom
MatthewBonanni:fix_ds32_nvfp4
Feb 5, 2026
Merged

[Bugfix] Fix DSV3.2 NVFP4#33932
robertgshaw2-redhat merged 2 commits intovllm-project:mainfrom
MatthewBonanni:fix_ds32_nvfp4

Conversation

@MatthewBonanni
Copy link
Copy Markdown
Collaborator

@MatthewBonanni MatthewBonanni commented Feb 5, 2026

Purpose

Fix #33859

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
@mergify
Copy link
Copy Markdown

mergify bot commented Feb 5, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MatthewBonanni.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added needs-rebase bug Something isn't working labels Feb 5, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a bugfix for Multi-Head Latent Attention (MLA), likely related to DeepSeek V3.2 models using specific quantization formats. The changes prevent an unnecessary and potentially incorrect view casting of the KV cache when the kv_cache_dtype is fp8_ds_mla. Additionally, a safeguard is added to ensure that FP8 query quantization is only performed if the attention implementation supports it, and this support is explicitly enabled for MLACommonImpl. These changes are correct, well-targeted, and improve the robustness of the code. The fix appears solid.

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
@mergify mergify bot removed the needs-rebase label Feb 5, 2026
@robertgshaw2-redhat robertgshaw2-redhat enabled auto-merge (squash) February 5, 2026 17:33
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 5, 2026
@robertgshaw2-redhat robertgshaw2-redhat merged commit 4145e50 into vllm-project:main Feb 5, 2026
55 checks passed
ItzDEXX pushed a commit to ItzDEXX/vllm that referenced this pull request Feb 19, 2026
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
tunglinwood pushed a commit to tunglinwood/vllm that referenced this pull request Mar 4, 2026
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: DeepSeek V3.2-NVFP4 with flashinfer moe reports q must have dtype torch::kBFloat16

2 participants