[Bugfix][MLA] Change default SM100 MLA prefill backend back to TRT-LLM#38562
Merged
vllm-bot merged 1 commit intovllm-project:mainfrom Mar 30, 2026
Merged
Conversation
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
MatthewBonanni
requested review from
ProExpertProg,
WoosukKwon,
hmellor,
houseroad,
mgoin,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256 and
youkaichao
as code owners
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the default value of use_trtllm_ragged_deepseek_prefill to True in the attention configuration. The reviewer suggests renaming this flag to use_trtllm_mla_prefill to better reflect its general purpose for MLA prefill backends and improve maintainability, as the current name is overly specific to DeepSeek.
MatthewBonanni
changed the title
LucasWilkinson
approved these changes
Mar 30, 2026
Collaborator
LucasWilkinson
left a comment
LucasWilkinson
left a comment
There was a problem hiding this comment.
Thank you for the quick fix!
LucasWilkinson
added
the
ready
tlrmchlsmth
approved these changes
Mar 30, 2026
mgoin
approved these changes
Mar 30, 2026
mgoin
reviewed
Mar 30, 2026
| """Whether to use cudnn prefill.""" | ||
|
|
||
| use_trtllm_ragged_deepseek_prefill: bool = False | ||
| use_trtllm_ragged_deepseek_prefill: bool = True |
Member
There was a problem hiding this comment.
Where do we control FA4 MLA prefill? I don't see a similar entry for it
Collaborator
Author
There was a problem hiding this comment.
It falls through to FA4 when trtllm isn't enabled. It's a messy interface, #32623 will clean this up
Collaborator
Author
There was a problem hiding this comment.
Relevant code block is here:
benenzhu
pushed a commit
to benenzhu/vllm
that referenced
this pull request
Mar 31, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: zhutaoyu <zhutaoyu97@gmail.com>
vllm-agent
pushed a commit
to vllm-agent/vllm
that referenced
this pull request
Mar 31, 2026
…o TRT-LLM (vllm-project#38562)" This reverts commit 2c734ed.
neweyes
pushed a commit
to neweyes/vllm
that referenced
this pull request
Mar 31, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: neweyes <328719365@qq.com>
EricccYang
pushed a commit
to EricccYang/vllm
that referenced
this pull request
Apr 1, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: EricccYang <yangyang4991@gmail.com>
bhargav-patel-29
pushed a commit
to Bharatgen-Tech/vllm
that referenced
this pull request
Apr 1, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: bhargav-patel-29 <bhargav.patel@tihiitb.org>
yzong-rh
pushed a commit
to yzong-rh/vllm
that referenced
this pull request
Apr 3, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
liuchenbing2026
pushed a commit
to liuchenbing2026/vllm
that referenced
this pull request
Apr 4, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
rishitdholakia13
pushed a commit
to rishitdholakia13/vllm
that referenced
this pull request
Apr 7, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
puririshi98
pushed a commit
to puririshi98/vllm
that referenced
this pull request
Apr 7, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Rishi Puri <riship@nvidia.com>
big-yellow-duck
pushed a commit
to EmbeddedLLM/vllm
that referenced
this pull request
Apr 8, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
mtparet
pushed a commit
to blackfuel-ai/vllm
that referenced
this pull request
Apr 9, 2026
vllm-project#38562) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FIX: #36763
Purpose
On SM100, FA4 MLA prefill appears to cause unusable output on Kimi-K2.5. This PR changes the default MLA prefill backend back to TRTLLM while we resolve the issues with FA4.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.