-
-
Notifications
You must be signed in to change notification settings - Fork 19.3k
[Attention] Abstract the MLA prefill backends and eliminate cuDNN #32623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
MatthewBonanni
merged 55 commits into
vllm-project:main
from
MatthewBonanni:mla_prefill_abstraction
May 1, 2026
Merged
Changes from 15 commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
e4228dc
Change default from CUTLASS MLA to FlashInfer MLA
MatthewBonanni 30cd686
Change log lines from debug to info
MatthewBonanni 0b4c746
Merge remote-tracking branch 'upstream/main'
MatthewBonanni 2cfba3b
First pass
MatthewBonanni 5417186
Fix typo
MatthewBonanni c427ff8
Fix pre-commit
MatthewBonanni b80db7e
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni 370d66d
Use device_config
MatthewBonanni 5151bd9
Remove dead code
MatthewBonanni 6922185
Bump deprecation version
MatthewBonanni f0a0acd
Update docs
MatthewBonanni 3dc0fba
Cleanup
MatthewBonanni 0225644
Format name
MatthewBonanni 8013037
Fix dagger inside quotes
MatthewBonanni 6688a67
Add selector test
MatthewBonanni a962bee
Update table
MatthewBonanni 25d976f
Comment
MatthewBonanni f30cb13
Add model dtype support
MatthewBonanni eb27ec8
Add type annotation to fix docs build
MatthewBonanni 17c9a9e
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni 0f42a95
Fix rebase issue
MatthewBonanni 97c9aa7
Introduce hashable config
MatthewBonanni fb7bead
Fix test
MatthewBonanni 3b25d20
Pass device capability directly
MatthewBonanni 0086b94
Fix hashing
MatthewBonanni 5051737
Fix tests
MatthewBonanni 115a75b
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni 88d0be8
Fix pre-commit
MatthewBonanni 0cd22de
Clean up FA import
MatthewBonanni 7bf5ae8
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni d4c77ab
wip
LucasWilkinson 2f0b2ad
cleanup
LucasWilkinson 1d04d07
cleanup
LucasWilkinson 9077fe2
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni 26d38ac
Fixes
MatthewBonanni f1f47f9
Fix pre-commit
MatthewBonanni de1c30f
Cleanup
MatthewBonanni b4be44d
Merge branch 'main' into mla_prefill_abstraction
MatthewBonanni 6ca7954
Fix test
MatthewBonanni a8edfd1
Fix old flag handling
MatthewBonanni d75923c
Clean up
MatthewBonanni d71ddcf
FA4 default
MatthewBonanni 37399f7
Clean up
MatthewBonanni 0433138
Remove cuDNN
MatthewBonanni 2255459
Eliminate metadata classes, cleanup
MatthewBonanni eada601
Merge impl into backend
MatthewBonanni 8489336
Update docs
MatthewBonanni 21874a4
Clean up
MatthewBonanni b14ba7e
Clean up
MatthewBonanni 83d9580
Update test
MatthewBonanni 7bf83a2
Fix test
MatthewBonanni 710581e
Put the prefill backend in the metadata
MatthewBonanni f42e555
Merge branch 'main' into mla_prefill_abstraction
mergify[bot] 0a1feb2
Fix test
MatthewBonanni f73e58b
Clean up and fix test
MatthewBonanni File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.