[Bugfix] Fix packed GDN decode launch for large batch-head grids - #52030
Merged
Merged
Conversation
Signed-off-by: mgoin <mgoin64@gmail.com>
mgoin
requested review from
AndreasKaratzas,
WoosukKwon,
ZJY0516,
tlrmchlsmth,
vadiklyutiy,
yewentao256 and
zyongye
as code owners
August 12, 2026 17:44
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #83590 for commit |
Member
Author
|
/ci retry |
|
✅ Queued 8 failed job(s) for retry in Buildkite CI #83590. |
pmanczak
added a commit
to pmanczak/vllm
that referenced
this pull request
Aug 19, 2026
The test hardcoded a CUDA device and skipped everywhere else, so the packed-decode kernel and its gated-delta-rule reference had no coverage on non-CUDA accelerators even though both are pure Triton and already build there. Resolve the device via current_platform and gate on CUDA-alike or XPU, matching the pattern already used by sibling Triton kernel tests such as tests/kernels/mamba/test_mamba_ssm_ssd.py. The gate becomes a module-level pytestmark so it also covers test_packed_decode_supports_large_batch_head_grid (added in vllm-project#52030), whose own torch.cuda.is_available() skip and hardcoded device string are dropped as redundant. That test is not a no-op off CUDA: B * HV = 65536 selects the SPLIT_BATCH_HEAD_GRID branch on XPU as well, so it is the only coverage this platform has for the 3D-grid launch path. Verified on Intel Arc Pro B70 (torch 2.13.0+xpu, Triton 3.7.2): 7 passed, six covering float16/bfloat16/float32 x strided/non-strided and one the large batch-head grid launch. Signed-off-by: pmanczak <pawel.manczak@intel.com>
pmanczak
added a commit
to pmanczak/vllm
that referenced
this pull request
Aug 20, 2026
The test hardcoded a CUDA device and skipped everywhere else, so the packed-decode kernel and its gated-delta-rule reference had no coverage on non-CUDA accelerators even though both are pure Triton and already build there. Resolve the device via current_platform and gate on CUDA-alike or XPU, matching the pattern already used by sibling Triton kernel tests such as tests/kernels/mamba/test_mamba_ssm_ssd.py. The gate becomes a module-level pytestmark so it also covers test_packed_decode_supports_large_batch_head_grid (added in vllm-project#52030), whose own torch.cuda.is_available() skip and hardcoded device string are dropped as redundant. That test is not a no-op off CUDA: B * HV = 65536 selects the SPLIT_BATCH_HEAD_GRID branch on XPU as well, so it is the only coverage this platform has for the 3D-grid launch path. Verified on Intel Arc Pro B70 (torch 2.13.0+xpu, Triton 3.7.2): 7 passed, six covering float16/bfloat16/float32 x strided/non-strided and one the large batch-head grid launch. Signed-off-by: pmanczak <pawel.manczak@intel.com>
zyp2014
pushed a commit
to zyp2014/vllm
that referenced
this pull request
Aug 21, 2026
…m-project#52030) Signed-off-by: mgoin <mgoin64@gmail.com>
zufangzhu
pushed a commit
to zufangzhu/vllm
that referenced
this pull request
Aug 24, 2026
…m-project#52030) Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
khushali9
pushed a commit
to khushali9/vllm
that referenced
this pull request
Aug 29, 2026
…m-project#52030) Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: khushali9 <khushali.desai9@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Avoid a CUDA launch failure in packed GDN decode when
batch_size * num_value_headsexceeds the maximum CUDA grid Y/Z dimension of 65,535.The existing launch is preserved for normal sizes. Only overflowing cases use a split
(value_tiles, value_heads, batch)grid.Test Plan
Test Result
B=1024,HV=64,K=V=128) launches successfully.vllm serve mgoin/Qwen3.8-2.4T-A95B-NVFP4-pruned94 -tp=2doesn't crash anymoreEssential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.