[Test] Fix one-sided MNNVL alltoall test workspace under-reservation - #44946
Merged
zyongye merged 1 commit intoJun 10, 2026
Merged
Conversation
`test_one_sided_dispatch_combine` initialized the MoeAlltoAll workspace
without declaring the fp8 block-scale payload it later dispatches
(`a1q_scale` of shape `(tokens, hidden // 16)`, i.e. `hidden // 16` bytes
per token). With `dispatch_scale_bytes_per_token` left at its default of 0,
the per-rank dispatch region is under-reserved by `hidden // 16` bytes per
token, which pushes the combine payload region past the end of the per-rank
workspace. FlashInfer's combine guard then fails:
Check failed: combinePayloadOffset + payloadBytes <= sizePerRank
workspace insufficient for combine payload region
Pass `dispatch_scale_bytes_per_token=hidden_size // 16` so the reserved
dispatch region matches the payloads actually dispatched.
Test: `pytest tests/distributed/test_mnnvl_alltoall.py` (run with CAP_SYS_PTRACE
on a 2-GPU MNNVL host) now passes 6/6 (was 5 passed, 1 failed).
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
zyongye
marked this pull request as ready for review
June 9, 2026 02:36
zyongye
enabled auto-merge (squash)
June 9, 2026 03:38
youkaichao
approved these changes
Jun 10, 2026
wcynb1023
pushed a commit
to wcynb1023/vllm
that referenced
this pull request
Jun 11, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Saddss
pushed a commit
to Saddss/vllm
that referenced
this pull request
Jun 14, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
vivek8123
pushed a commit
to odh-on-pz/vllm-upstream
that referenced
this pull request
Jun 18, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
divineearthly
pushed a commit
to divineearthly/vllm
that referenced
this pull request
Jun 19, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: divineearthly <divineearthly@gmail.com>
nkzhenhua
pushed a commit
to nkzhenhua/vllm
that referenced
this pull request
Jun 24, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Dao007forever
pushed a commit
to Dao007forever/vllm
that referenced
this pull request
Jul 18, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
philippesic
pushed a commit
to philippesic/vllm-semantic-cache
that referenced
this pull request
Jul 19, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
plasticchris
pushed a commit
to plasticchris/vllm
that referenced
this pull request
Jul 20, 2026
…llm-project#44946) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/distributed/test_mnnvl_alltoall.py::test_one_sided_dispatch_combineinitializes the FlashInfer one-sided
MoeAlltoAllworkspace withoutdeclaring the fp8 block-scale payload it later dispatches. The worker
dispatches four payloads:
a1q— nvfp4 hidden states,(tokens, hidden // 2)→hidden // 2B/tokena1q_scale— fp8 block scales,(tokens, hidden // 16)→hidden // 16B/tokentopk_ids,topk_weights—top_k * 4B/token eachbut
manager.initialize(...)is called withdispatch_scale_bytes_per_tokenleft at its default of
0. So the reserved per-rank dispatch region is shortby
hidden // 16bytes/token. That offset shortfall pushes the combinepayload region past the end of the per-rank workspace, and FlashInfer's
combine guard (
trtllm_moe_alltoall.cu) trips:The per-rank workspace is sized as
aux + ep_size·max_tokens·dispatch_payload_per_token + ep_size·max_tokens·combine_payload_per_token,so under-declaring the dispatch payload directly under-reserves the buffer.
Fix
Pass
dispatch_scale_bytes_per_token=hidden_size // 16toinitialize()sothe reserved dispatch region matches the payloads the test actually
dispatches. One-line, test-only change.
Testing
On a 2-GPU MNNVL host, run with
CAP_SYS_PTRACE(MNNVL needs it for IPC; thetests
@requires_ptrace-skip otherwise):5 passed, 1 failed(test_one_sided_dispatch_combine→"workspace insufficient for combine payload region").
one-sided dispatch-combine tests).
AI assistance
This change was developed with AI assistance (Claude Code). The diff has been
reviewed and the test result above was run locally.