Skip to content

[Test] Fix one-sided MNNVL alltoall test workspace under-reservation - #44946

Merged
zyongye merged 1 commit into
vllm-project:mainfrom
zyongye:fix-mnnvl-onesided-test-workspace
Jun 10, 2026
Merged

[Test] Fix one-sided MNNVL alltoall test workspace under-reservation#44946
zyongye merged 1 commit into
vllm-project:mainfrom
zyongye:fix-mnnvl-onesided-test-workspace

Conversation

@zyongye

@zyongye zyongye commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

tests/distributed/test_mnnvl_alltoall.py::test_one_sided_dispatch_combine
initializes the FlashInfer one-sided MoeAlltoAll workspace without
declaring the fp8 block-scale payload it later dispatches
. The worker
dispatches four payloads:

  • a1q — nvfp4 hidden states, (tokens, hidden // 2)hidden // 2 B/token
  • a1q_scale — fp8 block scales, (tokens, hidden // 16)hidden // 16 B/token
  • topk_ids, topk_weightstop_k * 4 B/token each

but manager.initialize(...) is called with dispatch_scale_bytes_per_token
left at its default of 0. So the reserved per-rank dispatch region is short
by hidden // 16 bytes/token. That offset shortfall pushes the combine
payload region past the end of the per-rank workspace, and FlashInfer's
combine guard (trtllm_moe_alltoall.cu) trips:

Check failed: (combinePayloadOffset >= 0 && combinePayloadOffset + payloadBytes <= sizePerRank)
workspace insufficient for combine payload region

The per-rank workspace is sized as
aux + ep_size·max_tokens·dispatch_payload_per_token + ep_size·max_tokens·combine_payload_per_token,
so under-declaring the dispatch payload directly under-reserves the buffer.

Fix

Pass dispatch_scale_bytes_per_token=hidden_size // 16 to initialize() so
the reserved dispatch region matches the payloads the test actually
dispatches. One-line, test-only change.

Testing

On a 2-GPU MNNVL host, run with CAP_SYS_PTRACE (MNNVL needs it for IPC; the
tests @requires_ptrace-skip otherwise):

pytest tests/distributed/test_mnnvl_alltoall.py -v
  • Before: 5 passed, 1 failed (test_one_sided_dispatch_combine
    "workspace insufficient for combine payload region").
  • After: 6 passed (all manager-lifecycle, workspace-grow, args/two-sided/
    one-sided dispatch-combine tests).

AI assistance

This change was developed with AI assistance (Claude Code). The diff has been
reviewed and the test result above was run locally.

`test_one_sided_dispatch_combine` initialized the MoeAlltoAll workspace
without declaring the fp8 block-scale payload it later dispatches
(`a1q_scale` of shape `(tokens, hidden // 16)`, i.e. `hidden // 16` bytes
per token). With `dispatch_scale_bytes_per_token` left at its default of 0,
the per-rank dispatch region is under-reserved by `hidden // 16` bytes per
token, which pushes the combine payload region past the end of the per-rank
workspace. FlashInfer's combine guard then fails:

    Check failed: combinePayloadOffset + payloadBytes <= sizePerRank
    workspace insufficient for combine payload region

Pass `dispatch_scale_bytes_per_token=hidden_size // 16` so the reserved
dispatch region matches the payloads actually dispatched.

Test: `pytest tests/distributed/test_mnnvl_alltoall.py` (run with CAP_SYS_PTRACE
on a 2-GPU MNNVL host) now passes 6/6 (was 5 passed, 1 failed).

Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
@zyongye
zyongye marked this pull request as ready for review June 9, 2026 02:36
@zyongye zyongye added ready ONLY add when PR is ready to merge/full CI is needed bug Something isn't working labels Jun 9, 2026
@zyongye
zyongye enabled auto-merge (squash) June 9, 2026 03:38
@zyongye
zyongye merged commit 2ba68d9 into vllm-project:main Jun 10, 2026
22 of 26 checks passed
wcynb1023 pushed a commit to wcynb1023/vllm that referenced this pull request Jun 11, 2026
Saddss pushed a commit to Saddss/vllm that referenced this pull request Jun 14, 2026
vivek8123 pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Jun 18, 2026
divineearthly pushed a commit to divineearthly/vllm that referenced this pull request Jun 19, 2026
…llm-project#44946)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: divineearthly <divineearthly@gmail.com>
nkzhenhua pushed a commit to nkzhenhua/vllm that referenced this pull request Jun 24, 2026
Dao007forever pushed a commit to Dao007forever/vllm that referenced this pull request Jul 18, 2026
philippesic pushed a commit to philippesic/vllm-semantic-cache that referenced this pull request Jul 19, 2026
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants