Skip to content

[ROCm][CI TG] refactor and fix deepep_moe test group - #46758

Merged
tjtanaa merged 3 commits into
vllm-project:mainfrom
divakar-amd:ci_fix_deepepmoe
Jun 27, 2026
Merged

[ROCm][CI TG] refactor and fix deepep_moe test group#46758
tjtanaa merged 3 commits into
vllm-project:mainfrom
divakar-amd:ci_fix_deepepmoe

Conversation

@divakar-amd

@divakar-amd divakar-amd commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Fix test_deepep_moe.py on ROCm and consolidate MoE test accuracy checks

  1. Buffer reuse (low-latency DeepEP): The modular kernel and its DeepEP all2all buffer are now built once and reused across all token chunks instead of being re-created per chunk. Re-creating it per chunk re-initializes rocSHMEM, which only supports a single allocation per process on ROCm and aborts with "Unknown allocator type". The low-latency buffer is sized by max_tokens_per_rank/hidden_size (not chunk size), so one buffer is valid for every chunk — mirroring production, where the all2all handle is cached and reused (DeepEPLLAll2AllManager). Because the buffer is reused (and thus left dirty by the previous chunk), clean_low_latency_buffer is called before each dispatch to restore the zero-initialized regions the low-latency kernels require.

  2. Accuracy check consolidation: ROCm's fp8 format (e4m3fnuz) rounds differently than NVIDIA's e4m3fn, so a few elements in the DeepEP MoE output can exceed a strict bit-for-bit tolerance even though the kernel is correct. Rather than add a bespoke comparison, this reuses the existing check_accuracy mismatch-fraction helper:

    (a.) Moved check_accuracy from test_ocp_mx_moe.py into the shared tests/kernels/moe/utils.py (already used by cuda MXFP4/MXFP8 paths for accuracy check). test_ocp_mx_moe.py now imports it; call sites unchanged.
    (b.) Re-use the above check_accuracy for test_deepep_moe.py. The base path uses percent=1.0 (equivalent to the previous assert_close); the relaxed tolerance is gated behind current_platform.is_fp8_fnuz().

Fixes: tests/kernels/moe/test_deepep_moe.py::test_low_latency_deep_ep_moe


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the rocm Related to AMD ROCm label Jun 25, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Jun 25, 2026
@AndreasKaratzas AndreasKaratzas added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 25, 2026
Comment thread tests/kernels/moe/test_deepep_moe.py
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
@divakar-amd
divakar-amd requested a review from tjtanaa June 26, 2026 16:39

@tjtanaa tjtanaa left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now

@tjtanaa
tjtanaa merged commit 00e045b into vllm-project:main Jun 27, 2026
29 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Jun 27, 2026
WindChimeRan pushed a commit to WindChimeRan/vllm that referenced this pull request Jun 27, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
wincent8 pushed a commit to wincent8/vllm that referenced this pull request Jun 29, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
rjrock pushed a commit to rjrock/vllm that referenced this pull request Jul 1, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Dao007forever pushed a commit to Dao007forever/vllm that referenced this pull request Jul 18, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
philippesic pushed a commit to philippesic/vllm-semantic-cache that referenced this pull request Jul 19, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
)

Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants