[RL] DeepEP support for --enable-return-routed-experts - #16859
Merged
Merged
Conversation
PrinsYin
requested review from
BBuf,
Edwardf0t1,
Fridge003,
HaiShaw,
Ying1123,
ch-wan,
ispobock and
merrymercy
as code owners
January 10, 2026 06:00
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
12 tasks
PR sgl-project#24450 moved RoutedExpertsCapturer into srt/state_capturer/. Re-apply the DeepEP attn-tp all-gather override to the new path. Co-Authored-By: Junrong Lin <33685709+ocss884@users.noreply.github.com>
The previous tp=2 dp=2 config has attn_tp_size=1, so the DeepEP all-gather added by this PR is dead code under that test. Switch reference to tp=4 dp=2 + --moe-a2a-backend deepep (attn_tp_size=2) to actually exercise RoutedExpertsCapturer.capture's gather path. Baseline stays attn_tp_size=1 (tp=4 dp=4) to keep correctness ground truth simple. Bumps suite to stage-c-test-4-gpu-h100. Co-Authored-By: Junrong Lin <33685709+ocss884@users.noreply.github.com>
Collaborator
|
/tag-and-rerun-ci |
…rf flags Mirror the original test pattern (vary perf flags, hold tp/dp + a2a fixed) so a mismatch points cleanly at perf-side bugs (overlap stream race, cuda graph capture mis-sync, radix cache key reuse) rather than at orthogonal config drift between the two servers. Co-Authored-By: Junrong Lin <33685709+ocss884@users.noreply.github.com>
Collaborator
|
/rerun-test test/registered/rl/test_return_routed_experts.py |
Contributor
|
✅ |
hnyls2002
changed the title
--enable-return-routed-experts
…nner
ep_moe/layer.py:238 has `assert False, "forward_deepgemm_contiguous is
deprecated"` for non-{aiter, NPU, w4afp8} models in the deepep_normal
path. The new deep_gemm runner (super().run_moe_core) only kicks in
when quant_config is Fp8Config — Qwen3-30B-A3B (bf16) hits the assert.
Use the FP8 variant Qwen/Qwen3-30B-A3B-FP8.
…n cuda graph capture
Contributor
|
✅ |
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
…#16859) Co-authored-by: hnyls2002 <lsyincs@gmail.com> Co-authored-by: Junrong Lin <33685709+ocss884@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: THUDM/slime#1316
Summary
RoutedExpertsCapturer(rebased ontosrt/state_capturer/after move topk capturers to srt/state_capturer/ #24450)topk_idsunder DeepEP, so all-gather across attn-TP at capture time before staging intodevice_cache--tp 4 --dp 2 --moe-a2a-backend deepep,attn_tp_size == 2); previoustp=dp=2config left the gather as dead code