Repository navigation
[Feat] [AMD] mori: Allow MoRI for DeepSeek-V4.1 DSpark PD - #42457
Merged
HaiShaw merged 1 commit intoOct 28, 2026
Merged
Conversation
AMD-yanfeiwang
force-pushed
the
feat/mori-dsv41-dspark-pd
branch
2 times, most recently
from
October 12, 2026 14:10
3ea388b to
d36e680
Compare
AMD-yanfeiwang
force-pushed
the
feat/mori-dsv41-dspark-pd
branch
from
October 13, 2026 01:46
d36e680 to
f0dc923
Compare
ChangLi811
added a commit
to ChangLi811/sglang
that referenced
this pull request
Oct 21, 2026
The target and draft SWA rings already ship as SWA_RING components; let the DSpark PD spec-layout handshake accept them, drop the unified + DSpark + PD rejection, and allow MoRI as the DSpark PD transfer backend (as sgl-project#42457).
4 of 5 tasks
ChangLi811
added a commit
to ChangLi811/sglang
that referenced
this pull request
Oct 23, 2026
The target and draft SWA rings already ship as SWA_RING components; let the DSpark PD spec-layout handshake accept them, drop the unified + DSpark + PD rejection, and allow MoRI as the DSpark PD transfer backend (as sgl-project#42457).
ChangLi811
added a commit
to ChangLi811/sglang
that referenced
this pull request
Oct 26, 2026
The target and draft SWA rings already ship as SWA_RING components; let the DSpark PD spec-layout handshake accept them, drop the unified + DSpark + PD rejection, and allow MoRI as the DSpark PD transfer backend (as sgl-project#42457).
HaiShaw
approved these changes
Oct 28, 2026
HaiShaw
changed the title
HaiShaw
enabled auto-merge (squash)
October 28, 2026 09:25
HaiShaw
disabled auto-merge
October 28, 2026 09:25
AMD-yanfeiwang
added a commit
to AMD-yanfeiwang/sgl-amd-mono
that referenced
this pull request
Oct 28, 2026
sgl-project/sglang#42457 lets SGLang serve DeepSeek-V4.1 DSpark PD over MoRI, so the gate goes back to MoRI only, the backend these kernels were tested on (Mooncake fails at prefill startup on the MI355X ionic NICs, monokernels off).
AMD-yanfeiwang
added a commit
to AMD-yanfeiwang/sgl-amd-mono
that referenced
this pull request
Oct 28, 2026
Allow the DeepSeek-V4.1-Flash target and DSpark draft monokernels on a MoRI PD decode server, as sgl-project#67 does for DeepSeek-V4-Pro; SGLang serves V4.1 DSpark PD over MoRI since sgl-project/sglang#42457. The kernels cover decode and DSpark verify steps only, so the KV, compressor-state and index-key buffers they read are the ones MoRI fills. Prefill servers and other transfer backends stay refused. The README names the index-key layout switch that both peers must share, since the PD handshake does not compare it.
HaiShaw
pushed a commit
to sgl-project/sgl-amd-mono
that referenced
this pull request
Oct 29, 2026
## Motivation The DeepSeek-V4.1-Flash monokernels refuse every PD server today: the target and DSpark draft specs do not declare `PD_DISAGGREGATION`, so `--enable-amd-monokernel` on a decode server stops with "PD disaggregation is in use and the spec does not accept it". #67 enabled MoRI PD decode for DeepSeek-V4-Pro; this does the same for V4.1. SGLang serves V4.1 DSpark PD over MoRI since sgl-project/sglang#42457. ## Modifications - `deepseek_v41/spec.py` and `deepseek_v41/draft.py` declare `PD_DISAGGREGATION`. - `_config_disable_reasons` refuses a prefill server and transfer backends other than MoRI, with the same reasons as V4-Pro. - `SGLANG_READS` pins `disaggregation_mode` and `disaggregation_transfer_backend`. - A unit test covers the gate; the README's MoRI PD note names both models. The kernels and their layouts are unchanged. They run decode and DSpark verify steps only, so all the state they read (paged SWA KV, compressor states, index keys, the draft's KV) lives in SGLang's own pools, which MoRI fills on the decode server. With a per-node shared Engram host table, decode also needs #65. Both peers must leave `SGLANG_HIP_DSV41_INDEX_K_PAGE8` unset: it changes the index keys' bytes but not their size, so SGLang's PD handshake does not catch a mismatch (the README says so). ## Accuracy Tests DeepSeek-V4.1-Flash + DSpark (block size 5), 1P1D on one MI355X node over MoRI with `SGLANG_RAGGED_VERIFY_MODE=static`: prefill without monokernels at the decode's TP. TP2 keeps the Engram tables in per-rank host memory on both peers (`--engram-config '{"placement":"cpu"}'`), as the AgentX TP2 recipe does; on GPU they leave TP2 no room for the KV cache. `lmsysorg/sglang-rocm:v0.5.22-rocm10-mi35x-20261028` running SGLang main. GSM8K is `sglang.test.few_shot_gsm8k` with 1319 questions, sent through the PD router. | decode | TP | `--enable-amd-monokernel` | startup | parallel 64 | parallel 8 | DSpark accept length | |---|---|---|---|---:|---:|---:| | `main` | any | yes | refused: PD not accepted | - | - | - | | this PR | 4 | no | serves | 0.889 | 0.908 | 3.39 | | this PR | 4 | yes | serves; target and draft kernels built on all ranks | 0.895 | 0.915 | 3.39 | | this PR | 2 | no | serves | 0.908 | 0.904 | 3.37 | | this PR | 2 | yes | serves; target and draft kernels built on all ranks | 0.904 | 0.906 | 3.38 | The monokernels run verify steps of up to 8 requests, so parallel 8 runs every verify step on them; at parallel 64 most steps are wider and take SGLang's own path. The spread matches run-to-run nondeterminism. Without PD, on the same image, the monokernels score 0.904 to 0.909 at parallel 64 and 0.903 to 0.914 at parallel 8 across Engram table placements. CPU, same image: `test_deepseek_v41_spec.py` passes, including the new PD test, and so does the `deepseek_v41` contract. The two `minimax` / `minimax_m31` contract failures are the same on `main`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
validate_deepseek_v41_featuresonly accepts Mooncake and NIXL (#43734) for DeepSeek-V4.1 DSpark PD, so--disaggregation-transfer-backend morifails at startup. The V4.1 DSpark PD support is backend-agnostic: the layout and TP checks run inCommonKVManager.try_ensure_parallel_info, and MoRI already transfers the target and draft SWA states through its generic state path.Modifications
Accept
morinext tomooncakeandnixlin the V4.1 DSpark PD check and update the error message.Accuracy Tests
DeepSeek-V4.1-Flash, 1P1D on one 8x MI355X node (prefill TP4 + decode TP4), DSpark, MoRI,
lmsysorg/sglang-rocm:v0.5.21-rocm724-mi35x-20261003+ this PR:With the UMBP direct linker on the prefill side (#42197), a 110K-token prompt after
flush_cacheon both servers restores 110,080 cached tokens and produces greedy output identical to the cold run.These runs use CP=1 on both servers. Since #40323 the same check also admits prefill CP; MoRI with prefill CP is not covered here.
Speed Tests and Profiling
N/A: this only widens a startup check; the transfer path is unchanged.
Checklist
CI States
Latest PR Test (Base): ❌ Run #38333740714
Latest PR Test (Extra): ❌ Run #38333740578
Latest PR Test (AMD ROCm 10): ❌ Run #38333740698