Repository navigation
[Feat] deepseek_v41: Enable monokernels for MoRI PD decode - #70
Merged
HaiShaw merged 1 commit intoOct 29, 2026
Merged
Conversation
AMD-yanfeiwang
marked this pull request as ready for review
October 28, 2026 08:20
AMD-yanfeiwang
requested review from
HaiShaw,
alexnails,
kevin-mii and
kkHuang-amd
as code owners
October 28, 2026 08:20
Allow the DeepSeek-V4.1-Flash target and DSpark draft monokernels on a MoRI PD decode server, as sgl-project#67 does for DeepSeek-V4-Pro; SGLang serves V4.1 DSpark PD over MoRI since sgl-project/sglang#42457. The kernels cover decode and DSpark verify steps only, so the KV, compressor-state and index-key buffers they read are the ones MoRI fills. Prefill servers and other transfer backends stay refused. The README names the index-key layout switch that both peers must share, since the PD handshake does not compare it.
AMD-yanfeiwang
force-pushed
the
feat/dsv41-mori-pd-decode
branch
from
October 28, 2026 11:52
1224a54 to
5d63b12
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The DeepSeek-V4.1-Flash monokernels refuse every PD server today: the target and DSpark draft specs do not declare
PD_DISAGGREGATION, so--enable-amd-monokernelon a decode server stops with "PD disaggregation is in use and the spec does not accept it". #67 enabled MoRI PD decode for DeepSeek-V4-Pro; this does the same for V4.1. SGLang serves V4.1 DSpark PD over MoRI since sgl-project/sglang#42457.Modifications
deepseek_v41/spec.pyanddeepseek_v41/draft.pydeclarePD_DISAGGREGATION._config_disable_reasonsrefuses a prefill server and transfer backends other than MoRI, with the same reasons as V4-Pro.SGLANG_READSpinsdisaggregation_modeanddisaggregation_transfer_backend.The kernels and their layouts are unchanged. They run decode and DSpark verify steps only, so all the state they read (paged SWA KV, compressor states, index keys, the draft's KV) lives in SGLang's own pools, which MoRI fills on the decode server.
With a per-node shared Engram host table, decode also needs #65. Both peers must leave
SGLANG_HIP_DSV41_INDEX_K_PAGE8unset: it changes the index keys' bytes but not their size, so SGLang's PD handshake does not catch a mismatch (the README says so).Accuracy Tests
DeepSeek-V4.1-Flash + DSpark (block size 5), 1P1D on one MI355X node over MoRI with
SGLANG_RAGGED_VERIFY_MODE=static: prefill without monokernels at the decode's TP. TP2 keeps the Engram tables in per-rank host memory on both peers (--engram-config '{"placement":"cpu"}'), as the AgentX TP2 recipe does; on GPU they leave TP2 no room for the KV cache.lmsysorg/sglang-rocm:v0.5.22-rocm10-mi35x-20261028running SGLang main. GSM8K issglang.test.few_shot_gsm8kwith 1319 questions, sent through the PD router.--enable-amd-monokernelmainThe monokernels run verify steps of up to 8 requests, so parallel 8 runs every verify step on them; at parallel 64 most steps are wider and take SGLang's own path. The spread matches run-to-run nondeterminism. Without PD, on the same image, the monokernels score 0.904 to 0.909 at parallel 64 and 0.903 to 0.914 at parallel 8 across Engram table placements.
CPU, same image:
test_deepseek_v41_spec.pypasses, including the new PD test, and so does thedeepseek_v41contract. The twominimax/minimax_m31contract failures are the same onmain.