Skip to content

[Feat] deepseek_v41: Enable monokernels for MoRI PD decode - #70

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
AMD-yanfeiwang:feat/dsv41-mori-pd-decode
Oct 29, 2026
Merged

HaiShaw merged 1 commit into
sgl-project:mainfrom
AMD-yanfeiwang:feat/dsv41-mori-pd-decode

Conversation

@AMD-yanfeiwang

@AMD-yanfeiwang AMD-yanfeiwang commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

The DeepSeek-V4.1-Flash monokernels refuse every PD server today: the target and DSpark draft specs do not declare PD_DISAGGREGATION, so --enable-amd-monokernel on a decode server stops with "PD disaggregation is in use and the spec does not accept it". #67 enabled MoRI PD decode for DeepSeek-V4-Pro; this does the same for V4.1. SGLang serves V4.1 DSpark PD over MoRI since sgl-project/sglang#42457.

Modifications

  • deepseek_v41/spec.py and deepseek_v41/draft.py declare PD_DISAGGREGATION.
  • _config_disable_reasons refuses a prefill server and transfer backends other than MoRI, with the same reasons as V4-Pro.
  • SGLANG_READS pins disaggregation_mode and disaggregation_transfer_backend.
  • A unit test covers the gate; the README's MoRI PD note names both models.

The kernels and their layouts are unchanged. They run decode and DSpark verify steps only, so all the state they read (paged SWA KV, compressor states, index keys, the draft's KV) lives in SGLang's own pools, which MoRI fills on the decode server.

With a per-node shared Engram host table, decode also needs #65. Both peers must leave SGLANG_HIP_DSV41_INDEX_K_PAGE8 unset: it changes the index keys' bytes but not their size, so SGLang's PD handshake does not catch a mismatch (the README says so).

Accuracy Tests

DeepSeek-V4.1-Flash + DSpark (block size 5), 1P1D on one MI355X node over MoRI with SGLANG_RAGGED_VERIFY_MODE=static: prefill without monokernels at the decode's TP. TP2 keeps the Engram tables in per-rank host memory on both peers (--engram-config '{"placement":"cpu"}'), as the AgentX TP2 recipe does; on GPU they leave TP2 no room for the KV cache. lmsysorg/sglang-rocm:v0.5.22-rocm10-mi35x-20261028 running SGLang main. GSM8K is sglang.test.few_shot_gsm8k with 1319 questions, sent through the PD router.

decode TP --enable-amd-monokernel startup parallel 64 parallel 8 DSpark accept length
main any yes refused: PD not accepted - - -
this PR 4 no serves 0.889 0.908 3.39
this PR 4 yes serves; target and draft kernels built on all ranks 0.895 0.915 3.39
this PR 2 no serves 0.908 0.904 3.37
this PR 2 yes serves; target and draft kernels built on all ranks 0.904 0.906 3.38

The monokernels run verify steps of up to 8 requests, so parallel 8 runs every verify step on them; at parallel 64 most steps are wider and take SGLang's own path. The spread matches run-to-run nondeterminism. Without PD, on the same image, the monokernels score 0.904 to 0.909 at parallel 64 and 0.903 to 0.914 at parallel 8 across Engram table placements.

CPU, same image: test_deepseek_v41_spec.py passes, including the new PD test, and so does the deepseek_v41 contract. The two minimax / minimax_m31 contract failures are the same on main.

@AMD-yanfeiwang
AMD-yanfeiwang marked this pull request as ready for review October 28, 2026 08:20
@AMD-yanfeiwang AMD-yanfeiwang changed the title [Feat] deepseek_v41: Enable monokernels for MoRI PD decode [Feat] deepseek_v41: Enable monokernels for MoRI and Mooncake PD decode Oct 28, 2026
@AMD-yanfeiwang AMD-yanfeiwang changed the title [Feat] deepseek_v41: Enable monokernels for MoRI and Mooncake PD decode [Feat] deepseek_v41: Enable monokernels for MoRI PD decode Oct 28, 2026
Allow the DeepSeek-V4.1-Flash target and DSpark draft monokernels on a MoRI
PD decode server, as sgl-project#67 does for DeepSeek-V4-Pro; SGLang serves V4.1 DSpark
PD over MoRI since sgl-project/sglang#42457. The kernels cover decode and
DSpark verify steps only, so the KV, compressor-state and index-key buffers
they read are the ones MoRI fills. Prefill servers and other transfer backends
stay refused. The README names the index-key layout switch that both peers
must share, since the PD handshake does not compare it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants