Skip to content

[Disagg] Support static DSpark PD for DeepSeek V4.1 - #38949

Merged
hnyls2002 merged 2 commits into
sgl-project:dsv4.1from
hnyls2002:codex/dsv41-dspark-pd
Sep 10, 2026
Merged

hnyls2002 merged 2 commits into
sgl-project:dsv4.1from
hnyls2002:codex/dsv41-dspark-pd

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Stack on dsv4.1: remaining model and runtime integration #38798 (dsv4.1) and enable DeepSeek V4.1 + DSpark + PD in static verify mode, including stochastic sampling.
  • Require Mooncake, matching TP and DSpark block size, and PP/DP/CP size 1. Validate target/draft transfer layouts through Python and Rust bootstrap servers before caching peer mappings or transferring KV.
  • Populate compression metadata on both workers and reuse the existing target/draft SWA and request-scoped pair-ring transfer.
  • Route decode retraction through the existing prefill rebootstrap path, preserving the emitted boundary token and logprob instead of attempting unsupported CPU KV backup.
  • Keep focused regression coverage for bootstrap compatibility and retraction boundary-token replay, plus compression metadata assertions in the existing wire tests.

Validation

  • After pruning tests: 3 retained DSpark PD unit tests and 28 existing PD wire tests passed, along with pre-commit checks. The unchanged Rust implementation previously passed all 5 bootstrap tests, workspace clippy, and formatting on this upstream port.
  • Before this port, on the official deepseek-ai/DeepSeek-V4.1-Flash@517ef625 checkpoint: full sgl-eval GSM8K 1,272/1,319 (96.4367%), zero errors/truncations/empty outputs. H200 TP4 prefill + TP4 decode, static block 5, non-thinking, temperature 0.7/top-p 0.9/seed 0, concurrency 8, max 2,048 output tokens. This used the support patch on the newer development base with official HF config support; no matched colocated GSM8K control was run.
  • Before this port, on the earlier checkpoint: B300 AIME 2025 472/480 (98.3333%) PD versus 464/480 (96.6667%) colocated DSpark, 30 questions x 16 repeats, zero errors/truncations/empty answers. Matched thinking-mode sampling settings, TP4/EP4, static block 5; no forced retractions. H200 forced-retraction regression also passed with 32 x 512 sampled tokens and 133 retractions.
  • The GPU results above remain historical validation; their diagnostic scripts and artifacts are retained outside this PR. GPU evaluations have not been rerun on this upstream commit. Cross-node RDMA and Rust frontend GPU E2E remain untested.

Known limitations

  • Keep draft: the existing rebootstrap payload omits multimodal conditioning, and its internal HTTP request does not forward worker API-key authentication. The accuracy runs cover text-only requests without worker API-key authentication.
  • Shape-dependent numerical differences also reproduce with PD and DSpark disabled. No production numerical fix is included; answer scores do not establish bitwise equivalence.
  • No new steady-state decode kernel or synchronization; bootstrap validation is cached per peer. Retraction adds a device drain and prefix recomputation. No throughput A/B or zero-overhead claim.

CI States

Latest PR Test (Base): ❌ Run #34536354018
Latest PR Test (Extra): ❌ Run #34536353731
Latest PR Test (AMD ROCm 10): ❌ Run #34536354102

@hnyls2002
hnyls2002 marked this pull request as ready for review September 10, 2026 22:14
@hnyls2002
hnyls2002 merged commit 7bbeafd into sgl-project:dsv4.1 Sep 10, 2026
81 of 91 checks passed
wonderisland pushed a commit to shinemo-ai/sglang that referenced this pull request Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant