[Spec] DSpark support prefill/decode disaggregation by zhangxiaolei123456 · Pull Request #31466 · sgl-project/sglang

zhangxiaolei123456 · 2026-07-16T13:27:50Z

Motivation

Roadmap: #30344

Reference PR: #30513

This PR adds DSpark support for prefill/decode disaggregation on DeepSeek-V4.

On main, DSpark can run in the normal non-disaggregated path, but the PD decode path does not receive the target-model hidden states required to bootstrap the draft-side DSpark state. As a result, decode may enter speculative decoding without valid DSpark spec_info / draft KV state.

This PR fixes that gap by transferring DSpark target hidden states from Prefill to Decode through the disaggregation protocol, then injecting them into the decode-side draft KV cache before the first draft step.

The implementation keeps the existing non-disaggregated DSpark behavior unchanged and scopes the new behavior to PD mode.

What Changed Compared With main

DSpark Hidden State PD Transfer

Add a new disaggregation state type: DSPARK_HIDDEN.
Let Decode describe the required DSpark hidden transfer in metadata:
- target layer ids
- hidden start offset
- hidden length
- decode radix-cache policy
- PP-aware slice metadata
- dynamic destination buffer information
Let Prefill capture target hidden states only on the PP rank that owns the DSpark target layers.
Transfer DSpark hidden states through Mooncake/NIXL using row-addressed dynamic destination buffers.
Trim hidden transfer windows according to the Prefill-side cached prefix while preserving absolute token offsets.
Keep fail-fast checks when required hidden rows are incomplete before transfer.

Decode-Side Draft Bootstrap

Assemble received DSpark hidden slices on Decode before committing the transferred request.
Attach the assembled hidden rows to prefill_tail_hidden_states.
Build DSpark draft input from the Prefill output token as the first decode anchor.
Inject transferred target hidden states into the decode-side draft KV cache before the first draft step.

PP-Aware Prefill Support

Add PP-aware hidden slice metadata so only the PP rank owning the target layers sends DSpark hidden states.
Propagate DSpark auxiliary hidden tensors through PP proxy outputs.
Match PP outputs by microbatch id / request identity to avoid FIFO mismatch.
Snapshot PP batch sequence lengths so delayed PP result processing does not read mutated request state.
Drain failed PP prefill bootstrap requests consistently across PP ranks.
Preserve PP admission behavior using side-effect-free resource credit probing before consensus.

Hidden Buffer and Transfer Resource Management

Add DSparkHiddenTransferPlan to describe row/chunk transfer layouts.
Add decode-side DSparkHiddenPagePool to reuse registered GPU receive buffers.
Add optional decode hidden receive prewarm:
- SGLANG_DSPARK_PD_HIDDEN_RECV_PREWARM_ROWS
- SGLANG_DSPARK_PD_HIDDEN_RECV_PREWARM_PAGES
Release Prefill hidden rows as soon as the hidden transfer finishes in the Mooncake worker, instead of waiting for the full KV request success.
Prefer contiguous hidden row allocation and slice copy on Prefill.
Use torch.empty instead of torch.zeros when assembling Decode hidden tensors to avoid unnecessary clearing.

Metadata and Speculative Decode Integration

Extend disaggregation metadata buffers to carry DSpark prefill-tail hidden state information.
Extend DFlashDraftInputV2 / DSpark draft input to carry:
- prefill_tail_hidden_states
- prefill_tail_valid_mask
- prefill_tail_start_positions
- prefill_tail_hidden_projected
Add non-padded token count metadata needed by DSpark draft forward batches.
Ensure Decode radix-cache and Prefill radix-cache policy are aligned for DSpark hidden correctness.

Streaming Hidden Chunk Transfer and Release Semantics

Add streaming DSpark hidden transfer for Mooncake so long prompts do not require materializing the full prompt_len * hidden_width tensor on Prefill or Decode.
Split DSpark hidden transfer state from KV request completion:
- hidden chunk ACK only controls streaming window reuse.
- hidden request done does not imply KV request success.
- request success still follows the original KV / metadata success path.
Release Prefill-side hidden source rows after the hidden request is done, before full KV request completion.
Release Decode-side hidden receive rows on normal request release / abort paths.
Serialize Mooncake DSpark hidden chunks per room to preserve chunk ordering under ACK-based flow control.
Send DSpark hidden-only chunks even when the aligned KV page count is zero.
Flush pending streaming hidden chunks before writing the next chunk, preventing a later chunk from overwriting an unsent dspark_hidden_current_* slot.
Use offset-based streaming source row writes so multi-chunk hidden transfer preserves the absolute hidden token range.
Add fail-fast checks for source-window overwrite hazards and invalid streaming hidden ordering.
Add backend capability boundaries:
- Mooncake supports streaming DSpark hidden transfer.
- NIXL / MORI keep the non-streaming/default path and do not enter partially implemented streaming release semantics.

Why These Changes Are Needed

DSpark speculative decoding depends on target-model hidden states to initialize the draft-side state correctly. In PD mode, Prefill and Decode run in separate processes, so Decode cannot derive those hidden states locally.

The new DSPARK_HIDDEN transfer path makes the hidden state an explicit part of the PD protocol, similar to KV/state transfer, but with PP-aware slicing because only specific Prefill PP ranks own the DSpark target layers.

The resource probing and transactional allocation logic are needed to avoid PP rank divergence: all PP ranks must agree on which requests enter the pipeline before any rank performs side-effectful resource allocation.

The dynamic registered GPU receive buffer pool avoids CPU bounce and keeps hidden transfer on the GDR path.

Compatibility

Non-disaggregated DSpark behavior is unchanged.
Existing KV / C128 / auxiliary state transfer paths remain intact.
DSpark PD hidden transfer is activated only when Decode requests DSPARK_HIDDEN metadata.
Default decode hidden receive prewarm is disabled, so default memory footprint is unchanged.

Current Scope and Future Work

This PR supports DSpark PD hidden-state transfer for Prefill with PP. The implementation is PP-aware: each Prefill PP rank only captures and transfers the DSpark target hidden slice owned by its local layer range.

Mooncake now supports streaming DSpark hidden transfer with hidden/KV/request completion semantics separated. NIXL and MORI do not yet implement streaming hidden release semantics and therefore keep the default non-streaming capability boundary.

Prefill radix cache cannot be enabled alone for DSpark PD today.

DSpark hidden transfer must match the KV transfer window. If Prefill radix cache is enabled but Decode radix cache is disabled, Prefill may skip cached prompt tokens and the corresponding target hidden states will be missing on Decode.

Therefore, DSpark PD currently requires Prefill and Decode radix-cache policies to be consistent. Until DeepSeek-V4 Decode radix cache is supported, Prefill radix cache should stay disabled. #31097

Future work will extend this to Prefill with PP + TP + CP and streaming support for NIXL/MORI. That requires generalizing the hidden metadata from PP-only layer slicing to a full PP/TP/CP layout, including tensor-parallel hidden sharding and context-parallel token-range reconstruction on Decode.

Accuracy Tests

Prefill
SGLANG_PP_LAYER_PARTITION="6,5,5,6,5,5,6,5" SGLANG_DISAGGREGATION_BOOTSTRAP_TIMEOUT=600 NCCL_SOCKET_IFNAME=eth0  NCCL_IB_DISABLE=0 SGLANG_DSV4_FP4_EXPERTS=1 GLOO_SOCKET_IFNAME=eth0 python3 -m sglang.launch_server --model-path /data00/models/DeepSeek-V4-Flash-DSpark --host 0.0.0.0 --port 30000 --trust-remote-code --kv-cache-dtype fp8_e4m3 --mem-fraction-static 0.8 --max-running-requests 64 --chunked-prefill-size 8192 --max-prefill-tokens 16384 --pp-size 8  --attention-backend dsv4 --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 --disable-overlap-schedule --disable-piecewise-cuda-graph  --disaggregation-mode prefill --disaggregation-transfer-backend mooncake --enable-metrics --disaggregation-ib-device mlx5_1,mlx5_2,mlx5_3,mlx5_4 --moe-runner-backend flashinfer_mxfp4 --disable-radix-cache

Decode
SGLANG_DSPARK_PD_HIDDEN_RECV_POOL_TOKENS=65536 SGLANG_DSV4_FP4_EXPERTS=1 SGLANG_JIT_DEEPGEMM_PRECOMPILE=1 SGLANG_OPT_DEEPGEMM_HC_PRENORM=1 SGLANG_OPT_USE_TILELANG_MHC_PRE=1 GLOO_SOCKET_IFNAME=eth0 NCCL_MIN_NCHANNELS=24 NCCL_IB_QPS_PER_CONNECTION=8 sglang serve --trust-remote-code --model-path /data00/models/DeepSeek-V4-Flash-DSpark --tp 8 --dp-size 8 --enable-dp-attention --cuda-graph-max-bs 32 --max-running-requests 256 --enable-metrics --host 0.0.0.0 --port 30000 --mem-fraction-static 0.85 --tool-call-parser deepseekv4 --reasoning-parser deepseek-v4 --moe-runner-backend flashinfer_mxfp4 --disaggregation-mode decode --disaggregation-ib-device "mlx5_1,mlx5_2,mlx5_3,mlx5_4" --speculative-algo DSPARK --tokenizer-worker-num 8 --enable-dp-lm-head --load-balance-method round_robin --swa-full-tokens-ratio 0.8

Hardward	MMLU	GSM8K	QPQA	aime25 repeats 16	status
H20	0.885	0.954	0.910	93.96	PD

Speed Tests and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review and Merge Process

Ping Merge Oncalls to start the process. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #29816426168
Latest PR Test (Extra): ❌ Run #29816426138

yz-tang · 2026-07-17T06:57:12Z

@zhangxiaolei123456 I fix a problem when use dsv4-flash-dspark in PD mode, Is it related to your changes? #31513

zhangxiaolei123456 · 2026-07-17T07:06:23Z

@yz-tang this PR focus on DSpark hidden state transfer from prefill to decode, your PR may not have noticed this.

hnyls2002

Seems like a lot of duplicated code hunks. Could you please get a roadmap for your PR? This should have a clear breakdown for each hunk of changes.

hnyls2002 · 2026-07-20T04:06:53Z

Can you move this dspark logic out of scheduler.py

…(DSpark PD disaggregation) Summarizes sgl-project#31466: the problem (decode-side draft KV cannot bootstrap without target hidden states in PD mode), the design (PD_HIDDEN state type, decode-driven negotiation, PP-aware slicing, streaming transfer with ACK flow control and split hidden/KV completion semantics, registered GPU row pools), a layer-by-layer implementation walkthrough of all 26 changed files, compatibility boundaries, test coverage, and review status with engineering observations. Co-authored-by: learning <learning-sketch@users.noreply.github.com>

dongyibo · 2026-08-06T15:12:08Z

@zhangxiaolei123456 hello~ any plan to merge it? Looking forward to this fea!

zhangxiaolei123456 · 2026-08-06T16:36:43Z

@dongyibo this pr transfer hidden state, main branch use transfer draft kv so the main branch can support PD and DSpark，but if you want support PP + PD + DSpark，you can use this pr #33863, if you want PP + PD + CP +DSpark, you can add this pr #33870

Eviannn · 2026-08-13T11:09:17Z

@zhangxiaolei123456 hi, can support split target layers across PP ranks?

Backport the final functional diff of sgl-project/sglang#31466 at c96e68b62 onto deepseek-v4-hcu while preserving HCU DCP, staging, CP, and graph-replay changes. PD runtime validation is pending dedicated prefill/decode resources. Static compilation and test_pd_hidden_state.py (12 tests) pass.

…r + fix concurrent deadlock Root cause chain (4+ concurrent SIGQUIT crash): 1. New prebuilt batch's build_disagg_draft_input returned None (spec_info.py lacked the DSPARK branch) -> merge_batch skipped spec_info merge -> running batch kept stale bs=1 draft_input while batch grew to bs=N -> verify ForwardBatch mixed bs=1 input_ids with bs=N req_pool_indices -> cuda graph fill_from shape [1] vs [N] -> SIGQUIT all ranks. Fixes: - Port sgl-project#31466 core (16 files, ~2900 lines): PD_HIDDEN state type, hidden row pool, prefill-side capture, mooncake streaming transfer, decode-side inject_pd_hidden_chunk, DSPARK build_disagg_draft_input branch. - schedule_batch.merge_batch: guard 'if self.spec_info and other.spec_info' (None-safe; other.spec_info=None no longer crashes AttributeError). - fake/nixl/mori send_metadata: accept spec_metadata kwarg (signature compat with CommonKVReceiver, fixes warmup FakeKVReceiver TypeError). - eager/verify preload diagnostics kept behind exception-only paths (FILL-MISMATCH prints slot+shape on mismatch). Verified: single + 2/4/8/8 concurrent all-correct output, 0 scheduler crash, decode health 200 sustained.

Backport the final functional diff of sgl-project/sglang#31466 at c96e68b62 onto deepseek-v4-hcu while preserving HCU DCP, staging, CP, and graph-replay changes. PD runtime validation is pending dedicated prefill/decode resources. Static compilation and test_pd_hidden_state.py (12 tests) pass.

…r + fix concurrent deadlock Root cause chain (4+ concurrent SIGQUIT crash): 1. New prebuilt batch's build_disagg_draft_input returned None (spec_info.py lacked the DSPARK branch) -> merge_batch skipped spec_info merge -> running batch kept stale bs=1 draft_input while batch grew to bs=N -> verify ForwardBatch mixed bs=1 input_ids with bs=N req_pool_indices -> cuda graph fill_from shape [1] vs [N] -> SIGQUIT all ranks. Fixes: - Port sgl-project#31466 core (16 files, ~2900 lines): PD_HIDDEN state type, hidden row pool, prefill-side capture, mooncake streaming transfer, decode-side inject_pd_hidden_chunk, DSPARK build_disagg_draft_input branch. - schedule_batch.merge_batch: guard 'if self.spec_info and other.spec_info' (None-safe; other.spec_info=None no longer crashes AttributeError). - fake/nixl/mori send_metadata: accept spec_metadata kwarg (signature compat with CommonKVReceiver, fixes warmup FakeKVReceiver TypeError). - eager/verify preload diagnostics kept behind exception-only paths (FILL-MISMATCH prints slot+shape on mismatch). Verified: single + 2/4/8/8 concurrent all-correct output, 0 scheduler crash, decode health 200 sustained.

Backport the final functional diff of sgl-project/sglang#31466 at c96e68b62 onto deepseek-v4-hcu while preserving HCU DCP, staging, CP, and graph-replay changes. PD runtime validation is pending dedicated prefill/decode resources. Static compilation and test_pd_hidden_state.py (12 tests) pass. (cherry picked from commit d04b624)

… v0.5.18 release. Based branch: feat/rye_20260814_deepseek-v4_open * [HCU][DSpark] Fix DP EP MoE routing on main (cherry picked from commit ac169c9) * DSpark support DeepEP & DeepGemm (cherry picked from commit 412dd6a638f1d1cc6ed162d40cfc4dfbb89915ea) (cherry picked from commit b781f71) * [HCU][DSpark] Backport PD hidden-state disaggregation Backport the final functional diff of sgl-project/sglang#31466 at c96e68b62 onto deepseek-v4-hcu while preserving HCU DCP, staging, CP, and graph-replay changes. PD runtime validation is pending dedicated prefill/decode resources. Static compilation and test_pd_hidden_state.py (12 tests) pass. (cherry picked from commit d04b624) * [HCU][DSpark] Fix DeepEP target and draft runtime state (cherry picked from commit ce62425) * [HCU][DSpark] Adapt PD initialization to current cache API (cherry picked from commit c00a0e4) * [BugFix][DeepSeek-V4-pd-disaggregation] fix the cp error in the disaggreation prefill phase (cherry picked from commit c8f44ba) * [FEAT] support PD disaggregation with DSpark as decode and cp+ep as prefill (cherry picked from commit 3aa5772) * [HCU][DSpark] Make PD hidden decode hooks tolerate unwired queues The PD hidden-state hooks added to DecodePreallocQueue.pop_preallocated and DecodeTransferQueue.pop_transferred dereferenced collaborators that are wired up after construction (transfer_queue, kv_manager) and per-request state that only exists once a request enters the PD hidden path (pd_hidden_state). Any caller holding a partially constructed queue -- including main's registered decode-queue cleanup tests -- hit AttributeError on the abort/failure paths. Guard the three sites: getattr for transfer_queue, a class-level kv_manager default on DecodeTransferQueue, and a None-tolerant pd_hidden_state read in _drain_pd_hidden_ready_chunks. Production wiring is unchanged (all three are always set by scheduler init), so this only affects the unwired case. Fixes the 3 test_decode_queue_cleanup failures that predate this rebase. (cherry picked from commit 45ea9b7) * [HCU][DSpark] Fold PD hidden-state init into Scheduler.init_disaggregation SchedulerDisaggregationInitMixin.__init_subclass__ swapped a full copy of init_disaggregation (SchedulerDisaggregationPrefillMixin, 175 lines) over the one defined in the Scheduler class body. That copy was forked from an older main and has since drifted: it lost the PD Decode DP-sync Gloo group, the rust-server ascend config store, the unified-memory disagg move gate, the get_disagg()/get_parallel() accessors, and -- the crash reported here -- `self.disagg_prefill_pending_chunk_rids`, which main's send_kv_chunk now maintains: File ".../disaggregation/prefill.py", line 2305, in send_kv_chunk self.disagg_prefill_pending_chunk_rids.discard(req.rid) AttributeError: 'Scheduler' object has no attribute 'disagg_prefill_pending_chunk_rids' Whole-method duplication cannot survive rebases: every main-side addition to init_disaggregation silently disappears on the PD path. The branch's actual delta is three lines wide -- resolve_disagg_metadata_config() widening the hidden-state geometry, plus **metadata_buffer_kwargs on the two MetadataBuffers constructions -- so fold exactly that into the single implementation, guarded on disaggregation_mode != NULL (a non-PD server must not inspect speculative workers), and delete the copy along with the installer mixin. Repro: P HCU_NUM=4 PD_OPEN=1 PD_MODE=prefill MTP_MODE=dspark DSPARK_MOE_MODE=deepep + D HCU_NUM=4 PD_OPEN=1 MTP_MODE=dspark DSPARK_MOE_MODE=deepep + router. (cherry picked from commit 1e4eeab) * [HCU] Fix PD Decode DP-sync block to use ParallelState accessors The PD Decode single-clock setup in init_disaggregation still reads self.pp_size / self.attn_tp_size / self.attn_cp_size / self.tp_rank, which no longer exist on Scheduler after the ParallelState (self.ps) refactor. The block only runs for disaggregation_mode == DECODE with dp attention, so nothing upstream exercises it and the stale names never surfaced: File ".../managers/scheduler.py", line 1315, in init_disaggregation if self.pp_size != 1: AttributeError: 'Scheduler' object has no attribute 'pp_size' Re-express the four reads on self.ps. Verified no bare self.{pp_size, attn_tp_size, attn_cp_size, tp_rank} remains in scheduler.py; the rest of the file already uses self.ps for all four. Inherited verbatim from the rebase base d20a475 -- it was reachable only once the duplicated init_disaggregation copy stopped shadowing the real one. Repro: D side HCU_NUM=4 PD_OPEN=1 PD_MODE=decode MTP_MODE=dspark DSPARK_MOE_MODE=deepep. (cherry picked from commit a511ebf) * [HCU] Use AITER TileLang MHC pre path (cherry picked from commit 22c5647) * [HCU] Add SlimQuant W4A8 Triton MoE backend (cherry picked from commit 60c48d0) * Fix PD hidden state transfer length (cherry picked from commit 4bde65d) * Support aiter dspark w4a8 (cherry picked from commit aa378d2) * perf(hcu): optimize DSV4 TopK transform with LightOp (cherry picked from commit a6617f2) * perf(hcu): run DSV4 FlashMLA with gathered BF16 KV (cherry picked from commit 790a529) * perf(hcu): add INT8 indexer cache for DeepSeek-V4 Persist C4 indexer K as signed INT8 with per-token FP32 scales, quantize and scatter cache rows with Triton, and route INT8 Q/K through the native LightOp Paged MQA path. Gate the feature with SGLANG_DSV4_HCU_INT8_INDEX_K_CACHE and keep DSpark draft workers on their existing cache path. Validated with a simple chat smoke test, a 128K BS=1 PD request, and full GSM8K (mean_acc=0.9689). (cherry picked from commit 505634b) * perf(hcu): use LightOp for DSV4 BF16 KV gather Add an environment-gated LightOp backend for the DSV4 single-cache and dual-cache gather/upconvert paths. Keep the existing Triton combined-cache implementation as the default fallback and validate the required LightOp symbols at startup. (cherry picked from commit 8b6e25f) * [BugFix][DeepSeek-V4-dspark] fix the compatible issues of deepep in dspark draft model due to a second LL Buffer (cherry picked from commit acae5ec) * fix(hcu): adapt _is_fake_transfer call to the release signature The PD hidden-state block cherry-picked from the DSV4 dev branch calls _is_fake_transfer(decode_req.req, self.scheduler.server_args) but this release line already refactored the helper to take the request only and read the transfer backend from get_disagg(); the two are semantically identical. On the release branch the stale call crashed the decode scheduler on the first preallocated batch: File ".../disaggregation/decode.py", line 1338, in pop_preallocated and not _is_fake_transfer( TypeError: _is_fake_transfer() takes 1 positional argument but 2 were given Neither py_compile nor ruff F821 sees an arity mismatch, so it only surfaced at runtime. A cross-tree sweep of every function whose arity differs between the release base and the dev branch (71 of them) over the cherry-picked files found no other call site that fails to fit the release signature. * w4a8 draft moe backend triton (cherry picked from commit 79c0fb1) * [docs] remove unused test unit test cases and relocate docs --------- Co-authored-by: elmo2019 <elmo2019@163.com> Co-authored-by: Yujiang Chao <chaoyujiang@hygon.cn> Co-authored-by: aivictor0901-tech <aivictor0901@gmail.com> Co-authored-by: chaoyujiang <chaoyujiang@hygon.com>

zhangxiaolei123456 added 30 commits July 15, 2026 11:25

Add DSpark PD hidden transfer support

8b7f4bf

Align DSpark hidden prefill with decode prefix

6a762cf

Harden DSpark PD prefill bootstrap failures

a79986e

Fix DSpark PD hidden debug env

264557a

Guard missing future indices in overlap spec extras

45b151b

Make DSpark PP hidden slices explicit

7a953fe

Seed DSpark disagg draft input in overlap

e0ae4d3

Pad DSpark prefill tail on draft input merge

d9b5dbf

Propagate DSpark hidden capture layers to forward batch

c526e8e

Carry DSpark hidden outputs through PP prefill

d3886fe

Enable DSpark hidden capture in PP disagg prefill

9ffef48

Route DSpark hidden metadata by prefill PP rank

ef5ab3a

Defer DSpark hidden transfer until rows are ready

547ec64

Wait for DSpark metadata before prefill admission

74aab38

Require matching radix cache policy for DSpark PD

f035f9d

Preserve PP consensus for DSpark prefill admission

72999df

Keep DSpark PP prefill forward mode consistent

4a294b7

Limit DSpark PP capture to local target layers

9566f30

Gate PP prefill admission on local bootstrap finalize

88c8db4

Fail fast on invalid DSpark hidden transfer

129b6df

Warn and skip invalid DSpark hidden

126a135

Relax DSpark hidden prealloc throttling

851fa3b

Use packetized DSpark hidden receive buffers

c37a784

Packetize DSpark hidden transfer state

0d039f1

Disable DSpark hidden request-count throttling

f01f281

Prune DSpark hidden transfer knobs

f9ad078

Tune DSpark hidden transfer defaults

25a9331

Move DSpark hidden transfer buffers to GPU

fbcb238

Default DSpark hidden transfer to one-shot

fbbe252

Add DSpark hidden transfer status logs

3d1170b

zhangxiaolei123456 added 2 commits July 17, 2026 23:02

DSpark streaming hidden inject (#640)

0f4273c

Fix DSpark PD streaming hidden ownership (#641)

f96b07b

hnyls2002 reviewed Jul 20, 2026

View reviewed changes

Comment thread python/sglang/srt/managers/scheduler.py

hnyls2002 Jul 20, 2026

Copy link
Copy Markdown

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this dspark logic out of scheduler.py

zhangxiaolei123456 added 7 commits July 20, 2026 23:00

Refactor DSpark disagg (#645)

28b0747

mooncake pd hidden cleanup (#649)

17562a2

Align DSpark PD hidden changes with project rules (#651)

9d61d03

schedule dspark slim (#652)

32990a0

Reduce DFlash spec input conflict with main

6c3d430

Merge branch 'main' into deepseev_v4_dpsark_pd_dev

6c67f78

Keep scheduler aligned with main for disaggregation init (#653)

c96e68b

cursor Bot mentioned this pull request Jul 22, 2026

docs(examples): DSpark 学习资料 — 交互式讲解页 + 深度解析笔记与 draw.io 图 learning-sketch/sglang#1

Draft

zhangxiaolei123456 mentioned this pull request Jul 26, 2026

[DSpark PD 1/3] Add hidden-state transfer primitives and draft injection #32422

Closed

5 tasks

learning-sketch mentioned this pull request Jul 28, 2026

[PoC] Minimal DSpark PD handoff with Prefill PP (no local DSPARK) learning-sketch/sglang#4

Draft

4 tasks

JustinTong0323 mentioned this pull request Jul 31, 2026

docs: add DSpark deployment guide #33081

Draft

bobcao3 mentioned this pull request Aug 1, 2026

spec: DSPARK PD-disaggregation via native draft-KV transfer #33204

Closed

amd-bot mentioned this pull request Aug 26, 2026

[CI Monitor] Daily Report - 2026-08-26 bingxche/sglang-ci-bot#178

Open

bingxche mentioned this pull request Sep 1, 2026

[CI Monitor] Daily Report - 2026-09-01 bingxche/sglang-ci-bot#182

Open

bingxche mentioned this pull request Sep 13, 2026

[CI Monitor] Daily Report - 2026-09-13 bingxche/sglang-ci-bot#194

Open

Navigation Menu

Uh oh!

[Spec] DSpark support prefill/decode disaggregation - #31466

[Spec] DSpark support prefill/decode disaggregation#31466
zhangxiaolei123456 wants to merge 104 commits into
sgl-project:mainfrom
bytedance-iaas:deepseev_v4_dpsark_pd_dev

zhangxiaolei123456 commented Jul 16, 2026 •

edited by github-actions Bot

Loading

Uh oh!

yz-tang commented Jul 17, 2026

Uh oh!

zhangxiaolei123456 commented Jul 17, 2026

Uh oh!

hnyls2002 left a comment

Uh oh!

hnyls2002 Jul 20, 2026

Uh oh!

dongyibo commented Aug 6, 2026

Uh oh!

zhangxiaolei123456 commented Aug 6, 2026

Uh oh!

Eviannn commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

zhangxiaolei123456 commented Jul 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

What Changed Compared With main

DSpark Hidden State PD Transfer

Decode-Side Draft Bootstrap

PP-Aware Prefill Support

Hidden Buffer and Transfer Resource Management

Metadata and Speculative Decode Integration

Streaming Hidden Chunk Transfer and Release Semantics

Why These Changes Are Needed

Compatibility

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

CI States

Uh oh!

yz-tang commented Jul 17, 2026

Uh oh!

zhangxiaolei123456 commented Jul 17, 2026

Uh oh!

hnyls2002 left a comment

Choose a reason for hiding this comment

Uh oh!

hnyls2002 Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

dongyibo commented Aug 6, 2026

Uh oh!

zhangxiaolei123456 commented Aug 6, 2026

Uh oh!

Eviannn commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zhangxiaolei123456 commented Jul 16, 2026 •

edited by github-actions Bot

Loading