[Test] Pin DSpark PD decode draft-input handoff regression

[Phoenix3334]

Motivation

This PR adds regression coverage only for the DSpark disaggregation-decode draft-input handoff. It contains no production code — current main already has the implementation:

• SpeculativeAlgorithm.build_disagg_draft_input() dispatches on is_dspark()
• python/sglang/srt/speculative/dspark_disaggregation.py builds the draft input and, under overlap, seeds future_indices and publishes/stashes the relay state

What is not currently pinned is the invariant that this path must produce a non-None SpecInput before the first decode step. An earlier stack (v0.5.16) lacked the DSPARK branch, so the call fell through to None and the first decode step failed with:

spec_prepare_for_decode
  -> batch.spec_info.prepare_for_decode(batch)
AttributeError: 'NoneType' object has no attribute 'prepare_for_decode'

DSpark, DP attention and disaggregation each passed independently; only the combination exposed this handoff gap. This PR pins that boundary so a future refactor cannot silently regress to DSPARK + PD -> first decode -> spec_info == None.

Design context / roadmaps

• DSpark in SGLang — original DSpark design and serving overview.
• [Tracking] DSpark in SGLang: roadmap #30344 — parent DSpark roadmap.
• [Roadmap] DSpark CUDA Graph correctness and robustness #34297 — DSpark CUDA Graph correctness and robustness roadmap; [Test] Pin DSpark PD decode draft-input handoff regression #34410 is tracked there as a PD-decode integration regression encountered while bringing up the B300 runtime matrix. It is not another CUDA-Graph root-cause fix.

What the regression checks

test/registered/spec/dspark/test_dspark_disaggregation.py (CPU, base-a-test-cpu), scoped strictly to the handoff contract — it is not a scheduler simulation:

1. build_disagg_draft_input() returns a non-None draft input for DSPARK
2. transferred bonus tokens are preserved
3. committed sequence lengths are preserved
4. with overlap enabled, future_indices is seeded from req_pool_indices
5. no stale DSA IndexShare seed is claimed (future_dsa_topk_indices_available is False)
6. sequence-length state is published into the FutureMap
7. the speculative relay payload carrying the bonus tokens is stashed before decode resumes

FutureMap is autospecced because publish() / stash() write device-side relay buffers that a CPU job cannot allocate; the calls themselves are part of the contract, so they are asserted rather than skipped. As a negative control, stubbing out the is_dspark() dispatch makes the builder return None and the regression fails at the first invariant.

Supporting runtime evidence (2026-08-11; not produced by this PR, not current main)

The following is supporting context only. It is an independent B300/B30Z runtime regression on a different stack, not an end-to-end result of this PR's code on current main.

Full stack disclosure:

• SGLang v0.5.16-cu130, torch 2.11.0+cu130, flashinfer 0.6.14, sglang-kernel 0.4.5
• source-equivalent backport of the final reviewer-selected no-init [BugFix] Fix race in c128 prefill plan kernel on ragged extend #32467 patch
• backport of merged Fix DSpark and DP/EP #33098 for DSpark DP-attention draft metadata
• backport of [Spec] Support DSPARK in disaggregation decode mode #31513 for the historical v0.5.16 DSpark disaggregation-decode handoff; [Spec] Support DSPARK in disaggregation decode mode #31513 was open/unmerged during this validation

Current main has since evolved a dedicated DSpark disaggregation path, which is why this PR is regression-only.

Hardware/model: 8x NVIDIA B30Z 275GB (B300-class), DeepSeek-V4-Pro-DSpark, dspark_block_size = 5.

Decode topology:

--tp-size 8 --dp-size 8 --enable-dp-attention --enable-dp-lm-head
--moe-dense-tp-size 1 --moe-a2a-backend none
--moe-runner-backend flashinfer_mxfp4 --disable-flashinfer-autotune
--speculative-algorithm DSPARK
--disaggregation-mode decode --disaggregation-transfer-backend fake
--disaggregation-bootstrap-port 8200 --load-balance-method follow_bootstrap_room
--mem-fraction-static 0.92 --max-running-requests 1024
--cuda-graph-max-bs-decode 128
env: SGLANG_RAGGED_VERIFY_MODE=compact SGLANG_DSV4_FP4_EXPERTS=1

Workload: random input 60000 / output 1000 tokens, no simulated acceptance.

A locally profiled SPS table was used. Its batch-size probes are [1, 8, 16, 32, 64, 96]; therefore the C1024 arm, which reaches per-rank batch 128, is out-of-domain and uses clamped interpolation. Its budget estimate should be treated as optimistic.

Sustained decode-only regression

concurrency	requests	success	failed	gen tok/s	TTFT (s)	TPOT (s)	per-rank peak running
64	640	640	0	3638.96	0.109	0.0132	15
256	2560	2560	0	8340.59	0.172	0.0227	46
512	5120	5120	0	13534.73	0.234	0.0268	87
1024	10240	10240	0	24793.49	0.423	0.0294	128

Totals: 18560/18560 successful requests, 0 client failures, 0 CUDA illegal-address faults, 0 device assertions, 0 scheduler exceptions, post-run /health = 200. CUDA Graph replay was observed throughout; C1024 reached the captured per-rank batch-size 128 tier. Observed acceptance length was ~1.00–1.12.

For compact ragged-verify coverage, the natural workload produced uniform verify lengths, so it does not count as ragged coverage. Using the existing budget control (dspark_force_budget_frac = 0.5) produced a genuinely ragged window with verify lengths spanning 1–6; among fully parsed multi-request scheduling blocks, 39/43 = 90.7% had min(verify_len) != max(verify_len) (for example [4, 3, 3]). No GPU fault, NaN, or verify-budget violation was observed in that window.

Follow-up external-concurrency / admission-pressure sweep

Same server limits (max-running-requests=1024, per-rank CUDA-Graph max batch 128):

external concurrency	requests	success	failed	gen tok/s	TTFT (s)	TPOT (s)	per-rank running peak	prealloc peak
1280	5120	5120	0	20430	6.87	0.0332	128	96
1536	6144	6144	0	21877	19.09	0.0322	128	121
2048	6144	6144	0	21275	35.06	0.0320	128	222
2560	7680	7680	0	23882	53.21	0.0306	128	245

All four arms completed with zero request failures, zero observed GPU correctness faults/retractions/preallocation warnings, CUDA Graph replay retained, and /health = 200.

Interpretation: these higher numbers are external concurrency / admission pressure, not proof of 1280–2560 simultaneously running requests. max-running-requests=1024 remains the active-running ceiling and per-rank running peak stays at 128. The next capacity phase must raise MRR / graph tiers and re-profile SPS before claiming a higher true in-flight capacity.

Scope

This PR does not claim the CPU test replaces PD runtime CI and does not add or change DSpark PD support. It only pins the first-decode handoff invariant that current main already implements.

Related

• DSpark blog post — design / serving overview
• [Tracking] DSpark in SGLang: roadmap #30344 — parent DSpark roadmap
• [Roadmap] DSpark CUDA Graph correctness and robustness #34297 — DSpark CUDA Graph correctness / robustness source-of-truth roadmap
• Fix DSpark and DP/EP #33098 — DSpark DP-attention draft metadata (merged)
• [Spec] Support DSPARK in disaggregation decode mode #31513 — earlier DSpark/DFlash disaggregation-decode implementation and failure analysis (open)
• [BugFix] Fix race in c128 prefill plan kernel on ragged extend #32467 / [Bug] CUDA illegal memory access during compact ragged-verify graph capture (c128 plan kernel race) #32470 — independent producer-side ragged-plan correctness bug and issue
• Fix(DSpark): sync device before CUDA graph capture to avoid JIT race #33795 / [BugFix] Synchronize Breakable CUDA Graph after warmup before capture #34286 — Full / Breakable CUDA-Graph capture-initialization ordering work
• [PD][Spec] Support DFLASH and DSPARK in PD disagg #33411 / spec: DSPARK PD-disaggregation via native draft-KV transfer #33204 — later PD + DSPARK integration work / implementation evolution

---

CI States

Latest PR Test (Base): ❌ Run #34925212412
Latest PR Test (Extra): ❌ Run #34925211967
Latest PR Test (AMD ROCm 10): ❌ Run #34925212214

[Phoenix3334]

@Ying1123 @EanWang211123 @DarkSharpness @Leoyzen could one of you please help trigger CI with /tag-and-rerun-ci when convenient?

This PR only adds a base-a-test-cpu regression for the existing DSpark PD decode draft-input handoff. The current red checks are from the missing run-ci gate; the registered CPU test itself has not run yet. Local validation is green: 2 tests passed, pre-commit passed, and git diff --check is clean.

[Phoenix3334]

2026-08-11 runtime-context update: I refreshed the PR body with the full B300 stack disclosure and the follow-up D-only pressure data.

The PR scope is still unchanged: test-only regression for the existing current-main DSpark PD decode handoff. The runtime data come from a separate v0.5.16 backport stack (#32467 final no-init + merged #33098 + historical/open #31513) and are included only as supporting context.

Newly documented follow-up: external concurrency C1280/C1536/C2048/C2560 all completed with zero request failures and no observed GPU correctness faults, but MRR stayed 1024 and per-rank running peak stayed 128. The body now explicitly labels those points as admission/queue-pressure evidence rather than a claim of >1024 true simultaneously running requests.

The complete root-cause / integration / capacity map is maintained in #34297.