Skip to content

[DeepSeek-V4][DSpark] Handle zero-length CUDA graph padding in online C128 planner - #35848

Open
BJWang-ant wants to merge 2 commits into
sgl-project:mainfrom
BJWang-ant:fix_c128_online_v4_extend_len
Open

BJWang-ant wants to merge 2 commits into
sgl-project:mainfrom
BJWang-ant:fix_c128_online_v4_extend_len

Conversation

@BJWang-ant

@BJWang-ant BJWang-ant commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Motivation

Fix online C128 compression for DeepSeek-V4 with DSpark compact ragged verification and CUDA graph enabled.

The issue can be reproduced with:

SGLANG_OPT_USE_ONLINE_COMPRESS=1
SGLANG_EXPERIMENTAL_ONLINE_C128_DSPARK=1
SGLANG_RAGGED_VERIFY_MODE=compact

DSpark compact verification uses ragged verify lengths. During CUDA graph replay, the request batch is padded to the selected graph bucket size. Rows that do not correspond to active requests are represented by:

extend_len = 0

These padded rows contain no query tokens and should not generate online C128 compression or state-pool write plans.
Previously, _plan_prefill_partial required every row to satisfy:

0 < extend_len && extend_len <= seq_len

As a result, the first CUDA graph replay containing padded DSpark rows failed with a RuntimeCheck.
This was not normally observed in the Eagle/MTP path because its verification layout uses a fixed number of draft tokens for every request row. DSpark compact verification, however, has variable verify lengths and zero-length graph padding rows.
Changes

  • Allow extend_len == 0 in the online C128 prefill planner.
  • Skip zero-length padded rows because they own no query tokens and must not produce PlanC, PlanW, or state-pool entries.
  • Preserve validation that extend_len must not exceed seq_len.
  • Add a detailed error message containing the invalid lengths and batch row index.
    This PR is bound to PR32035 and needs to be tested on PR32035.[DeepSeek-V4]support dspark c128 online compressor #32035

CI States

Latest PR Test (Base): ❌ Run #32686880436
Latest PR Test (Extra): ❌ Run #32686880377
Latest PR Test (AMD ROCm 7.2): ❌ Run #32686880437

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant