Skip to content

fix(pd): make DSV4 retraction safe - #36185

Closed
jessiewei7 wants to merge 1 commit into
sgl-project:mainfrom
jessiewei7:fix/dsv4-pd-retract-offload
Closed

jessiewei7 wants to merge 1 commit into
sgl-project:mainfrom
jessiewei7:fix/dsv4-pd-retract-offload

Conversation

@jessiewei7

@jessiewei7 jessiewei7 commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add host-pool backup and restore for supported non-speculative
    DeepSeek-V4-Flash-0731 PD decode requests.
  • Persist the SWA window used for backup and require matching FULL/SWA and
    DSV4 sidecar transfer lengths during restore.
  • Abort only the affected request when backup or restore is unsupported.
  • Allow cleanup after retraction before a new radix match, when no lock exists.

Unsupported configurations

Host-pool retraction is supported only for the non-speculative CUDA
DeepSeek-V4-Flash-0731 decode layout with a separate SWA pool and without
HiSparse.

DSPARK/MTP is not eligible for host-pool retraction. With auto selection it
uses the CPU-tensor fallback; if that cannot snapshot the request state, only
the affected request is aborted and the decode scheduler continues.

Validation

  • 23 passed: focused DSV4 retraction and decode-queue tests.
  • Regression coverage: cleanup before a new radix match; the parent code
    reproduces KeyError(None).
  • 8×H20 1P1D PD run with forced retraction: four retractions completed with
    token IDs and text identical to the same-batch non-retracted control.
  • Ruff, isort, Black, codespell, git diff --check, and repository pre-push
    checks passed.

Refs #33385

Restore supported non-speculative DSV4 requests through HiCache
sidecars. Abort unsupported speculative state per request instead of
crashing the decode scheduler.

Refs sgl-project#33385

Signed-off-by: jessiewei7 <jessiewei747@gmail.com>
@hnyls2002

Copy link
Copy Markdown
Collaborator

Thanks @jessiewei7! The decode crash this targets is fixed on main by #43936: a retracted DeepSeek-V4 PD decode request is now sent back to prefill for recompute instead of raising NotImplementedError, and a request that cannot be recomputed (multimodal input, or the none backup backend) is aborted on its own while the scheduler keeps running. Keeping the KV on the host across retraction is still a valid optimization; that path is open in #37239. Closing as superseded; please reopen if I've missed a case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants