Repository navigation
fix(pd): make DSV4 retraction safe - #36185
Closed
jessiewei7 wants to merge 1 commit into
Closed
jessiewei7 wants to merge 1 commit into
jessiewei7 wants to merge 1 commit into
Conversation
Restore supported non-speculative DSV4 requests through HiCache sidecars. Abort unsupported speculative state per request instead of crashing the decode scheduler. Refs sgl-project#33385 Signed-off-by: jessiewei7 <jessiewei747@gmail.com>
jessiewei7
requested review from
ByronHsu,
Duyi-Wang,
HaiShaw,
ShangmingCai,
Ying1123,
alphabetc1,
hanming-lu,
hnyls2002,
huangtingwei9988,
hzh0425,
iforgetmyname,
ispobock,
merrymercy,
ping1jing2,
sogalin,
whybeyoung,
xiezhq-hermann and
yizhang2077
as code owners
August 24, 2026 12:30
4 of 5 tasks
This was referenced Aug 26, 2026
Collaborator
|
Thanks @jessiewei7! The decode crash this targets is fixed on main by #43936: a retracted DeepSeek-V4 PD decode request is now sent back to prefill for recompute instead of raising |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DeepSeek-V4-Flash-0731 PD decode requests.
DSV4 sidecar transfer lengths during restore.
Unsupported configurations
Host-pool retraction is supported only for the non-speculative CUDA
DeepSeek-V4-Flash-0731 decode layout with a separate SWA pool and without
HiSparse.
DSPARK/MTP is not eligible for host-pool retraction. With auto selection it
uses the CPU-tensor fallback; if that cannot snapshot the request state, only
the affected request is aborted and the decode scheduler continues.
Validation
23 passed: focused DSV4 retraction and decode-queue tests.reproduces
KeyError(None).token IDs and text identical to the same-batch non-retracted control.
git diff --check, and repository pre-pushchecks passed.
Refs #33385