[P/D Disagg] DeepSeek-V4 decode radix cache with MTP support - #31097

Open

TobyMint wants to merge 25 commits into

sgl-project:mainfrom

TobyMint:dsv4-decode-radix-cache-mtp-support

Commits on Jul 27, 2026

[P/D disagg] Support device-only SWA decode radix cache

ishandhanani
authored and
TobyMint
committed
f6b36e8
Enable experimental DSV4 decode radix cache

TobyMint
and
zhangxiaolei123456
committed
05272b3
Fix DSV4 decode radix protected prefix length

TobyMint
and
zhangxiaolei123456
committed
98cfabf
Instrument SWA allocator over-free

TobyMint
and
zhangxiaolei123456
committed
d7e0643
Protect DSV4 decode radix prefix from SWA recovery

TobyMint
and
zhangxiaolei123456
committed
0c40d08
Allow DSV4 decode radix EAGLE args

TobyMint
and
zhangxiaolei123456
committed
507e955
Log DSV4 decode radix transfer stats

TobyMint
and
zhangxiaolei123456
committed
df8500d
Avoid duplicate pending frees in SWA allocator

TobyMint
and
zhangxiaolei123456
committed
dbf4323
Avoid DSV4 decode radix reinsertion

TobyMint
and
zhangxiaolei123456
committed
c9f16a5
Cache DSV4 decode radix prompt once

TobyMint
and
zhangxiaolei123456
committed
0b3aa06
Defer DSV4 prompt radix insert until after prebuilt

TobyMint
and
zhangxiaolei123456
committed
1a87224
Support DSV4 decode radix with EAGLE MTP

TobyMint
and
zhangxiaolei123456
committed
5688fb1
Harden DSV4 decode radix speculative args

TobyMint
and
zhangxiaolei123456
committed
171cde0
Single-flight DSV4 decode radix prompt inserts

TobyMint
and
zhangxiaolei123456
committed
104b461
Insert DSV4 prompt after first decode result

TobyMint
and
zhangxiaolei123456
committed
1933c33
Polish DSV4 decode radix prompt donation: full-only leaves, SWA tail free, full match

TobyMint
and
zhangxiaolei123456
committed
f8b24c1
Fix pre-existing issues: add PureSWATokenToKVPoolAllocator, fix disable_piecewise_cuda_graph attribute
TobyMint
committed
c116faa
Fix: handle None pre_alloc_size in DecodeReqToTokenPool (default to 0)
TobyMint
committed
17d8080
Fix: use Range immutably (don't set .end); get_fill_ids() already returns prompt snapshot
TobyMint
committed
1cb9419
Fix lint: import ordering, line length, test location, missing indexer import
TobyMint
committed
4f9821f
Keep SWA insertion margin for decode radix cache
TobyMint
committed
4494dfe
Fix SWA decode admission under cache pressure
TobyMint
committed
87eb10d
Add SWA decode radix cache admission tests
TobyMint
committed
7bfa072
Use direct attribute access for compression_ratios in DSV4 helpers
TobyMint
committed
e840ef2
Use direct attribute access for standard req/tree_cache fields
TobyMint
committed
0539913

Navigation Menu

Uh oh!

[P/D Disagg] DeepSeek-V4 decode radix cache with MTP support - #31097

[P/D Disagg] DeepSeek-V4 decode radix cache with MTP support#31097
TobyMint wants to merge 25 commits into
sgl-project:mainfrom
TobyMint:dsv4-decode-radix-cache-mtp-support

Commits on Jul 27, 2026

[P/D disagg] Support device-only SWA decode radix cache

Enable experimental DSV4 decode radix cache

Fix DSV4 decode radix protected prefix length

Instrument SWA allocator over-free

Protect DSV4 decode radix prefix from SWA recovery

Allow DSV4 decode radix EAGLE args

Log DSV4 decode radix transfer stats

Avoid duplicate pending frees in SWA allocator

Avoid DSV4 decode radix reinsertion

Cache DSV4 decode radix prompt once

Defer DSV4 prompt radix insert until after prebuilt

Support DSV4 decode radix with EAGLE MTP

Harden DSV4 decode radix speculative args

Single-flight DSV4 decode radix prompt inserts

Insert DSV4 prompt after first decode result

Polish DSV4 decode radix prompt donation: full-only leaves, SWA tail free, full match

Fix pre-existing issues: add PureSWATokenToKVPoolAllocator, fix disable_piecewise_cuda_graph attribute

Fix: handle None pre_alloc_size in DecodeReqToTokenPool (default to 0)

Fix: use Range immutably (don't set .end); get_fill_ids() already returns prompt snapshot

Fix lint: import ordering, line length, test location, missing indexer import

Keep SWA insertion margin for decode radix cache

Fix SWA decode admission under cache pressure

Add SWA decode radix cache admission tests

Use direct attribute access for compression_ratios in DSV4 helpers

Use direct attribute access for standard req/tree_cache fields