Skip to content

feat: support Kimi-K3 DSPARK PD decode radix cache - #7

Merged
zhaozx-cn merged 1 commit into
zhaozx-cn:a5-k3-0828from
qybnb:feature/k3-dspark-graph-decode-radix-cache
Sep 3, 2026
Merged

zhaozx-cn merged 1 commit into
zhaozx-cn:a5-k3-0828from
qybnb:feature/k3-dspark-graph-decode-radix-cache

Conversation

@qybnb

@qybnb qybnb commented Sep 1, 2026 •

Copy link
Copy Markdown

Summary

  • allow the narrow Kimi-K3 + DSPARK + PD decode-radix combination while keeping the prompt-end KDA/conv state from Prefill authoritative
  • reuse Decode-local target/draft token KV prefixes and transfer only the missing KV suffix from Prefill
  • keep the Ascend/NPU graph path from selecting the CUDA-only Kimi-K3 verification backend
  • add opt-in runtime diagnostics and correct Decode radix cache-pool accounting

State and transfer contract

For prompt length N and Decode-local radix hit L:

  • Decode radix owns the per-token target MLA/draft KV prefix [0, L)
  • Prefill computes or restores state through N
  • Prefill transfers target/draft KV only for [L, N)
  • Prefill independently transfers the complete prompt-end KDA/conv state S_N
  • Decode does not index KDA checkpoints in its radix tree

The path is intentionally limited to Kimi-K3 + DSPARK with DCP=1 and device-resident cache. HiCache, other speculative algorithms, other hybrid-SSM models, and draft prefetch are outside this change.

Runtime diagnostics

Set SGLANG_PD_RADIX_DIAGNOSTIC_LOG=1 on Prefill and Decode to emit rank-0 diagnostics for Decode prefix matching, Prefill KV suffix transfer, prompt-end KDA/conv state payloads, and final Decode cache accounting. The switch is disabled by default.


CI States

Latest PR Test (Base): ⚠️ Run #33570754761
Latest PR Test (Extra): ⚠️ Run #33570754543
Latest PR Test (AMD ROCm 7.2): ⚠️ Run #33570754805

@github-actions github-actions Bot added documentation Improvements or additions to documentation npu labels Sep 1, 2026
@qybnb
qybnb force-pushed the feature/k3-dspark-graph-decode-radix-cache branch from f3de12e to b1ed09f Compare September 1, 2026 22:14
@qybnb
qybnb force-pushed the feature/k3-dspark-graph-decode-radix-cache branch from b1ed09f to 1a67755 Compare September 1, 2026 23:22
@zhaozx-cn
zhaozx-cn merged commit b541381 into zhaozx-cn:a5-k3-0828 Sep 3, 2026
2 checks passed
zhaozx-cn added a commit that referenced this pull request Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation npu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants