[II] Replicate Kimi DSpark cache under target DCP - #364
voipmonitor wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The behavior implemented by this pull request is preserved in vLLM #389 as commit |
Behavior
Kimi-K3 DSpark may run with a B12X MLA target that uses decode context parallelism. The external draft keeps a complete DCP1 KV cache on every target rank unless
VLLM_DCP_SHARD_DRAFT=1explicitly requests draft sharding.The speculative configuration accepts target DCP only when the external Kimi draft selects
B12X_MLA; other attention backends retain the DCP1 requirement.Technical reason
Kimi-K3 target KV is partitioned by DCP, while an external DSpark model executes independently on every target rank. Marking the non-causal draft MLA cache as
dcp_replicatedgives each rank complete draft context without changing the target cache layout.Compatibility
VLLM_DCP_SHARD_DRAFT=1preserves explicit draft-cache sharding.Validation