Fix DCP sparse indexer decode metadata - #4
Conversation
GLM/Kimi DCP decode uses an uncompressed MLA indexer cache (compress_ratio == 1), but the sparse indexer still has to write and read through DCP rank-local cache pages. The previous path only remapped slots when compress_ratio > 1, so DCP4 wrote the indexer K cache with global slot ids while B12X attention consumed it through a DCP-local page table. Use DCP-local slot mapping whenever decode-context parallelism is active, and feed DCP-local seq_lens to the decode sparse-indexer metadata. For MTP/native expanded decode rows, compute the per-token lengths from global seq_lens first, then convert the expanded lengths to DCP-local lengths so query_start_loc arithmetic remains correct. Validated on GLM-5.1 NVFP4 DCP4 nomtp with B12X_MLA_SPARSE, V2 model runner, FULL+PIECEWISE CUDA graphs, 50k context smoke, and cc1 decode bench.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughDeepseekV32IndexerMetadataBuilder now routes global decode sequence lengths into decode tensor preparation, enabling each expansion path to apply DCP-local conversion consistently. Buffer allocation conditions broaden to support DCP paging without compression, and downstream DCP conversion is guarded to prevent redundancy. ChangesDCP-local sequence length support in decode tensor preparation
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
Summary
Fix GLM/Kimi DCP decode sparse-indexer metadata for the uncompressed indexer-cache path (
compress_ratio == 1).DCP writes the indexer KV cache through rank-local pages, but the previous path only built a remapped slot mapping when
compress_ratio > 1. For GLM/Kimi DCP4 this meant the sparse indexer could write with global slot ids while B12X attention consumed the selected indices through a DCP-local page table, causing incoherent long-context decode.This patch:
dcp_world_size > 1, not only for compressed KV paths.seq_lensto the sparse-indexer metadata when available.seq_lensfirst, then converts those expanded lengths to DCP-local lengths soquery_start_locarithmetic remains correct.Validation
Validated on GLM-5.1 NVFP4 DCP4 no-MTP with
B12X_MLA_SPARSE, V2 model runner, DCP4ag_rs, and FULL+PIECEWISE CUDA graphs.Commands/results:
python3 -m py_compile vllm/v1/attention/backends/mla/indexer.pygit diff --check lil/main..HEADpython3 /mnt/test.py --port 5331 --model GLM-5.1 --max-tokens 180 --quiet --json-summary -: coherent,chinese_count=0python3 /mnt/test.py --port 5331 --model GLM-5.1 -c 50000 --max-tokens 220 --quiet --json-summary -: coherent,chinese_count=0, prompt tokens31,876, TTFT12.43spython3 /root/llm-inference-bench/llm_decode_bench.py --port 5331 --concurrency 1 --contexts 0k --max-tokens 2048 --skip-prefill:58.1 tok/s, TTFT/ITL248/17 msRuntime image used for validation:
voipmonitor/vllm:cu132-vllm611a842-b12xf9226c-a16nativew4a16-20260606Only
vllm/v1/attention/backends/mla/indexer.pywas overlaid for the final validation run.Summary by CodeRabbit