Skip to content

Fix DCP sparse indexer decode metadata - #4

Merged
lukealonso merged 1 commit into
mainfrom
codex/dcp4-indexer-clean-20260606
Jun 7, 2026
Merged

Fix DCP sparse indexer decode metadata#4
lukealonso merged 1 commit into
mainfrom
codex/dcp4-indexer-clean-20260606

Conversation

@voipmonitor

@voipmonitor voipmonitor commented Jun 6, 2026

Copy link
Copy Markdown

Summary

Fix GLM/Kimi DCP decode sparse-indexer metadata for the uncompressed indexer-cache path (compress_ratio == 1).

DCP writes the indexer KV cache through rank-local pages, but the previous path only built a remapped slot mapping when compress_ratio > 1. For GLM/Kimi DCP4 this meant the sparse indexer could write with global slot ids while B12X attention consumed the selected indices through a DCP-local page table, causing incoherent long-context decode.

This patch:

  • Builds DCP-local slot mapping whenever dcp_world_size > 1, not only for compressed KV paths.
  • Feeds DCP-local decode seq_lens to the sparse-indexer metadata when available.
  • For expanded/native MTP decode rows, computes per-token lengths from global seq_lens first, then converts those expanded lengths to DCP-local lengths so query_start_loc arithmetic remains correct.

Validation

Validated on GLM-5.1 NVFP4 DCP4 no-MTP with B12X_MLA_SPARSE, V2 model runner, DCP4 ag_rs, and FULL+PIECEWISE CUDA graphs.

Commands/results:

  • python3 -m py_compile vllm/v1/attention/backends/mla/indexer.py
  • git diff --check lil/main..HEAD
  • python3 /mnt/test.py --port 5331 --model GLM-5.1 --max-tokens 180 --quiet --json-summary -: coherent, chinese_count=0
  • python3 /mnt/test.py --port 5331 --model GLM-5.1 -c 50000 --max-tokens 220 --quiet --json-summary -: coherent, chinese_count=0, prompt tokens 31,876, TTFT 12.43s
  • python3 /root/llm-inference-bench/llm_decode_bench.py --port 5331 --concurrency 1 --contexts 0k --max-tokens 2048 --skip-prefill: 58.1 tok/s, TTFT/ITL 248/17 ms

Runtime image used for validation:

voipmonitor/vllm:cu132-vllm611a842-b12xf9226c-a16nativew4a16-20260606

Only vllm/v1/attention/backends/mla/indexer.py was overlaid for the final validation run.

Summary by CodeRabbit

  • Bug Fixes
    • Enhanced distributed context processing in Deepseek model attention mechanisms to ensure consistent behavior and correct sequence length computation across various configuration settings.

GLM/Kimi DCP decode uses an uncompressed MLA indexer cache (compress_ratio == 1), but the sparse indexer still has to write and read through DCP rank-local cache pages. The previous path only remapped slots when compress_ratio > 1, so DCP4 wrote the indexer K cache with global slot ids while B12X attention consumed it through a DCP-local page table.

Use DCP-local slot mapping whenever decode-context parallelism is active, and feed DCP-local seq_lens to the decode sparse-indexer metadata. For MTP/native expanded decode rows, compute the per-token lengths from global seq_lens first, then convert the expanded lengths to DCP-local lengths so query_start_loc arithmetic remains correct.

Validated on GLM-5.1 NVFP4 DCP4 nomtp with B12X_MLA_SPARSE, V2 model runner, FULL+PIECEWISE CUDA graphs, 50k context smoke, and cc1 decode bench.
@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c94f244b-ef08-4e10-ab13-a2425dc35360

📥 Commits

Reviewing files that changed from the base of the PR and between 8f6c0a7 and 95f53f4.

📒 Files selected for processing (1)
  • vllm/v1/attention/backends/mla/indexer.py

📝 Walkthrough

Walkthrough

DeepseekV32IndexerMetadataBuilder now routes global decode sequence lengths into decode tensor preparation, enabling each expansion path to apply DCP-local conversion consistently. Buffer allocation conditions broaden to support DCP paging without compression, and downstream DCP conversion is guarded to prevent redundancy.

Changes

DCP-local sequence length support in decode tensor preparation

Layer / File(s) Summary
Buffer allocation and compression condition broadening
vllm/v1/attention/backends/mla/indexer.py
Compressed slot mapping buffer allocation and compressed seq lens computation conditions are expanded from compress_ratio > 1 to compress_ratio > 1 or dcp_world_size > 1, enabling graph-stable slot mapping when DCP paging occurs without compression.
Decode tensor preparation with DCP-local seq lens support
vllm/v1/attention/backends/mla/indexer.py
_prepare_decode_tensors gains an optional global_seq_lens parameter. All three expansion paths (uniform decode, variable decode-length, and native MTP) now accept this parameter as the base for expansion and apply get_dcp_local_seq_lens conversion when DCP is enabled before storing into decode_seq_lens_buffer.
Decode metadata build and DCP-local seq lens selection
vllm/v1/attention/backends/mla/indexer.py
The decode metadata build logic now computes global_decode_seq_lens and conditionally derives dcp_local_seq_lens from common_attn_metadata.dcp_local_seq_lens when compress_ratio == 1 and DCP is enabled. It then passes the appropriate seq lens to _prepare_decode_tensors via the new global_seq_lens parameter.
DCP conversion guard tightening
vllm/v1/attention/backends/mla/indexer.py
A later DCP conversion branch is tightened from elif self.dcp_world_size > 1: to elif self.dcp_world_size > 1 and dcp_local_seq_lens is None:, preventing redundant DCP-local conversion when the build step already selected a DCP-local seq_lens.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Fix DCP sparse indexer decode metadata' directly addresses the main change in the pull request—fixing DCP sparse indexer metadata handling in the MLA indexer.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/dcp4-indexer-clean-20260606

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@lukealonso
lukealonso marked this pull request as ready for review June 6, 2026 18:48
@lukealonso

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants