fix(dflash): isolate replicated draft cache groups

[voipmonitor]

Status

Implemented. Qualified in the GLM-5.3-Flash TP4/DCP4 integration described below.

Behavior

DCP-replicated speculative cache groups are grouped independently from DCP-sharded target cache groups. Each partition then uses the existing vLLM grouping logic.

This preserves the native MLA and Mamba page geometry of a hybrid target while preserving the DCP1 allocation geometry of a replicated DFlash draft cache. It prevents cross-model page-size unification from padding or resizing an incompatible target cache.

The behavior changes only a cache specification containing both dcp_replicated=True and non-replicated groups. Cache specifications with one replication mode retain the existing grouping path.

Dependency

This pull request is stacked on #513 at e5e7bf99182833c6ce25042c29252bbb4107539c. The base branch deps/pr513-dflash-dcp is an exact mirror of that revision so the review diff contains only the two files changed here.

Merge order: #513, then this pull request. Retarget this pull request to dev/jovian-judgement after #513 is present there.

Reproducer

With #513 and without this commit:

• an MLA target plus a replicated sliding-window draft was page-size-unified instead of grouped independently;
• an MLA-plus-Mamba target plus an incompatible replicated draft page raised NotImplementedError: Layer model.mla: page size is not divisible by the maximum page size and cannot be padded during KV-cache grouping;
• GLM-5.3-Flash DCP4 startup stopped during CUDA-graph memory profiling for the same cache geometry.

The regression tests model both the MLA-only and MLA-plus-Mamba cases.

Validation

Exact pull-request tree dd361d82253fc1df223ec02e15027cc7c0fc4f8f:

• 2 passed for the two added grouping regressions in a source-overlay test image whose staged vLLM tree matched that hash.
• The broader CPU-only file run produced 66 passed, 22 failed; every failure was an existing test constructing DeviceConfig(device="auto") without an available device. No implementation or assertion failure was observed outside that environment limitation.

Composed GLM-5.3-Flash integration tree 89e9e3c6cda543e53b950e4034f115dbd7543ec0:

• 113 passed across DFlash replicated-cache, DCP compatibility, mixed slot-mapping, and KV-cache grouping tests.
• Source-locked image sha256:5d98788110b15f3e14c5668250d846da2cf819cf850d436d26aca1a1c5a52737 completed model loading, KV allocation, B12X warmup, and target plus DFlash2 CUDA graph capture without source bind mounts.
• Hardware: four RTX PRO 6000 Blackwell GPUs, physical devices 4–7; TP4, DCP4, interleave 4.
• Target: local-inference-lab/GLM-5.3-Flash-NVFP4@520de24eabf507659eaef7c70f14fd584527facc.
• Draft: local-inference-lab/GLM-5.3-Flash-DFlash2-MXFP8@b6d33aa93fc1ac5b23a88251a1c0ce0bfe2ad17c.
• B12X: d56c1163b6e019d828ed24f135c2efd05fdca6ea.
• Runtime backends: B12X target attention, NVFP4 MoE, MXFP8 draft linear kernels, and PCIe all-reduce; FlashAttention draft attention over replicated KV.
• CUDA graphs: full decode capture through concurrency 16 for target and DFlash2. GLM GDN selects FULL_DECODE_ONLY, so prefill remains outside a full graph.
• 30-second decode smoke: 153.3 tok/s at CC1 and 888.6 aggregate tok/s at CC16. Acceptance varied between runs; normalized engine throughput remained 62.2–62.4 steps/s at CC1 and reached 333.1 steps/s at CC16.
• 32,320-token prefill over a 30-second window: 10,529 client prompt tok/s.

Review requirement

OpenAI Codex assisted with analysis, implementation, testing, and pull-request text. Every changed line requires human review before merge.

[coderabbitai]

Warning

Review limit reached

Next included review available in 59 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ceaf6519-4d21-4577-8df4-9a19f44b761c

📥 Commits

Reviewing files that changed from the base of the PR and between e5e7bf9 and 00c4634.

📒 Files selected for processing (2)

• tests/v1/core/test_kv_cache_utils.py
• vllm/v1/core/kv_cache_utils.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

The cache-group isolation in #519 is the correctness prerequisite for the independently reviewed capacity optimization in #520. Pull request #520 keeps backend-selected DFlash geometry during model KV-spec construction, then aligns the replicated sliding-window group with the sharded target physical block only when the planner proves that pool stride and scheduler LCM do not grow. On the qualified GLM-5.3 TP4/DCP4 runtime, that changes group-aware 12 GiB capacity from 5,688,681 to 5,974,904 tokens (+5.03%) without a measurable normalized decode or 32k prefill regression.

[voipmonitor]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.