perf(glm5next): gather full C4 cache for DCP prefill

[voipmonitor]

Purpose

Give GLM5Next B12X C4 prefill a global compressed-key/value view under decode
context parallelism. Each rank normally selects candidates from only its local
KV shard, which reduces selector quality and leaves most C4 candidates
invisible during prefill.

Resulting behavior

When VLLM_B12X_MLA_CKV_GATHER=1, an eligible pure-prefill batch:

• packs each DCP rank's visible FP8 compressed-KV page-tail records;
• gathers the records synchronously into caller-owned workspace;
• applies the B12X C4 selector to the rank-major full cache;
• remaps selected indices to the gathered cache layout;
• supplies global causal lengths and each rank's local query heads; and
• supports multiple requests, chunked prefill, and prefix-cache hits.

Eligibility requires num_decode_tokens == 0. Decode and MTP verification
batches cannot enter this path. The implementation uses one synchronous
workspace slot and does not introduce side-stream prefetch.

Cache page geometry is finalized before memory profiling. B12X plans, packed
capacity, gather workspace, persistent-pool reservation, and cache binding use
the same immutable physical page size. Full-CKV prefill executes eagerly;
CUDA-graph profiling uses ordinary DCP attention, while decode and eligible
serving paths retain PIECEWISE and FULL graphs.

The environment-variable source default is disabled. No B12X source or kernel
is changed.

Review-stack boundary

This pull request is based on #516, which supplies startup-only B12X profiling
warmup safety. Pull request #516 is based on #515, which supplies CUDA-graph
profiling resource lifetime. The lower review stack also contains #531 and
#532, which reuse immutable B12X C4 plans and parallelize packed C4 pool writes.
The head tree for this pull request is
265353fc996f5a788aaac069f58f9484e3d7be41.

The diff in this pull request contains only the GLM5Next synchronous full-CKV
operation contract, immutable page geometry, cache-binding lifecycle, and
associated tests.

Duplicate-work check

Searches covered GLM5Next, B12X, DCP, CKV gathering, and CKV prefetch in both
local-inference-lab/vllm and vllm-project/vllm.

• perf(glm5next): complete B12X CKV support for DCP and MTP #488 overlaps in purpose but combines synchronous CKV gathering with MTP
  routing, DFlash cache ownership, and depth-1 side-stream prefetch. It also
  permits workspace sizing before the hybrid physical page is finalized. This
  pull request isolates the depth-0 pure-prefill contract and fixes page
  geometry before memory profiling.
• feat(dcp): add native depth-N CKV prefetch #160 implements a generic depth-N prefetch design on
  dev/gilded-gnosis. Its main commit is not an ancestor of
  dev/gilded-gnosis or dev/infernal-invocation, and it is used only as a
  design reference here.
• fix(b12x): prewarm full-CKV prefill kernels before KV sizing #271 prewarms full-CKV kernels on dev/gilded-gnosis; it does not provide
  the Jovian runtime path.
• Merged feat(glm5next): support B12X C4 DCP #486 supplies the GLM5Next pooled C4 DCP selector foundation used by
  this implementation.

Verified review findings from #488

Luke Alonso's #488 review notes were checked against
e72447af820d4f996b33710cc0232a3144460722 and minimal reproducers:

1. A provisional 64-token page reserves 131,200 local records for DCP4, 32
   sequences, interleave 4, and a 524,288-token CKV limit. Binding the
   2,304-token hybrid page requires 131,328 records, leaving a depth-1 lane
   608,256 bytes short. This implementation finalizes the page before memory
   profiling and rejects post-allocation geometry changes.
2. VLLM_B12X_MLA_SPEC_EXTEND_AS_DECODE defaults to disabled even though a
   routing test passes an enabled flag directly. Full-CKV eligibility therefore
   derives from the batch's decode-token count and excludes MTP verification.
3. perf(glm5next): complete B12X CKV support for DCP and MTP #488 defaults to depth-1 prefetch without a demonstrated GLM5Next gain over
   synchronous gathering and allocates approximately 1.2 GiB for the second
   lane. This implementation remains synchronous. Side-stream prefetch requires
   independent profiling and qualification.

Source provenance

Substantially transferred commits retain their Git authors:

• Koushik Dutta authored the synchronous full-C4-cache gathering derived from
  [gg-rebased] perf(dcp): split prefill queries and gather selected CKV #120 commit d98b9b245c1430e02090e06edc4a836018f1e727.
• Jack Zampolin authored parallel-configuration scratch sizing, local query
  selection, and GLM5Next packed page-tail fixtures.
• Martin Vit authored immutable pre-profile geometry, selector-metadata
  fixtures, and decode-token eligibility.

Validation

Hardware and model conditions: four NVIDIA RTX PRO 6000 Blackwell Workstation
Edition GPUs, physical device IDs 4, 5, 6, and 7; TP4; DCP4; FP8 target KV;
max_num_batched_tokens=4096; GLM-5.3-Flash-NVFP4 revision
520de24eabf507659eaef7c70f14fd584527facc; composed vLLM head
3609a3db498698314bdc44920cad9f2d25796eb9; and B12X head
0a73e97fe4dd67500999b0152d8a3c41ba238715.

Correctness:

• Six greedy cases produced identical text and token sequences against
  ordinary DCP and perf(glm5next): complete B12X CKV support for DCP and MTP #488 depth 0: 256 tokens, chunked 32,768 tokens, two
  concurrent 32,768-token requests, and two 32,768-token requests sharing a
  30,720-token prefix.
• Maximum selected-token log-probability delta was 0.019841991 against ordinary
  DCP and 0.0041269148 against perf(glm5next): complete B12X CKV support for DCP and MTP #488 depth 0.
• CKV, selector, cache-binding, and CUDA-graph tests: 90 passed, 302 skipped,
  with no implementation failure.
• GPU-worker page-geometry ordering: 3 passed.
• Ruff check, Ruff format check, and git diff --check: passed.

Performance used llm-decode-bench 0.4.30, concurrency 1, its 32k standalone
prefill workload with 32,320 measured prompt tokens, one output token, unique
cold-prefix requests, and a 30-second client measurement window:

Configuration	Prompt throughput samples	Median
Synchronous full-CKV	9,602, 9,598, 9,567 tok/s	9,598 tok/s
Ordinary DCP on the same tree	8,731, 8,758, 8,755 tok/s	8,755 tok/s

The adjacent median gain is 9.63%. The measured gain applies to non-speculative
serving at the declared 4,096-token scheduler batch limit; it is not a claim
for a different batch limit or speculative mode.

An earlier source-isolation qualification compared this implementation with
#488 depth 0 under the same model, B12X revision, hardware, and 4,096-token
scheduler batch limit. Their median difference was 0.15%, which establishes
performance parity with #488 depth 0 rather than a speedup over #488.

AI assistance disclosure

OpenAI Codex assisted with source analysis, implementation, tests, runtime
qualification, and pull-request preparation. A human maintainer must review
every changed line and understand and defend the behavior before merge.

[coderabbitai]

Warning

Review limit reached

Next included review available in 48 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4e71fff8-8314-4ee7-8f79-4787a2a17e77

📥 Commits

Reviewing files that changed from the base of the PR and between db0f144 and 92d807a.

📒 Files selected for processing (10)

• tests/v1/attention/test_b12x_sparse_mla_api.py
• tests/v1/attention/test_mla_backends.py
• tests/v1/worker/test_workspace.py
• vllm/envs.py
• vllm/model_executor/layers/attention/mla_attention.py
• vllm/model_executor/layers/attention_layer_base.py
• vllm/platforms/interface.py
• vllm/v1/attention/backend.py
• vllm/v1/attention/backends/mla/b12x_mla_sparse.py
• vllm/v1/worker/workspace.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[voipmonitor]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.