[Kimi-K3][DCP] Publish prefill KV directly in MLA layout

[GirasoleY]

Summary

Replace the direct DCP prefill KV path's rank-major gather, local materialization, and Python reorganization with a multicast publisher that writes valid rows directly into the compact request-major MLA layout.

• Publish separate contiguous kv_c and k_pe planes using per-chunk destination-row metadata.
• Skip DCP padding rather than copying and removing it afterward.
• Keep two ping-pong slots per DBO ubatch; different ubatches never alias.
• Reduce direct-path local scratch from rank-major gathered storage to this rank's padded rows.
• Preserve the NCCL all-gather/reorganization fallback when multicast, dtype, or plane alignment is unsupported.

Buffer lifetime

The returned tensors alias symmetric memory. Within an ubatch, consumption stays on the same stream. A gather on the alternate slot is the all-rank rendezvous that makes the previous slot reusable; after the final context chunk, the tensor-parallel output collective is the post-consumption rendezvous before the next layer/forward resets to slot 0. DBO ubatches own disjoint slot pairs.

The distributed test covers both transitions without an explicit barrier in the critical sequence, including asymmetric delayed consumption.

Validation

• Pre-commit suite: Ruff lint/format, mypy, clang-format 21.1.2, typos, SPDX, config validation, and repository checks.
• Unit coverage for exact destination maps, continuation/padding handling, per-request coverage, DCP-group geometry, direct/fallback scratch sizing, and sparse-MLA compatibility.
• Distributed coverage for BF16/FP16/FP8 byte-exact publication, zero-copy plane aliases, stale tails, completion/epoch state, DBO isolation, ping-pong lifetime, downstream-collective lifetime, and CUDA graph replay.
• Clean Linux/GB300 source build at 888f83d3; 14/14 context-map tests and the focused four-rank direct-KV publication/lifetime test passed. The PR remains a draft for maintainer review.

Paired benchmark evidence

On the pinned Kimi-K3 prototype tree (0a131869, tree 47c98e9), using identical 8-GPU TP8+EP8+model-SP engines with replicated shared experts and the same controlled 60K–180K prefill workload:

topology	fixed cohort 14 (5 requests) P50 TTFT	max passing offered QPS	first 15s/25s SLA miss
pure TP8	3.130 s	1.15	1.175
direct DCP8, final-layout KV	3.149 s	1.15	1.175

At the 1.15-QPS boundary, DCP8 was 127 ms behind TP8 at P50 while exposing about 7.5x the KV-token capacity. This PR deliberately excludes the benchmark harness, scheduler observability, connector changes, MRV2 block-table fix, and DeepGEMM compatibility fix from that tested tree.

Scope

This builds on the merged DCP implementation from #50484 and preserves the current fused Kimi-K3 chunked-context K/V packing path. No scheduler, model-weight, or benchmark-only changes are included.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @GirasoleY.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.