[Venkat2811]

Stack 4/4. Depends on #41694. Targets main, so the full diff is cumulative. Incremental diff. Refreshed onto 4cca7fcd3e on 16 October; remains draft. The reviewed recurrent-file dependency was replayed locally; #41692 itself was not changed.

Motivation

Decode must honor the prefix it promised to prefill without replacing the live recurrent state received through P-to-D transfer. Enable role-local DCP L3 reuse while preserving that contract.

This work grew out of experiments with Kimi-K3 prefill/decode serving configurations.

Modifications

• Restore canonical cached state separately from live request state; reuse main's capacity and lock ownership paths.
• Reserve outstanding checkpoint slots and advertise only admitted prefetch coverage.
• Require effective wait_complete on DCP decode and reject decode KV offload with DCP storage. Runtime attachment forwards the selected policy through both controller layers.

Live-state preservation and admission fixes also affect DCP1. Roles can have different layouts, but each reads compatible role-local objects. Persisted cross-layout conversion is unsupported. Losing promised data fails the request safely; it does not turn an incomplete restore into a hit. Broader timeout/liveness redesigns are outside this PR.

Accuracy Tests

Before = hybrid stage 39e4d07b34, which rejects DCP P/D storage. Final stage = ed850058f8. Kimi-Linear uses TP4/DCP1 prefill and TP4/DCP4 decode on one eight-GPU B300 node, with live RDMA transfer and separate role-local namespaces.

Scenario	Before	After
File L3, live P/D	Rejected	22 requests pass: main cache-off control, role-local reuse, three concurrent restores, cancellation during I/O, lost promised state and recovery
Mooncake DRAM L3, live P/D	Rejected	16 requests pass, including main control, fresh readers and missing-state fallback; both roles attach through the real HTTP API
Start with default timeout, attach using wait_complete	Rejected incorrectly by this draft's old controller check	Real controller regression is red before the fix and green after; HTTP attachment and inference pass
HiCache SSD offload via file L3	Rejected	Physical pressure verified on the hybrid parent; not repeated as live P/D in this refresh
Mooncake SSD offload	Rejected	Disk-only readback verified on the hybrid parent; not repeated as live P/D in this refresh

File P/D ran before the final attach-policy fix; that fix changes runtime attachment, not its startup configuration. Mooncake P/D uses the final runtime code. Compared output IDs match the relevant control. The largest concurrent/retry logprob delta is 0.01291 (threshold 0.2); fresh-reader deltas against the matching warm control are zero. A fresh process can acquire a 64-token shared device prefix during successive requests, so P/D cache counters are not represented as universally zero L1.

25 CPU files plus two scheduler-attach tests pass with CUDA hidden. Four Python/Rust GPU restore tests pass, including preserving live state. Reservation and effective-policy regressions have failing-before/passing-after evidence. Pre-commit and docs checks pass. Hosted CI, B200, Kimi-K3, cross-node P/D, speculative L3 and live P/D SSD pressure were not rerun.

Run the committed regression tests

Use an x86 NVIDIA host. The GPU results below were obtained on B300, with driver 580.178.04. This container pins the tested dependency image. These are portable equivalents of the commands run in a Kubernetes pod, not a claim that this Docker wrapper was itself executed. Model tests download pinned public snapshots; allow approximately 100 GB for Kimi-Linear weights.

docker run --rm -it --gpus all --ipc=host --network=host \
  --cap-add IPC_LOCK lmsysorg/sglang@sha256:0a1ba33aa4e032b8c5bcebdc69c45664074c032074add4f7db01629ec1fcc381 bash

# Inside the container:
git clone --filter=blob:none https://github.com/Venkat2811/sglang.git /tmp/sglang-check
cd /tmp/sglang-check
git checkout --detach ed850058f8397ebd371856562dd9ed052110642d
python -m pip install parameterized==0.9.0
export PYTHONPATH="$PWD/python:$PWD"

CUDA_VISIBLE_DEVICES=999 python test/registered/unit/disaggregation/test_decode_queue_cleanup.py
CUDA_VISIBLE_DEVICES=999 python test/registered/unit/mem_cache/test_mamba_capacity.py
CUDA_VISIBLE_DEVICES=999 python test/registered/unit/mem_cache/test_hicache_dcp_storage_guards.py
CUDA_VISIBLE_DEVICES=999 python test/registered/unit/managers/test_scheduler_hicache_attach.py

# Actual H2D state copies, both tree cores, cancellation and live-state checks.
CUDA_VISIBLE_DEVICES=0 python test/registered/unit/mem_cache/test_hybrid_l3_restore.py

The live P/D campaign was a separate local integration harness, not a registered test in this diff. It used page size 64, BF16 weights, FP8 KV, context 4096, chunked prefill 1024, tokenspeed_mla, Triton linear decode, 10 GB HiCache, page_first, write_through, eager prefill and decode radix caching. The roles use separate file directories or Mooncake extra_backend_tag values. Decode needs --hicache-storage-prefetch-policy wait_complete. A working RDMA setup is required; the local run used DMA-BUF and explicit IB devices. The commands above validate the committed contracts, not that full live deployment or its fault injection.