[sammysun0711]

Motivation

This PR aim to enable MiMO-V2.5-Pro PD Disaggregated Deployment with vectorized_5d KV cache and provide MiMo TP8 1P1D guide and scripts under evaluation/mimo_pd_scripts

Modifications

• Fix Mooncake PD KV transfer for SGLANG_AITER_KV_CACHE_LAYOUT=vectorized_5d:
  get_contiguous_buf_infos() treated buf[0] as one token (NHD) and used nbytes * page_size, which overshoots registered MRs and blacklists the decode session. vectorized_5d KV is page-major, use page-major item_len for vectorized_5d Mooncake KV transfer as buf[0].nbytes.

• Add MiMo TP8 1P1D launch/bench scripts and guide under evaluation/mimo_pd_scripts

Accuracy Tests

python3 -m pytest -q test/manual/test_disagg_page_nbytes.py: 3 passed.

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[Copilot]

🟡 Changes recommended

The decode launcher can terminate before startup, several scripts mask failures, and the regression fix lacks coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Fixes Mooncake KV transfer sizing for page-major vectorized_5d caches and adds MiMo-V2.5-Pro PD deployment tooling.

Changes:

• Corrects RDMA page-byte calculations.
• Adds TP8 prefill/decode launch and benchmark scripts.
• Documents the 1P1D deployment workflow.

File summaries

File	Description
python/sglang/srt/mem_cache/memory_pool.py	Calculates layout-aware KV page sizes.
evaluation/mimo-v2.5-pro-evaluation-guide.md	Links the PD guide.
evaluation/mimo_pd_scripts/run_benchmark_mimo_pro_prefill.sh	Adds prefill benchmarking.
evaluation/mimo_pd_scripts/run_benchmark_mimo_pro_decode.sh	Adds decode benchmarking.
evaluation/mimo_pd_scripts/mimo-v2.5-pro-pd-disaggregated-guide.md	Documents deployment steps.
evaluation/mimo_pd_scripts/launch_tp8_noep_prefill_aiter_mtp.sh	Configures the prefill server.
evaluation/mimo_pd_scripts/launch_tp8_noep_decode_aiter_mtp.sh	Configures the decode server.
evaluation/mimo_pd_scripts/launch_router.sh	Launches the PD router.

Review details

Suppressed comments (1)

evaluation/mimo_pd_scripts/launch_tp8_noep_decode_aiter_mtp.sh:73

• Because this script enables set -u but never initializes MC_CUSTOM_TOPO_JSON, the documented launch exits here with “unbound variable” before starting the decode server whenever that optional Mooncake variable is unset. Render an explicit fallback in the status line.

echo "PD decode: mooncake, ib-device=${DISAGGREGATION_IB_DEVICE}, gid-index=${MC_GID_INDEX}, host-ip=${SGLANG_HOST_IP}, topo=${MC_CUSTOM_TOPO_JSON}"

• Files reviewed: 8/8 changed files
• Comments generated: 7
• Review effort level: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.