[BJWang-ant]

issue：#39700
base pr：#40323

Motivation

Enable DeepSeek V4.1 prefill nodes to use interleave context parallelism (CP) together with breakable CUDA graphs (BCG), including prefill/decode disaggregated serving.
This allows prefill computation to run across CP ranks while preserving graph replay correctness for batches with different prompt lengths and request layouts.

Modifications

• Integrate interleave CP with BCG capture and replay. Keep CP-local embeddings, positions, token IDs, and live-token metadata consistent with the captured buffers. Align token IDs with the layouts used by TP-MoE and MegaMoE.
• Handle variable-length batches and capture padding correctly. Select replay buckets using per-rank capacity, preserve captured metadata addresses, and process only live tokens when updating low-ratio compressed state.
• Preserve global context and output ordering. Compute Engram hashes from global prompt context, gather CP outputs in token order, and trim padding before the logits tail, including DSPARK auxiliary hidden states.
• Support prefill CP for vision-enabled V4.1 models. Prepare multimodal embeddings before CP sharding. Text requests can use BCG; requests containing image inputs fall back to eager CP.

gms8k preicition test:

perf：

hardware

B200 && H200

---

CI States

Latest PR Test (Base): ⏳ Run #42168284764
Latest PR Test (Extra): ❌ Run #42168284579
Latest PR Test (AMD ROCm 10): ⏳ Run #42168284814

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[SYChen123]

Fix: CP BCG megamoe path with vision inputs is now supported.

Image dataset test is ok.

[BJWang-ant]

/rerun-test test_dsv41_shared_prefill_kv.py test_deepseek_v41_pd_cp_dspark.py test_deepseek_v4_flash_fp4_h200.py test_deepseek_v4_flash_fp4_b200.py test_deepseek_v4_flash_fp4_megamoe_b200.py test_deepseek_v41_flash_gb300_dp4ep4.py test_deepseek_v41_flash_gb300_encoder_bcg.py test_deepseek_v41_flash_gb300_tp4ep4.py test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

[github-actions]

🚀 8-gpu-b300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_pd_cp_dspark.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v4_flash_fp4_h200.py

🚀 4-gpu-b200 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v4_flash_fp4_b200.py
cd test/ && python3 registered/e2e/models/test_deepseek_v4_flash_fp4_megamoe_b200.py

🚀 4-gpu-gb300 (4 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_dp4ep4.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_encoder_bcg.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_tp4ep4.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

⛔ test_dsv41_shared_prefill_kv.py: No test file found matching test_dsv41_shared_prefill_kv.py under test/registered/ or python/sglang/multimodal_gen/test/.

[BJWang-ant]

new base ci pass : https://github.com/sgl-project/sglang/actions/runs/39376452741

[Oasis-Git]

PR ready. Will merge later

[BJWang-ant]

[BJWang-ant]

Required checks failed on 85af6fa; this PR cannot be merged until they pass.

Lint (run) Run pre-commit run --all-files locally and push the result. The run log shows any other failing step.