[HZY-Wade]

Motivation

Follow up on the accuracy-test request in #42973. Extend the existing test_dsa_glm52_pd_kv_shard_hicache.py with GSM8K accuracy coverage for KV sharding and CPU HiCache together.

Modifications

• Use a single base-c / 4-gpu-gb300 registration for the existing reload regression and both accuracy cases. All use P2/D2 with prefill CP2; decode uses GPUs 2–3. Retain the known GB300 fabric transport and exportable allocator: MC_FORCE_MNNVL=true, SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK, staging disabled.
• Preserve the original cold/device-hit/host-reload assertions. Reuse its request and cache-counter helpers.
• Retain the layer-split scoring settings: 1319-question cap, 20 shots, 48 evaluation threads and a 0.935 accuracy floor. The evaluator excludes the 20 demonstrations, leaving 1299 held-out questions. The 32-question Mixed-MoE smoke retains its 0.8 floor and uses target DeepEP with D-side DP2/EP2 plus a TP-only draft.
• Seed the evaluator prefix into write-through HiCache and wait for its D2H backup. Two independent 12,288-token prompts exceed CP2's 16,384 logical GPU slots while fitting in the 4× host cache. Score without flushing the host copy and require H2D growth during evaluation.
• Bound the accuracy fixture's context length to 16,384, decode pool to 65,536 tokens and ordinary decode concurrency/graph size to 48; Mixed-MoE retains an eight-request limit. Disable decode prefix reuse and check decode-side MTP activity.
• Add the standalone KV sharing guide, its navigation entry and updated argument reference. Remove only the duplicated GLM KV sharing section from the P/D overview. The guide distinguishes its deployment recipe from the four-GPU CI fixtures.

Accuracy Tests

Local validation using the project .venv:

• Exactly one CI registration and three active tests with four visible GPUs; hardware tests skip when fewer than four are available.
• All six P/D launch configurations parse, including DP2/EP2 for Mixed-MoE; both sides retain the GB300 transport environment.
• The original reload test body and both accuracy floors remain unchanged.
• Eight mocked control-flow cases pass: success and expected failures for missing H2D reload, insufficient accuracy and missing MTP activity, for both accuracy cases.
• Applicable pre-commit checks and documentation build/link validation pass.

On head e19de3aef0, the four-GB300 hardware job passed the reload regression and the full GSM8K test, scoring 0.9438029253271748 >= 0.935, including its H2D and MTP assertions. Mixed-MoE failed during decode setup: target + draft weights required a static-memory fraction above approximately 0.870, exceeding the inherited 0.85 budget. This change overrides only Mixed-MoE decode to 0.90, retaining the 65,536-token KV cap and accuracy floors; parser checks and pre-commit pass. The corrected Mixed-MoE case still needs hardware verification. The CI time estimate remains 600 seconds.

Speed Tests and Profiling

Test and documentation changes only; no inference runtime changes or performance claims.

Checklist

• Format and pre-commit checks.
• Extend the existing E2E file and validate registration, topology and control flow.
• Update the standalone guide and test-source reference.
• Run the added accuracy cases on the four-GB300 CI runner.

---

CI States

Latest PR Test (Base): ❌ Run #40681717071
Latest PR Test (Extra): ❌ Run #40681717003
Latest PR Test (AMD ROCm 10): ❌ Run #40681717107

[mintlify]

✅ Preview approved

@hzh0425 approved a Mintlify preview for commit 2e97133.

[hzh0425]

/rerun-test -c

[github-actions]

🚀 4-gpu-gb300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_pd_kv_shard_hicache.py

🚀 8-gpu-b300 (1 test): ❌ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_pd_kv_shard_hicache.py

[HZY-Wade]

/rerun-test -c

[github-actions]

⛔ /rerun-test requires cooldown_interval_minutes: 0 in .github/CI_PERMISSIONS.json, or write permission on the repo.

Please ask a maintainer to run this command, or use the normal CI flow.

[hzh0425]

/rerun-test -c

[github-actions]

🚀 4-gpu-gb300 (1 test): ❌ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_pd_kv_shard_hicache.py

[hzh0425]

/rerun-test -c

[github-actions]

🚀 4-gpu-gb300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_pd_kv_shard_hicache.py

[ch-wan]

There's no need to run CI tests for this PR. I have removed its run-ci label. Please ping maintainers on slack for review and merge.

[hzh0425]

/rerun-test -c

[github-actions]

🚀 4-gpu-gb300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_dsa_glm52_pd_kv_shard_hicache.py

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Oct 28, 2026, 11:47 AM

💡 Tip: Enable Automations to automatically generate PRs for you.