[nvpohanh]

[by Claude Code]

Motivation

The cookbook's Agentic Long-Context Deployment sections (#42726, #43083, #43799, #43983, #44065, #44228, #44654, #44712, #45039, #45111, #45311, #45513, #45749, #46005, #46506, #46547, #46807, #46948, #47027, #47214, #47464, #47929, #48392, #48551) mirror SemiAnalysis InferenceX AgentX submissions as raw launch commands, each linked to the run that measured it. The configurations are mirrored from SemiAnalysis submissions. This PR:

1. Syncs every page to InferenceX main (2ad2c9f9). One new merged lane comes from its dashboard run:
   • Kimi-K3.1-Pro B300 SGLang (DFlash): InferenceX #4803, run 41932307764, 2026-11-05. The Kimi-K3.1-Pro page gets a B300 hardware option (GB300, B300, MI355X). The MI355X data stays the hand-maintained file from [Doc] cookbook: Add Kimi-K3.1-Pro deployment recipes #48413.
2. Overlays seven open InferenceX PRs:
   • Kimi-K3.1-Pro GB300 Dynamo+SGLang: DFlash disaggregated and aggregated megakernel points (as in [Doc] cookbook: Update Kimi-K3.1-Pro GB300 AgentX launches #48551, at the PR's newer head; the commands are unchanged).
   • Qwen3.8-Flash-Next GB300 SGLang config update (new in this refresh).
   • Muse Spark Open GB300 Dynamo+SGLang: retuned aggregated points and new TP8 disaggregated points (newer head than in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main 57116fff #48392).
   • Muse Spark Open B200 and GB200 SGLang, retuned per concurrency.
   • MiniMax-M3.1-Flash GB200 Dynamo+SGLang config.
   • DeepSeek-V4.1-Flash GB300 Dynamo+SGLang HiCache disagg points on the 20261030 nightly, now verified by the PR's /use run.

Source

InferenceX main (2ad2c9f9), with these InferenceX PRs overlaid:

PR	Page	Lanes	Pinned to
#4811 Kimi-K3.1-Pro GB300 Dynamo+SGLang: DFlash disagg and agg megakernel points	Kimi-K3.1-Pro	kimik3.1-fp4-gb300-dynamo-sglang-agentic-dflash-agg-mk, -dflash-disagg	no /use yet: in progress at head 52b22ba3
#4787 Qwen3.8-Flash-Next GB300 SGLang AgentX update	Qwen3.8-Flash-Next	qwen3.8next-fp4-gb300-sglang-agentic-mtp	no /use yet: in progress at head 5178cd36
#4757 Muse Spark Open GB300 Dynamo+SGLang: retuned agg points, new TP8 disagg points	Muse Spark Open	musespark-fp4-gb300-dynamo-sglang-agentic-dflash-agg, -disagg	no /use yet: in progress at head 7caf5718
#4707 Muse Spark Open B200 SGLang, retuned per concurrency	Muse Spark Open	musespark-fp4-b200-sglang-agentic-dflash	run 41297696082, named by the PR's /use comment (head c965ca2e, unchanged), verified
#4695 MiniMax-M3.1-Flash GB200 Dynamo+SGLang config	MiniMax-M3.1	minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-agg, -disagg	run 41271615328, named by the PR's /use comment (head fa292b90, unchanged), verified
#4673 DeepSeek-V4.1-Flash GB300 Dynamo+SGLang HiCache disagg, 20261030 nightly	DeepSeek-V4.1	dsv41flash-fp4-gb300-dynamo-sglang-agentic-agg, -dspark-disagg-hicache	run 41581664540, named by the PR's /use comment (head be7bc6df, unchanged), verified
#4670 Muse Spark Open GB200 SGLang, retuned per concurrency	Muse Spark Open	musespark-fp4-gb200-sglang-agentic-dflash	run 41096285400, named by the PR's /use comment (head c9eb455f, unchanged), verified

"In progress" lanes are rendered at the PR head and labeled "In progress in SA InferenceX AgentX". A later sync pins them once the PR gets a /use run or merges.

• Qwen3.8-Flash-Next GB300 is in progress. InferenceX #4787 changes the lane's recipe, which replaces the points main measured in InferenceX #4552's run (2026-11-03). They show as verified again once [Bug] gemma-3-27b-it-bnb-4bit Running failed #4787 gets a /use run or merges.
• Muse Spark Open GB300 Dynamo+SGLang stays in progress. InferenceX #4757's newer head keeps only SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS and drops SGLANG_TRTLLM_MHA_DECODE_SEQ_LEN_SPLITS and SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS_MIN_BS. As in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main 57116fff #48392, the old key musespark-fp4-gb300-dynamo-sglang-agentic-dflash (which fix typo: disagg_prefill_infight_queue -> disagg_prefill_inflight_queue #4757 deletes) is not shown.
• DeepSeek-V4.1-Flash GB300: with InferenceX #4673 now verified, its HiCache worker is the closest measured GB300 HiCache donor. The derived (not run) HiCache tier of the GB300 points measured without KV offload now uses its --hicache-ratio 1.2 (was 2). Measured commands are unchanged.
• DeepSeek-V4 page unchanged, as in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main ea8b3439 #47464: InferenceX #4664 retired DeepSeek-V4-Pro.
• Not mirrored (unmeasured, unchanged): musespark-fp4-h100-sglang-agentic-dflash and qwen3.5-fp8-h100-sglang-agentic. The dashboard has no rows for them.
• InferenceX main was frozen for this PR at 2ad2c9f9. Later merges are left for the next refresh.

What this PR does

Page	Lanes	Deployments	Points	Hardware
Kimi-K3.1-Pro	3 (was 2)	10 (was 6)	20 (was 9)	GB300, B300 (+ MI355X, hand-maintained)
Qwen3.8-Flash-Next	4	8 (was 6)	25 (was 27)	GB300, B300, B200, H200
Muse Spark Open	7	15	61	GB300, GB200, B300, B200, H200
DeepSeek-V4.1	10	22	67	GB300, GB200, B300, B200, H200, H100
MiniMax-M3.1	8	40	119	GB300, GB200, B300, B200
GLM-5.2	14	34	75	GB300, GB200, B300, B200, H200
GLM-5.3	1	1	10	B300
Qwen3.5	18	52	222	GB300, GB200, B300, B200, H200, H100
DeepSeek-V4	unchanged

• Five new mirrored settings, all on the Qwen3.8-Flash-Next GB300 lane (InferenceX #4787), kept as submitted:
  • --enforce-disable-flashinfer-allreduce-fusion: never use FlashInfer allreduce fusion (TP4 point).
  • --min-free-slots-delay: hold new prefills until N running-request slots are free, so they are admitted in one batch.
  • --no-ple-offload-embedding: keep the per-layer embedding (N-gram) table on the GPU instead of the default host offload.
  • --qsa-indexer-dtype fp8_e4m3: storage dtype of the compressed QSA indexer cache (the main KV cache is unaffected).
  • IN_BATCH_PREFIX_CACHING_CHECK_THRESHOLD=-1: turns off the scheduler's in-batch prefix-caching check (SGLang default 32).
• The GLM-5.2, GLM-5.3, MiniMax-M3.1 and Qwen3.5 data change only in their shared source block (the InferenceX main SHA and the overlay list).

Validation

• Browser checks: every page this PR changes was rendered in a headless browser (mint dev, Mintlify as in CI).

  • Every hardware / checkpoint / deployment / concurrency / KV tier / router combination renders exactly the decoded commands: 3,254 command sets, 0 mismatches, 0 unseen, no browser errors. (On Kimi-K3.1-Pro the 13 MI355X combinations come from the hand-maintained file, which the sync does not generate, so they have no decoded reference and were not compared.)
  • Every tab was clicked, and all 186 "Copy link" URLs were reopened.
  • The section links from the Deployment panels and the copy buttons were also checked.

• Decode and KV-flag checks:

  • The component's own decoder reproduces every generated plan (3,254 router × KV-tier variants).
  • Every KV flag exists in its image's SGLang commit (223 image × flag pairs).

• Arg-parse: all 108 commands not validated in an earlier round parse:

  Image	Commands	Parsed with
  dev-kimi-k31-pro	67	the image (both platforms are built from 586355a8)
  nightly-dev-cu13-20261103-797af717 (arm64-only digest c7c348bc)	37	the amd64 build of the same tag (index b4323a89), same build commit 797af717
  nightly-dev-cu13-20261030-302b78cc	4	the image, and again with SGLang 52604e1d (its arm64 build's commit) for the GB300 lanes

• node scripts/check_cookbook_configs.mjs, mint broken-links --check-anchors --check-redirects, mint validate and pre-commit run --all-files (in lmsysorg/sglang:dev-cu13) pass.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #42010218750
Latest PR Test (Extra): ❌ Run #42010219011
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mintlify]

✅ Preview approved

@nvpohanh approved a Mintlify preview for commit 09d9d28.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Nov 5, 2026, 11:12 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

[nvpohanh]

Preview looks good. Merging