[nvpohanh]

[by Claude Code]

Motivation

The cookbook's Agentic Long-Context Deployment sections (#42726, #43083, #43799, #43983, #44065, #44228, #44654, #44712, #45039, #45111, #45311, #45513, #45749, #46005, #46506, #46547, #46807, #46948, #47027, #47214, #47464, #47929) mirror SemiAnalysis InferenceX AgentX submissions as raw launch commands, each linked to the run that measured it. The configurations are mirrored from SemiAnalysis submissions. This PR:

1. Syncs to InferenceX main (57116fff). These merged lanes now come from their dashboard runs:
   • MiniMax-M3.1-Flash B300 SGLang (TP2 / TP4 points): InferenceX #4726, run 41488743791, 2026-11-03. It was in progress in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main c5e8b9b3 and revert #47560 #47929 and is now verified.
   • MiniMax-M3.1-Flash GB300 Dynamo+SGLang agg and disagg: InferenceX #4750, run 41528865560, 2026-11-03. [Doc] cookbook: Add MiniMax-M3.1-Flash GB300 MonoKernel and DEP16 AgentX points #48070 had added these points by hand (6P1D c220 / c240 / c260 and the TP4 c1 MonoKernel point). The sync regenerates the file from InferenceX main and the dashboard, with the same points.
   • Overlaid in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main c5e8b9b3 and revert #47560 #47929 and since merged, on the same runs: Muse Spark Open B300 (InferenceX #4725) and GB300 SGLang (InferenceX #4724), Qwen3.8-Flash-Next B300 (InferenceX #4683) and DeepSeek-V4.1-Flash GB300 SGLang (InferenceX #4603).
   • PowerX re-runs, with provenance changes only (same commands): GLM-5.2 B200 Dynamo+SGLang agg / disagg (InferenceX #4656), GLM-5.2 FP8 B300 SGLang (InferenceX #4702) and Qwen3.8-Flash-Next GB300 (InferenceX #4552), all 2026-11-03.
2. Overlays five open InferenceX PRs:
   • Muse Spark Open GB300 Dynamo+SGLang: retuned aggregated points and new TP8 disaggregated points.
   • Muse Spark Open B200 and GB200 SGLang, retuned per concurrency.
   • DeepSeek-V4.1-Flash GB300 Dynamo+SGLang HiCache disagg points on the 20261030 nightly.
   • MiniMax-M3.1-Flash GB200 Dynamo+SGLang config.

Source

InferenceX main (57116fff), with these InferenceX PRs overlaid:

PR	Page	Lanes	Pinned to
#4757 Muse Spark Open GB300 Dynamo+SGLang: retuned agg points, new TP8 disagg points	Muse Spark Open	musespark-fp4-gb300-dynamo-sglang-agentic-dflash-agg, -disagg	no /use yet: in progress at head b37254d0
#4707 Muse Spark Open B200 SGLang, retuned per concurrency	Muse Spark Open	musespark-fp4-b200-sglang-agentic-dflash	run 41297696082, named by the PR's /use comment (head c965ca2e, unchanged), verified
#4670 Muse Spark Open GB200 SGLang, retuned per concurrency	Muse Spark Open	musespark-fp4-gb200-sglang-agentic-dflash	run 41096285400, named by the PR's /use comment (head c9eb455f, unchanged), verified
#4673 DeepSeek-V4.1-Flash GB300 Dynamo+SGLang HiCache disagg, 20261030 nightly	DeepSeek-V4.1	dsv41flash-fp4-gb300-dynamo-sglang-agentic-agg, -dspark-disagg-hicache	no /use yet: in progress at head be7bc6df
#4695 MiniMax-M3.1-Flash GB200 Dynamo+SGLang config	MiniMax-M3.1	minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-agg, -disagg	run 41271615328, named by the PR's /use comment (head fa292b90, unchanged), verified

"In progress" lanes are rendered at the PR head and labeled "In progress in SA InferenceX AgentX". A later sync pins them once the PR gets a /use run or merges.

• Muse Spark Open GB300 Dynamo+SGLang is all in progress. InferenceX #4757 replaces the key musespark-fp4-gb300-dynamo-sglang-agentic-dflash with -dflash-agg and -dflash-disagg. The old key leaves the page, as it will when the PR merges. Its points measured on main (InferenceX #4344, 2026-10-22) are no longer shown.
• DeepSeek-V4.1-Flash GB300 Dynamo+SGLang is all in progress. InferenceX #4673 changes both GB300 Dynamo+SGLang keys, which takes all of their points. That includes the points main measured in InferenceX #4595 / #4679's run on 2026-10-30. They show as verified again once [Bug] CUDA error: an illegal memory access was encountered when launching server with dp_size=16 and benching with request-rate option #4673 gets a /use run or merges.
• Not overlaid: InferenceX #4536 (MiniMax-M3.1-Flash B200 DRAM-offload points and TP2). It conflicts with InferenceX main. Its head predates InferenceX #4598's FP8 draft KV (speculative-draft-kv-cache-dtype: fp8_e4m3) in the same B200 recipe, so overlaying it would replace [Bug] KeyError: 'model.layers.13.mlp.experts.w2_weight_scale' #4598's measured points with a revision without that change. B200 stays on [Bug] KeyError: 'model.layers.13.mlp.experts.w2_weight_scale' #4598's dashboard run. A later refresh can add [Bug] bench_speculative.py got error #4536 once it is rebased.
• DeepSeek-V4 page unchanged, as in [Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main ea8b3439 #47464: InferenceX #4664 retired DeepSeek-V4-Pro.
• Not mirrored (unmeasured, unchanged): musespark-fp4-h100-sglang-agentic-dflash and qwen3.5-fp8-h100-sglang-agentic. The dashboard has no rows for them.
• InferenceX main was frozen for this PR at 57116fff. Later merges are left for the next refresh.

What this PR does

Page	Lanes	Deployments	Points	Hardware
MiniMax-M3.1	8	40 (was 36)	119 (was 118)	GB300, GB200, B300, B200
GLM-5.2	14	34	75	GB300, GB200, B300, B200, H200
GLM-5.3	1	1	10	B300
DeepSeek-V4	unchanged
DeepSeek-V4.1	10	22 (was 23)	67	GB300, GB200, B300, B200, H200, H100
Qwen3.5	18	52	222	GB300, GB200, B300, B200, H200, H100
Qwen3.8-Flash-Next	4	6	27	GB300, B300, B200, H200
Muse Spark Open	7 (was 6)	15 (was 9)	61 (was 56)	GB300, GB200, B300, B200, H200

• Eight new mirrored settings, all on the Muse Spark Open GB300 Dynamo+SGLang lanes (InferenceX #4757), kept as submitted:
  • SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS and SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS_MIN_BS: TRT-LLM MHA full-attention decode / verify kernel tuning (minimum KV splits per request and KV head, and the batch size from which it applies).
  • SGLANG_DISAGGREGATION_QUEUE_SIZE and SGLANG_DISAGGREGATION_THREAD_POOL_SIZE: PD KV-transfer queues and thread pool.
  • SGLANG_DISAGG_PREFILL_EARLY_SEND_CACHED_PREFIX=0: the prefill worker does not send the KV of a cached prefix ahead of the forward pass.
  • --optimistic-prefill-attempts: prefill forward passes that may start before the bootstrap handshake completes.
  • DYN_ROUTER_USE_KV_EVENTS=false and DYN_ROUTER_TTL_SECS (Dynamo frontend): the KV router predicts cache state from its own routing decisions instead of worker KV events, and keeps predicted blocks for that long.
• The GLM-5.3 and Qwen3.5 data change only in their shared source block (the InferenceX main SHA and the overlay list).

Validation

• Browser checks: every page this PR changes was rendered in a headless browser (mint dev, Mintlify as in CI).

  • Every hardware / checkpoint / deployment / concurrency / KV tier / router combination renders exactly the decoded commands: 3,152 command sets, 0 mismatches, 0 unseen, no browser errors.
  • Every tab was clicked, and all 170 "Copy link" URLs were reopened.
  • The section links from the Deployment panels and the copy buttons were also checked.

• Decode and KV-flag checks:

  • The component's own decoder reproduces every generated plan (3,152 router × KV-tier variants).
  • Every KV flag exists in its image's SGLang commit (215 image × flag pairs).

• Arg-parse: all 142 commands not validated in an earlier round parse:

  Image	Commands	Parsed with
  nightly-dev-cu13-20261103-797af717	63	the image (both platforms are built from 797af717)
  nightly-dev-cu13-20261030-302b78cc	50	the image, and again with SGLang 52604e1d (its arm64 build's commit) for the GB300 lanes
  v0.5.22-cu130	17	the image
  dev-minimax-m31-flash	6	the image
  nightly-cu134-20261030-e371881 (arm64 only)	6	SGLang e3718813 source and the image's bundled ai-dynamo 1.7.0.dev20261030, on the amd64 20261030 nightly

• node scripts/check_cookbook_configs.mjs, mint broken-links --check-anchors --check-redirects, mint validate and pre-commit run --all-files (in lmsysorg/sglang:dev-cu13) pass.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #41827698133
Latest PR Test (Extra): ❌ Run #41827698056
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mintlify]

✅ Preview approved

@nvpohanh approved a Mintlify preview for commit ec69f3b.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Nov 4, 2026, 12:40 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

[nvpohanh]

Preview looks good. Merging