[Doc] cookbook: Refresh SemiAnalysis AgentX launches for Qwen3.5 and DeepSeek-V4.1-Flash GB300

[nvpohanh]

[by Claude Code]

Motivation

The cookbook's Agentic Long-Context Deployment sections (#42726, #43083, #43799, #43983, #44065, #44228, #44654) mirror SemiAnalysis InferenceX AgentX submissions as raw launch commands, each linked to the run that measured it. This PR:

1. Replaces the Qwen3.5 NVFP4 GB300 Dynamo+SGLang deployments with the nine points of the open InferenceX #4162. That PR retires the four earlier configs of this lane.
2. Refreshes DeepSeek-V4.1-Flash GB300 SGLang to the open InferenceX #4033: the 20261014 nightly, the fastokens tokenizer, decoder SWA bounded replay and a new DEP2 c160 point.
3. Syncs to InferenceX main (12dbddac). All seven InferenceX PRs of [Doc] cookbook: Add SemiAnalysis AgentX launches to Muse Spark Open, refresh DeepSeek-V4.1-Flash #44654 have merged, and so has InferenceX #4155, which retunes Muse Spark Open B200.

Source

InferenceX main (12dbddac), with these open InferenceX PRs overlaid:

PR	Page	Lanes	Pinned to
#4162 Qwen3.5 NVFP4 GB300 Dynamo+SGLang	Qwen3.5	qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-src, -disagg-src (new); retires -agentic-agg, -agentic-disagg, -agentic-agg-pareto, -agentic-pp-pareto	no /use yet: in progress at head cd917605
#4033 DSV4.1-Flash GB300 SGLang	DeepSeek-V4.1	dsv41flash-fp4-gb300-sglang-agentic-dspark	no /use yet: in progress at head a61b7eba

"In progress" lanes are rendered at the PR head and labeled "In progress in SA InferenceX AgentX". A later sync pins them once the PR gets a /use run or merges.

The overlays of #44654 have all merged into InferenceX main, and their lanes now come from main, pinned to the run the dashboard shows. Their commands are unchanged:

PR	Lane	Now pinned to
InferenceX #4146	musespark-fp4-b300-sglang-agentic-dflash	run 38825818513, verified (was in progress)
InferenceX #4141	musespark-fp4-gb300-sglang-agentic-dflash	run 38835086328, verified (was in progress)
InferenceX #4112	dsv41flash-fp4-gb200-sglang-agentic-dspark	run 38749229680, verified (was in progress)
InferenceX #4111	dsv41flash-fp4-b200-sglang-agentic-dspark	run 38749770474, the run its /use named
InferenceX #4094	dsv41flash-fp4-b300-sglang-agentic-dspark	run 38705241139, the run its /use named

InferenceX #4147 and InferenceX #4140 were already pinned from main in #44654. #4147's B200 lane is now replaced by #4155 (below).

What this PR does

Page	Lanes	Deployments	Points	Hardware
GLM-5.2	16	27	67	GB300, GB200, B300, B200, H200
DeepSeek-V4	10	20	61	GB300, GB200, B300, B200, H200
DeepSeek-V4.1	9	22	74 (was 73)	GB300, GB200, B300, B200, H200, H100
Qwen3.5	17 (was 19)	47 (was 50)	208 (was 221)	GB300, GB200, B300, B200, H200, H100
Qwen3.8-Flash-Next	4	4	22	GB300, B300, B200, H200
Muse Spark Open	4	4	41 (was 40)	GB300, GB200, B300, B200

Only the generated data files change. The component (_agentx.jsx), page text, Deployment panels and Playgrounds are untouched.

• Qwen3.5 NVFP4 GB300 Dynamo+SGLang (InferenceX #4162), replaced. All new points run MTP 6-1-7 (NEXTN, the checkpoint's BF16 draft head) with HiCache (host DRAM), on lmsysorg/sglang:dev-qwen35-agentx-gb300-ccebaceeeca3. That image is built from the SGLang branch nvidia/qwen35-agentx-gb300 at ccebacee.

  Before (measured on main)	After ([Docs] Fix links and grammar issues #4162, in progress)
  Agg TP2 MTP 3-1-4 c1–c64; Agg TP8 (2 nodes) c7; Agg TP2 c44 (two --max-running-requests variants); Disagg 1P1D TP4/TP4 no spec c8–c128; Disagg 1P1D TP2/TP2 c72; Disagg 3P4D c565; Disagg 3P2D c704	Agg TP2 c5, 7, 10; Agg TP4 c7; Agg TP8 (2 nodes) c7, 10 (all --max-running-requests 1); Disagg 1P1D TP1 prefill + TP2 decode on one node (3 GPUs) c38, c56; Disagg 3P1D (3 × PP4 prefill, DEP8 decode on 2 nodes, 20 GPUs) c704
  4 configs, 8 deployments, 19 points; nightly-dev-cu13-20260724-433429b1 / 20260831-bb5e6198	2 configs, 5 deployments, 6 points (the PR's nine concurrencies)

  • Aggregated points were submitted with the SGLang router, disaggregated ones with Dynamo (KV router; 2 frontends behind nginx at c704). The disaggregated launches use Dynamo built from source at b208989f, shown as the cell's install note.
  • The workers set the branch's SGLANG_Q35MF_* switches, mirrored as submitted. They are off by default, and each point enables the ones its recipe lists.
  • The Mooncake tier is now offered (derived): this image includes the external linker, which the retired lanes' images predated.
  • The four retired configs leave the page because [Docs] Fix links and grammar issues #4162 deletes them. Their measured main versions (run 31042542308 of InferenceX #2477, run 34345050756 of InferenceX #2812) remain HiCache donors only as a fallback, which no deployment needed this time. No other Qwen3.5 command changes.

• DeepSeek-V4.1-Flash GB300 SGLang (InferenceX #4033). The image moves from nightly-dev-cu13-20261012-e6072935 to nightly-dev-cu13-20261014-684fea8a (pinned by digest). The same three deployments get these changes:

  • --tokenizer-backend fastokens on every point except DEP2 c160, which keeps huggingface;
  • --enable-decoder-swa-bounded-replay on DEP2 c64 / c128 and TP4 c1;
  • TP4 c1: --max-running-requests 1 (was 2) and --cuda-graph-backend-prefill disabled;
  • a new Agg DEP2 c160 point (c128 settings with --prefill-decode-interval 8).

  The lane is in progress until Share target model embed and head weights for nextn #4033 gets a /use run. Its measured main version (InferenceX #3859, run 38221097558) stays a donor-only fallback.

• Muse Spark Open B200 (InferenceX #4155, merged). The lane is pinned to run 38851010434, the run [Feature] Support multiple Languages for Docs #4155's /use named. It moves from Agg TP4·EP4 (c1–c32) to Agg TP8·EP8 on all 8 GPUs:

  • c1–c12 run without KV offload;
  • c16–c48 run HiCache (--hicache-size 120 GB at c16, 160 GB from c24) with prefill sequence parallelism and MegaMoE;
  • all points use --attention-backend trtllm_mha, --page-size 128, --max-running-requests 192, breakable prefill CUDA graphs, --speculative-use-block-verification and --enable-http2.

  c1–c12 now also offer a derived HiCache tier, using the B200 lane's own HiCache flags. B300 and GB300 still have no HiCache tier, since they have no Muse Spark Open HiCache lane on the same hardware.

• GLM-5.2, DeepSeek-V4, Qwen3.8-Flash-Next: no command changes.

• qwen3.5-fp8-h100-sglang-agentic is still not mirrored, as before: no dashboard changelog names it, so it was never measured.

How the commands are produced

The pipeline is the same as in #44654, with these changes:

• Retired configs. A config key that an overlaid InferenceX PR deletes now leaves the page with the overlay, as it will when the PR merges. Before, the pipeline ignored deletions, so these four keys would have stayed next to their replacements. Their measured main versions are kept only as KV-offload donors, like a key that an overlay overrides.

• Run attribution. When two InferenceX PRs name the same key on one dashboard date (Fix nightly ci Gsm8k & Fix flashinfer backend kvcache quant #4147, then [Feature] Support multiple Languages for Docs #4155, on 2026-10-16), the lane is credited to the PR whose run produced the dashboard rows. No other lane's attribution changed.

• Rules. These are kept as serving configuration:

  • worker flags --flashinfer-allreduce-fusion-backend, --speculative-use-block-verification and --enable-http2 (Granian HTTP server with HTTP/2);
  • env SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN and SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND;
  • the SGLANG_Q35MF_* switches of the [Docs] Fix links and grammar issues #4162 image.

  TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC is kept as a timeout.

Each lane is still rendered at the exact commit of the run that produced its dashboard data, or at the PR head while in progress. SA-only settings are removed: the benchmark harness, cluster paths and caches, telemetry, and InferenceX's synthetic speculative acceptance (disclosed per deployment). Placeholders replace IPs, ports and NICs. Router and KV-offload variants that were not run are derived and labeled "Derived — not run". NCCL_CUMEM_ENABLE=1 from the SemiAnalysis cluster environment (#44228) applies to the new lanes as well: their GB300 and B200 clusters export it.

The data files are generated, so there is no hand-written code to review here.

Validation

• mint validate (strict build) and mint broken-links --check-anchors --check-redirects pass. node docs/scripts/check_cookbook_configs.mjs passes, and so does pre-commit run --all-files.

• Browser click-through. A headless browser clicked through every combination of hardware, checkpoint, deployment, concurrency, KV tier and router on all six pages under mint dev, opening every tab. That is 2,384 command sets: GLM-5.2 328, DeepSeek-V4 316, DeepSeek-V4.1 444, Qwen3.5 1,018, Qwen3.8-Flash-Next 76 and Muse Spark Open 202.

  • The rendered commands match the expected text exactly, with no browser errors.
  • The "Copy link" URL of every deployment was reopened in a fresh tab and restored the same selection and tab (124 links).
  • The Deployment-panel footer link and section placement checks pass on all six pages, and two pages that did not opt in show no link. The copy buttons work on a plain-HTTP preview.

• Decoder and KV flags. The component's own decoder, run through Node, reproduces the Python-generated commands for all 2,384 combinations, including every point's measured tier. Every KV-cache flag in a shown command exists in the SGLang commit of its deployment's image (190 image/flag pairs). For the [Docs] Fix links and grammar issues #4162 image that is ccebacee on nvidia/qwen35-agentx-gb300.

• Diff against main.

  Page	Command sets
  GLM-5.2, DeepSeek-V4, Qwen3.8-Flash-Next	all byte-identical
  Qwen3.5	982 identical; 76 removed (the four retired configs), 36 added ([Docs] Fix links and grammar issues #4162)
  DeepSeek-V4.1	396 identical; 42 changed and 6 added (GB300 SGLang, Share target model embed and head weights for nextn #4033)
  Muse Spark Open	148 identical; B200's 32 command sets replaced by 54 ([Feature] Support multiple Languages for Docs #4155)

• Mock against a real run. The rendered Muse Spark Open B200 c16 launch equals the worker command and environment in that job's sweep_*.log of run 38851010434: 0 differences after placement values. The worker fingerprint shows SGLANG_SIMULATE_ACC_LEN=4.05 (removed and disclosed) and NCCL_CUMEM_ENABLE=1.

• Cluster environment. The table still matches the worker fingerprints of measured runs on every cluster with srt-slurm lanes.

• Argument parsing. 107 distinct sglang.launch_server, dynamo.sglang, sglang_router.launch_router and dynamo.frontend command lines are new since [Doc] cookbook: Add SemiAnalysis AgentX launches to Muse Spark Open, refresh DeepSeek-V4.1-Flash #44654. All parse with their own submission images:

  • dev-qwen35-agentx-gb300-ccebaceeeca3: 45/45 (amd64 build of the multi-arch tag). The 12 Dynamo commands of the disaggregated points were checked against the Dynamo source at b208989f that the run builds. The derived Dynamo variants of the aggregated points use the image's bundled ai-dynamo.
  • nightly-dev-cu13-20261014-684fea8a@sha256:ddae0e39…: 48/48.
  • dev-muse-spark-open@sha256:8cfc5a72…: 14/14.

  The other 1,971 of the 2,078 command lines are identical to ones parsed for earlier cookbook PRs. Environment variables are not part of this check.

No serving or benchmark runs were performed for this PR; the numbers live in the linked SemiAnalysis runs. SGLang CI was not triggered.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #38874230020
Latest PR Test (Extra): ❌ Run #38874230033
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

[nvpohanh]

Merging to unblock SA AgentX submission