[Doc] cookbook: Refresh SemiAnalysis AgentX launches for Qwen3.8-Flash-Next H200 (v0.5.22, TP2 points)

[Ankur-singh]

[by Claude Code]

Motivation

The cookbook's Agentic Long-Context Deployment sections (#42726, #43083 and later syncs) mirror SemiAnalysis InferenceX AgentX submissions as raw launch commands, each linked to the run that measured it. The configurations are mirrored from SemiAnalysis submissions.

The Qwen3.8-Flash-Next H200 lane still shows the day-zero submission: InferenceX #2753, measured in run 33038487711 on the lmsysorg/sglang:qwen38flashnext image. InferenceX #4272 moves that lane to the v0.5.22 release and adds TP2 points. This PR mirrors the new lane.

Source

InferenceX main (9a132463), with this open InferenceX PR overlaid:

PR	Page	Lane	Pinned to
#4272 Qwen3.8-Flash-Next FP8 H200 SGLang	Qwen3.8-Flash-Next	qwen3.8next-fp8-h200-sglang-agentic-mtp	run 39109720946, named by the PR's /use comment (head 0d749be1), verified

Only the Qwen3.8-Flash-Next page is regenerated. Its source line now lists InferenceX #4272 only. The other pages still name InferenceX #4033 (merged since) and InferenceX #4162 (Qwen3.5); neither has a Qwen3.8-Flash-Next lane.

What this PR does

Page	Lanes	Deployments	Points	Hardware
Qwen3.8-Flash-Next	4	5 (was 4)	24 (was 22)	GB300, B300, B200, H200

• Qwen3.8-Flash-Next FP8 H200 SGLang (InferenceX #4272).

  	Before (InferenceX #2753)	After (InferenceX #4272)
  Image	lmsysorg/sglang:qwen38flashnext	lmsysorg/sglang:v0.5.22-cu130, pinned by digest (sha256:e3a4f060…)
  Deployments	Agg TP4·EP4: c1, 4, 8, 12, 16	Agg TP2·EP2: c1, 8, 12, 14, 16 Agg TP4·EP4: c1, 8
  Admission and decode graphs	--max-running-requests = 2× concurrency --cuda-graph-max-bs = concurrency	--max-running-requests = --cuda-graph-max-bs-decode = 2× concurrency
  ReplaySSM	—	--enable-linear-replayssm-spec on TP2·EP2 c8, c12, c14, c16
  Environment	SGLANG_ENABLE_SPEC_V2=1	none: the image's SGLang (baa9befa) no longer reads this variable
  --tokenizer-path	the model id	not set (defaults to the model)

  • Unchanged:
    • checkpoint Qwen/Qwen3.8-Flash-Next-FP8 with its in-checkpoint NEXTN head (MTP 3-1-4);
    • --mamba-ssm-dtype float32;
    • FlashInfer linear-attention prefill and decode backends;
    • --chunked-prefill-size 8192 and --mem-fraction-static 0.85;
    • --stream-interval 50, --scheduler-recv-interval 10 and --tokenizer-worker-num 6;
    • SGLang router and no KV offload, as measured;
    • the synthetic-acceptance disclosure (golden AL 2.32).
  • Derived variants (not run):
    • Dynamo + SGLang, as on every lane.
    • The Mooncake store tier is new for H200: the old image predated --enable-unified-cache-external-linker, and v0.5.22 has it.
    • HiCache stays unavailable for this lane.

• GB300, B300 and B200: unchanged. Their deployments are byte-identical to main.

How the commands are produced

The pipeline and rules are the same as in #44712, with these changes. None of them alters an existing deployment: GB300, B300 and B200 come out byte-identical to main.

• Composed recipes. InferenceX #4272 is the first Qwen3.8-Flash-Next lane in InferenceX's composed-recipe layout.
  • The recipe is a fragment: InferenceX binds the master config's model, image, precision and point values at launch, then picks one override_tp{N}_c{M} variant per point.
  • The pipeline now does that binding with InferenceX's own selector (infx.srt_slurm.single_node.select_recipe) at the pinned commit. Each point therefore renders the variant the sweep ran.
• Rules. Removed as telemetry:
  • srt-slurm's job-local OTLP collector (otelcol-contrib);
  • the variables it injects (DYN_LIFECYCLE_TRACE_ENABLED, OTEL_EXPORT_ENABLED, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_SERVICE_NAME).
• GPU count and cluster environment follow [Doc] cookbook: Mirror SA cluster NCCL_CUMEM_ENABLE in AgentX launches, fix GPU counts #44228.
  • H200 TP2·EP2 shows 2 GPUs and TP4·EP4 shows 4.
  • The H200 cluster does not export NCCL_CUMEM_ENABLE, so the H200 commands carry no cluster variable.

Each lane is still rendered at the exact commit of the run that produced its dashboard data.

• Removed (SA-only settings): the benchmark harness, cluster paths and caches, telemetry, and InferenceX's synthetic speculative acceptance (disclosed per deployment).
• Replaced with placeholders: IPs, ports and NICs.
• Derived variants: router and KV-offload variants that were not run are derived and labeled "Derived — not run".

The data file is generated, so there is no hand-written change to review.

Validation

• CI gates.
  • node docs/scripts/check_cookbook_configs.mjs passes.
  • mint broken-links --check-anchors --check-redirects passes (mint 4.2.559, the CI pin).
• mint validate fails on main today, for a reason outside this PR:
  • Since [Doc] popular-models: Replace Kimi-K3 with Muse Spark Open #44693, docs/src/snippets/configs/popular-models.jsx has a stray }, at line 100, so that snippet does not parse.
  • With that one line removed locally (not part of this PR), mint validate passes on this branch.
• Browser click-through of the Qwen3.8-Flash-Next page under mint dev:
  • All 94 combinations of hardware, checkpoint, deployment, concurrency, KV tier and router render exactly the decoded commands (0 mismatches, 0 unseen, no browser errors), with every tab clicked.
  • Each deployment's "Copy link" URL restores the same selection in a fresh tab (5 of 5).
  • The Deployment-panel footer link is present on this page and absent on two control pages.
  • Copy, Copy all and Copy link work.
• Decoder and KV flags.
  • The component's own decoder reproduces the generated commands for all 94 router × KV combinations, including the 48 measured tiers.
  • Every KV-cache flag shown exists in its deployment's SGLang commit (11 image/flag pairs, 170 uses).
• Round trip and leak scan. Every flag and variable of every submitted launch is kept, rewritten or removed by a rule. No SemiAnalysis cluster artifact appears in the shown commands.
• Diff against main.
  • The 66 GB300, B300 and B200 command sets are unchanged.
  • H200's 10 command sets (5 points × 2 routers) are replaced by 28 (7 points × 2 routers × 2 KV tiers).
• Against the real run. For all seven H200 points, the rendered sglang.launch_server flags equal those in the public job logs of run 39109720946 (25 or 26 flags each).
  • The only difference is --model-path: a container mount versus the Hugging Face id. The page shows the id.
• pre-commit.
  • The hooks that apply to the changed file pass with pre-commit run --files: trailing whitespace, end of file, large files, merge conflicts, private key and codespell.
  • pre-commit run --all-files in lmsysorg/sglang:dev-cu13 was not run.
• Argument parsing in the submission image was not run.
  • The H200 commands are the ones that started and served successfully with this exact image in run 39109720946 (previous bullet).

No serving or benchmark runs were performed for this PR; the numbers live in the linked SemiAnalysis run. SGLang CI was not triggered.

cc @nvpohanh

AI assistance

Prepared with Claude Code (model claude-opus-5-5): the sync, the validation above and this description. Reviewed by @Ankur-singh before opening.

---

CI States

Latest PR Test (Base): ✅ Run #39158205806
Latest PR Test (Extra): ❌ Run #39158205688
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.