[niehen6174]

Motivation

We benchmarked the cookbook commands on 2x H200 and 4x B200. In five places the builder's default was slower than another sglang command on the same hardware and workload. All of these are cookbook choices, not runtime changes.

All numbers are end-to-end client latency through the HTTP server, sglang at 3a01d4c (2026-10-28). Image cases are the median of 5 requests after 2 warmups; video cases are the median of 2 requests after 3 warmups.

Modifications

1. MiniMax-H3 on 4x B200: resident, not FSDP (minimax-h3.jsx)

Auto placement takes the first verified recipe for a GPU count, and on 4x B200 that was b200-fsdp-4. 4x B200 holds the DiT resident, and resident is faster:

Workload (4x B200, Ulysses 4)	Resident	FSDP (previous default)
FL2VA 1120x768, 209 frames	64.90 s	67.60 s (+4%)
T2VA 1344x768, 124 frames	36.03 s	38.71 s (+7%)

The slowdown is all in denoise: 62.87 vs 65.58 s, and 34.58 vs 37.18 s.

b200-resident-4 is added ahead of b200-fsdp-4, so FSDP stays available as an explicit placement.

2. FastH3 Omni: drop the three CPU offloads from the 2x H200 command (MiniMax-H3.mdx)

The 2x H200 command offloads the DiT, the text encoder and the VAE. Everything fits on 141 GB and 180 GB cards, and the offloads only cost time. Measured with an image reference, 768 short edge, 124 frames:

GPUs	Resident	Three CPU offloads
2x H200, Ulysses 2	39.02 s	43.81 s (+12%)
4x B200, Ulysses 4	10.28 s	15.35 s (+49%)

The command is now resident. The offload flags stay documented as the capacity fallback, and the existing latency table is relabelled as measured with offload.

3. MOSS-Image on 2+ GPUs: CFG parallel first (moss-image.jsx)

Every MOSS-Image guidance recipe runs two or three branches, but autoTopology gave all GPUs to Ulysses. Now an even GPU count gets --cfg-parallel-size 2, and Ulysses gets the rest. 1024x1024, 50 steps, CFG 3 with the default unconditional skip of layer 22:

2x B200	Latency
Ulysses 2 (previous builder output)	3.175 s
--cfg-parallel-size 2	2.445 s (-23%)

The exact builder command (--tp-size 1 --ulysses-degree 1 --ring-degree 1 --cfg-parallel-size 2) measured 2.441 s on 2x B200. On 2x H200, --cfg-parallel-size 2 runs 4.275 s against 5.087 s for Ulysses 2 (-16%). On 4x B200 the builder now emits CFG 2 x Ulysses 2: 1.884 s, against 2.344 s for its previous Ulysses 4 (-20%).

4. Qwen-Image-2.1 on 2x B200: TP2 by default (qwen-image-2.1.jsx)

b200-2-tp was already a verified recipe, but autoTopology picked Ulysses 2 on B200. 1024x1024, 40 steps, no CFG, run back to back in one session:

2x B200	Latency	Denoise
Ulysses 2 + FA (previous builder output)	1.585 s	1.43 s
TP2	1.427 s (-10%)	1.27 s

H200 keeps its current topology. TP2 was not measured there.

5. Qwen-Image-2.1 on one B200: cuDNN SDPA instead of a pinned FlashAttention (qwen-image-2.1.jsx, Qwen-Image-2.1.mdx)

Automatic attention on one B200 emitted --attention-backend fa. Without it the runtime picks its Blackwell default, dynamic cuDNN SDPA. 1024x1024, 40 steps, two runs each, same build (sglang 3a01d4c):

1x B200	FlashAttention (previous output)	cuDNN SDPA
Generation	2.32 / 2.32 s	2.22 / 2.22 s (-4.3%)
Edit	2.89 / 2.89 s	2.70 / 2.69 s (-6.6%)

Denoise per step is 2.004 s against 2.095 s for the 40-step loop: cuDNN runs this model's joint attention in 246 us per layer against 354 us for FlashAttention 4.

The one-B200 request matrix ran under cuDNN SDPA: 1/2/4 outputs, transparent text and edit, and the two-reference edit. Every shape succeeded and was faster (4-13%); the transparent outputs keep the same alpha coverage as FlashAttention. b200-1-resident now lists cudnn among its attentions and batch attentions, and the picker gains an explicit cuDNN SDPA option (soft outside that recipe). Two B200s keep FlashAttention, the attention their recipes were verified with. The table's B200 row now shows cuDNN SDPA, with the FlashAttention figures from the same build in the note below it.

Accuracy Tests

No runtime change; only the builder output changes. Each new default is a configuration the cookbook already lists or the runtime already supports. cuDNN SDPA and FlashAttention are both exact attention but reduce in a different order, so the one-B200 images change slightly: PSNR against the FlashAttention outputs was 32-49 dB for generation and transparent edits, and lower (20-25 dB) for the opaque edit and the two-reference edit, where the 40-step trajectories drift apart; those images were checked by eye and keep the same composition.

Speed Tests and Profiling

See the tables above. Each builder was also executed with node to check what it now emits:

• MOSS-Image 2 GPUs: --num-gpus 2 --tp-size 1 --ulysses-degree 1 --ring-degree 1 --cfg-parallel-size 2
• Qwen-Image-2.1 B200 x2: TP 2. H200 x2 (base weights) is unchanged at Ulysses 2.
• Qwen-Image-2.1 B200 x1: no --attention-backend flag; the summary reads cuDNN SDPA (auto), serve and request verified (including 4 outputs). B200 x2 still emits --attention-backend fa.
• MiniMax-H3 B200 x4: the recipe is b200-resident-4, placement resident.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests. Docs only.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ✅ Run #41283540079
Latest PR Test (Extra): ❌ Run #41283540099
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[mintlify]

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 9a4edee.

Approve preview

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[niehen6174]

/tag-and-rerun-ci