[NV] Add system configs for DSR1 B200 fp4 sglang MTP (New)

[jgangani]

Summary

Add B200 disaggregated multinode configs for DeepSeek R1 FP4 SGLang with MTP (Multi-Token Prediction) speculative decoding.

• Config key: dsr1-fp4-b200-dynamo-sglang-mtp
• Image: lmsysorg/sglang:v0.5.8.post1-cu130
• Model: deepseek-r1-fp4
• Runner: b200-multinode-slurm (multinode, disaggregated)
• Precision: FP4
• Framework: dynamo-sglang

Configurations

1k1k (ISL=1024, OSL=1024) — 4 search-space entries:

Profile	Recipe	Prefill	Decode	Concurrencies
Low-latency	dep4-1p-tep8-5d	1P, TP4/EP4, dp-attn	5D, TP8/EP8	16, 512
Low-latency	dep4-1p-tep8-6d	1P, TP4/EP4, dp-attn	6D, TP8/EP8	32, 64, 256, 512
Max-throughput	dep4-1p-dep8-1d	1P, TP4/EP4, dp-attn	1D, TP8/EP8, dp-attn	512, 1024
Max-throughput	dep4-1p-dep8-2d	1P, TP4/EP4, dp-attn	2D, TP8/EP8, dp-attn	512

8k1k (ISL=8192, OSL=1024) — 4 search-space entries:

Profile	Recipe	Prefill	Decode	Concurrencies
Low-latency	dep4-1p-tep8-1d	1P, TP4/EP4, dp-attn	1D, TP8/EP8	64, 128
Low-latency	dep4-1p-tep8-5d	1P, TP4/EP4, dp-attn	5D, TP8/EP8	8
Low-latency	dep4-2p-tep8-5d	2P, TP4/EP4, dp-attn	5D, TP8/EP8	4, 128
Low-latency	tp4-1p-tp8-1d	1P, TP4/EP1, no dp-attn	1D, TP8/EP1	4, 8, 16, 64

Notes

• MTP counterpart to existing STP config dsr1-fp4-b200-dynamo-sglang (PR [NV] Add B200 FP4 SGlang STP configs #672)
• Recipe files sourced from srt-slurm (mtp/ subdirectory)
• New branch created from main to avoid complex merge conflict resolution

[jgangani]

@claude please review. please add a PR description for posterity, mainly based on perf changelog

[Klaud-Cold]

Claude finished @jgangani's task in 2m 44s —— View job

---

Review of PR #683

• Gather context and read changed files
• Review code changes
• Update PR description based on perf changelog
• Post final review

---

PR Description Updated

Added a detailed PR description based on the perf-changelog entry, including tables for both 1k1k and 8k1k search-space configurations.

---

Code Review

Overall: The config is well-structured and mirrors the existing STP counterpart (dsr1-fp4-b200-dynamo-sglang from PR #672) with the correct mtp/ recipe paths and spec-decoding: "mtp" fields. The perf-changelog entry is present and correct. A few items to note:

Observations

1. Image tag difference from STP config — The STP config uses lmsysorg/sglang:v0.5.8.post1-cu130-runtime while this MTP config uses lmsysorg/sglang:v0.5.8.post1-cu130 (no -runtime suffix). Assuming this is intentional (MTP may need full image, not runtime-only), but worth confirming.

2. Conc=512 overlap in 1k1k — All 4 search-space entries for 1k1k include conc=512, meaning 4 different recipes will be benchmarked at that concurrency. The STP config only has 2 entries overlapping at conc=512. This seems intentional for broader coverage, but generates more benchmark runs. The 8k1k overlaps match the STP config exactly.

3. No max-throughput recipes for 8k1k — All 8k1k entries are low-latency-* recipes, unlike 1k1k which has both low-latency and max-throughput. This matches the STP config structure, so it's consistent.

Minor Issues (inline comments posted)

• .github/configs/nvidia-master.yaml: Missing trailing newline at EOF
• .github/configs/nvidia-master.yaml:6856: Trailing whitespace + extra blank lines between 1k1k and 8k1k sections
• perf-changelog.yaml:524: Double space in "Add B200 configs..." → should be "Add B200 configs..."

LGTM with the minor nits above.

---

[jgangani]

@cquil11 Can you review and merge this? The CI runs actually finished here: https://github.com/InferenceMAX/InferenceMAX/actions/runs/21920062078/job/63297569384?pr=683

[cquil11]

@claude For all of the config file links in this diff, please pull the config and ensure that:

Nothing is hardcoded, I.e., nothing is cached, no harcoded paths specific to a system, no hardcoded IB devices, etc.
All of the upstream InferenceMAX master.yaml fields (ep, tp, dpa, etc) match up with the remote configs.

[Klaud-Cold]

Claude finished @cquil11's task in 2m 6s —— View job

---

Remote Config Validation Report

• Gather all 8 remote config URLs from the diff
• Fetch and inspect each remote config for hardcoded values
• Cross-reference master.yaml fields (ep, tp, dp-attn, num-worker) with remote configs
• Post final validation report

---

1. Hardcoded Values Check

All 8 remote configs were fetched and inspected. None contain:

• Hardcoded filesystem paths or cached paths
• System-specific paths (e.g., /home/user/..., /mnt/...)
• Hardcoded InfiniBand devices (e.g., mlx5_0, mlx5_1)
• Hardcoded IP addresses or hostnames
• Environment-specific values that would prevent portability

All configs use generic, portable references: model identifier deepseek-ai/DeepSeek-R1, relative model path dsr1, framework backends nixl/nats/trtllm_mla/flashinfer_trtllm, and standard SGLang environment variables.

Result: PASS — No hardcoded values found.

---

2. Master.yaml ↔ Remote Config Field Cross-Reference

For each search-space entry, I verified tp, ep, dp-attn, and num-worker in nvidia-master.yaml against tensor-parallel-size, expert-parallel-size, enable-dp-attention, and prefill_workers/decode_workers in the remote configs.

1k1k (ISL=1024, OSL=1024)

Recipe	Side	master tp	remote tp	master ep	remote ep	master dp-attn	remote dp-attn	master num-worker	remote workers	Match
dep4-1p-tep8-5d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-tep8-5d	Decode	8	8	8	8	false	(absent=false)	5	5	✅
dep4-1p-tep8-6d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-tep8-6d	Decode	8	8	8	8	false	(absent=false)	6	6	✅
dep4-1p-dep8-1d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-dep8-1d	Decode	8	8	8	8	true	true	1	1	✅
dep4-1p-dep8-2d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-dep8-2d	Decode	8	8	8	8	true	true	2	2	✅

8k1k (ISL=8192, OSL=1024)

Recipe	Side	master tp	remote tp	master ep	remote ep	master dp-attn	remote dp-attn	master num-worker	remote workers	Match
dep4-1p-tep8-1d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-tep8-1d	Decode	8	8	8	8	false	(absent=false)	1	1	✅
dep4-1p-tep8-5d	Prefill	4	4	4	4	true	true	1	1	✅
dep4-1p-tep8-5d	Decode	8	8	8	8	false	(absent=false)	5	5	✅
dep4-2p-tep8-5d	Prefill	4	4	4	4	true	true	2	2*	✅
dep4-2p-tep8-5d	Decode	8	8	8	8	false	(absent=false)	5	5	✅
tp4-1p-tp8-1d	Prefill	4	4	1	1	false	(absent=false)	1	1	✅
tp4-1p-tp8-1d	Decode	8	8	1	1	false	(absent=false)	1	1	✅

* dep4-2p-tep8-5d remote config has prefill_nodes: 2, prefill_workers: 2 matching master's num-worker: 2.

Result: PASS — All 16 prefill/decode entries match between master.yaml and the remote configs.

---

Summary

All 8 remote config files are clean (no hardcoded values) and all tp, ep, dp-attn, and num-worker fields in nvidia-master.yaml correctly match their corresponding remote config counterparts.

---