Skip to content

Update DSR1 B200 FP4 SGLang config with low-latency search space #1975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adibarra merged 4 commits into from
Jul 2, 2026
Merged

Update DSR1 B200 FP4 SGLang config with low-latency search space #1975

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
81f4f17
Update DSR1 B200 FP4 SGLang config with low-latency search space
RohitNagraj Jul 1, 2026
0c15e9a
Update perf-changelog pr-link for #1975
RohitNagraj Jul 1, 2026
bcc72e9
fix: point dsr1-fp4 b200-dgxc MODEL_PATH at DeepSeek-R1-0528-NVFP4-v2
RohitNagraj Jul 1, 2026
c8fb463
Merge remote-tracking branch 'origin/main' into HEAD
adibarra Jul 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1682,14 +1682,13 @@ dsr1-fp4-b200-sglang:
- isl: 1024
osl: 1024
search-space:
- { tp: 4, ep: 4, conc-start: 1, conc-end: 256 }
- { tp: 8, ep: 8, conc-list: [1] }
- { tp: 4, ep: 1, conc-start: 1, conc-end: 32 }
- { tp: 4, ep: 4, conc-start: 64, conc-end: 256 }
- isl: 8192
osl: 1024
search-space:
- { tp: 4, ep: 4, conc-start: 1, conc-end: 128 }
- { tp: 4, ep: 1, conc-start: 1, conc-end: 32 }
- { tp: 4, ep: 4, dp-attn: true, conc-start: 64, conc-end: 256 }
- { tp: 8, ep: 8, conc-list: [1] }
# agentic-coding: temporarily disabled — blocked by e2e-tests.yml artifact
# name mismatch (downloads `agentic_*` but benchmark-tmpl.yml uploads as
# `bmk_agentic_*`). Re-enable once that workflow is aligned.
Expand Down
8 changes: 8 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4392,3 +4392,11 @@
description:
- "Bump SGLang image from lmsysorg/sglang:deepseek-v4-blackwell (digest sha256:df18bfc4...) to mainline nightly lmsysorg/sglang:nightly-dev-cu13-20260628-da802ddc."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1923

- config-keys:
- dsr1-fp4-b200-sglang
description:
- "Update the B200 FP4 SGLang aggregate search space to low-latency lanes: TP4/EP1 for the low-concurrency range and TP4/EP4 for the higher-concurrency range."
- "Use 1k/1k TP4/EP1 c1-c32 and TP4/EP4 c64-c256; use 8k/1k TP4/EP1 c1-c32 and TP4/EP4 DP-attention c64-c256."
- "Drop the TP8/EP8 single-concurrency points."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1975
2 changes: 1 addition & 1 deletion runners/launch_b200-dgxc.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ set -x
# NOTE: per-node /raid/models/* would be faster but is only populated on a
# subset of dgxc nodes today, so we use Lustre for reliability.
if [[ $MODEL_PREFIX == "dsr1" && $PRECISION == "fp4" ]]; then
export MODEL_PATH="/lustre/fsw/models/dsr1-0528-nvfp4-v2"
export MODEL_PATH="/scratch/fsw/models/DeepSeek-R1-0528-NVFP4-v2"
export SRT_SLURM_MODEL_PREFIX="dsr1"
elif [[ $MODEL_PREFIX == "dsr1" && $PRECISION == "fp8" ]]; then
export MODEL_PATH="/lustre/fsw/models/dsr1-0528-fp8"
Expand Down