fix(runners): scope b200-dgxc qwen3.5 fp4 model path to the V2 checkpoint / 修复:将 b200-dgxc 的 qwen3.5 fp4 模型路径限定到 V2 检查点 - #2414
Merged
Ankur-singh merged 1 commit intoJul 29, 2026
Conversation
…oint / 修复:将 b200-dgxc 的 qwen3.5 fp4 模型路径限定到 V2 检查点 The qwen3.5 + fp4 MODEL_PATH branch in launch_b200-dgxc.sh keyed only on MODEL_PREFIX and PRECISION, but two checkpoints now share that pair: qwen3.5-fp4-b200-sglang / -sglang-mtp -> nvidia/Qwen3.5-397B-A17B-NVFP4-V2 qwen3.5-fp4-b200-trt / -trt-mtp -> nvidia/Qwen3.5-397B-A17B-NVFP4 Since this script later does `export MODEL="$MODEL_PATH"`, the shared branch made the two TRT configs serve V2 weights while still publishing results under the old checkpoint name whenever they landed on a b200-dgxc_* runner (launch_b200-cw.sh / launch_b200-nb.sh pin no path, so the same key served the old weights there -- i.e. runner-dependent). Introduced by 25ef4a1, which reverted the TRT keys to NVFP4 but left the launcher pointing at the V2 path. Branch on the checkpoint so each key resolves to its own weights. The old-checkpoint arm restores the exact /lustre path used on main, so TRT behaviour is unchanged and needs no re-sweep. 中文:launch_b200-dgxc.sh 中 qwen3.5 + fp4 的 MODEL_PATH 分支仅依据 MODEL_PREFIX 与 PRECISION 判断,而这两个字段现已对应两个不同的检查点,导致 TRT 配置在 b200-dgxc 运行器上会加载 V2 权重,却仍以旧检查点名称发布结果。 改为按检查点分支:V2 走 /scratch,旧检查点恢复为 main 上的 /lustre 路径, 因此 TRT 行为保持不变,无需重跑 sweep。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Targets
qwen3.5-fp4-nvfp4-v2(the branch behind #2205), notmain.Problem
In
runners/launch_b200-dgxc.shat #2205's head, the model-path branch keys only onMODEL_PREFIX+PRECISION:But two checkpoints now share that pair:
model:at headqwen3.5-fp4-b200-sglang,-sglang-mtpnvidia/Qwen3.5-397B-A17B-NVFP4-V2qwen3.5-fp4-b200-trt,-trt-mtpnvidia/Qwen3.5-397B-A17B-NVFP4Because the script later does
export MODEL="$MODEL_PATH", the two TRT configs would serve V2 weights while publishing results under the old checkpoint name.It is also runner-dependent: the
b200pool spansb200-cw_*,b200-nb_*andb200-dgxc_*, and onlylaunch_b200-dgxc.shpins a path. So the same config key served old weights on cw/nb and V2 on dgxc, depending on scheduling.Origin: 25ef4a1 ("remove trt configs") reverted the TRT keys to
NVFP4but left the launcher on the V2 path — a partial revert.Fix
Branch on the checkpoint so each key resolves to its own weights. The old-checkpoint arm restores the exact
/lustrepath used onmain.Verification
Extracted the real resolution chain from the file and ran the configs through it:
…NVFP4-V2(sglang)/lustre/…NVFP4/scratch/…NVFP4-V2/scratch/…NVFP4-V2✅…NVFP4(trt)/lustre/…NVFP4/scratch/…NVFP4-V2❌/lustre/…NVFP4✅bash -nclean;fp8andbf16qwen3.5 arms unaffected.Because the TRT arm now resolves identically to
main, TRT behaviour is unchanged and no re-sweep is needed; #2205's existing sweep covers the sglang keys it actually changes.Draft because #2205 itself still has open items (a
CHANGES_REQUESTEDreview, and the SGLang cookbook still documenting the pre-V2 checkpoint). Merging this into that branch changes its head SHA, so its sweep evidence will need a/reuse-sweep-runauthorization afterwards.中文:
launch_b200-dgxc.sh的 qwen3.5+fp4 分支仅依据 model-prefix 与 precision 判断,而这两个字段现对应两个检查点,导致 TRT 配置在 b200-dgxc 运行器上加载 V2 权重却以旧检查点名称发布结果。改为按检查点分支;旧检查点恢复 main 上的 /lustre 路径,故 TRT 行为不变、无需重跑 sweep。