feat(nemotron_3.5_super): report the model under a caller-supplied name - #2786
Merged
Conversation
The P/D launcher derived the reported model name from the serve path: `vllm serve "$MODEL"` and `++policy_model_name=$MODEL` read the same variable. A caller that mounts every checkpoint at one fixed container path therefore reports every run under that path, and cannot tell two checkpoints apart in the rollouts. MODEL_NAME now names the model independently of where it is mounted. vLLM serves under it and the eval reports it, so the two stay consistent without the caller having to move the mount. It defaults to $MODEL, which is what vLLM would advertise anyway, so unset behaviour is unchanged. --served-model-name is passed before VLLM_COMMON_ARGS, so a VLLM_CONFIG that sets its own still wins. Signed-off-by: plaszkiewicz <plaszkiewicz@nvidia.com>
Signed-off-by: plaszkiewicz <plaszkiewicz@nvidia.com>
gchlebus
approved these changes
Aug 26, 2026
laszkiewiczp
enabled auto-merge (squash)
August 26, 2026 10:11
Contributor
Author
|
/ok to test fd05aae |
Contributor
Author
|
Data point on the cleanup-job bug I mentioned above, in case it helps prioritise it. Three direct P/D runs today, all of which failed to submit the cleanup job with Running
2804 in total, from about sixteen minutes of two runs that never reached the end of the benchmark. All reaped now. The script itself works fine — it is only the submission that fails, because |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In
benchmarks/nemotron_3.5_super/sbatch_external_vllm.shthe reported model name is derived from the serve path — one variable feeds both:Under the current coupling, the only way to give a run a meaningful name is to rename the mount, and that is a trap: anything else addressing that mount by path (
--chat-template /checkpoint/...,--reasoning-parser-plugin /checkpoint/...) silently points at a path that is no longer mounted, and vLLM fails to load it. We hit exactly that.Change
MODEL_NAMEnames the model independently of where it is mounted:--served-model-name), so the endpoint answers to it.++policy_model_name), so the two cannot disagree.This lets the caller keep the mount fixed and still say which checkpoint a run used.
Tested on a run