Skip to content

[Bugfix] Fix nemotron accuracy drop introduced by #41184 - #45037

Merged
robertgshaw2-redhat merged 3 commits into
vllm-project:mainfrom
neuralmagic:fix-nemotron
Jun 10, 2026
Merged

[Bugfix] Fix nemotron accuracy drop introduced by #41184#45037
robertgshaw2-redhat merged 3 commits into
vllm-project:mainfrom
neuralmagic:fix-nemotron

Conversation

@bnellnm

@bnellnm bnellnm commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Purpose

The routed_scaling_factor was getting applied twice in the Nemotron model. We needed to check apply_routed_scaling_factor_to_output before passing the scaling factor onto the RoutedExperts.

Test Plan

pytest -sv tests/evals/gsm8k/test_gsm8k_correctness.py::test_gsm8k_correctness[Nemotron-3-Super-120B-A12B-NVFP4] --config-list-file=configs/models-blackwell-ep.txt

Test Result

Test passed.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: Bill Nell <bnell@redhat.com>
@mergify mergify Bot added the bug Something isn't working label Jun 9, 2026
@DarkLight1337
DarkLight1337 enabled auto-merge (squash) June 9, 2026 16:08
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 9, 2026
@GirasoleY

Copy link
Copy Markdown
Contributor

Additional validation on another model that hits the same generic DeepSeek/FusedMoE routed-scale path:

  • Kimi-K2.5-NVFP4, GSM8K, NQ=100, dcp=1: current main before this fix was 25% accuracy; with this routed_scaling_factor fix it recovers to 92%.
  • This is the dense generic FusedMoE / ModelOpt NVFP4 path, not the sparse-attention path, so it validates the same double-application of routed_scaling_factor beyond Nemotron.

This matches the root cause described in this PR: when apply_routed_scale_to_output=True, RoutedExperts must receive the router-side scale (1.0) while MoERunner owns the output-side scale.

@robertgshaw2-redhat
robertgshaw2-redhat merged commit 2902668 into vllm-project:main Jun 10, 2026
78 of 80 checks passed
wcynb1023 pushed a commit to wcynb1023/vllm that referenced this pull request Jun 11, 2026
Saddss pushed a commit to Saddss/vllm that referenced this pull request Jun 14, 2026
vivek8123 pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Jun 18, 2026
divineearthly pushed a commit to divineearthly/vllm that referenced this pull request Jun 19, 2026
…llm-project#45037)

Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: divineearthly <divineearthly@gmail.com>
nkzhenhua pushed a commit to nkzhenhua/vllm that referenced this pull request Jun 24, 2026
Dao007forever pushed a commit to Dao007forever/vllm that referenced this pull request Jul 18, 2026
philippesic pushed a commit to philippesic/vllm-semantic-cache that referenced this pull request Jul 19, 2026
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants