Repository navigation
[Bugfix] Load and apply Nemotron Super final vision LayerNorm - #61401
Conversation
Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com> Signed-off-by: Shaun Kotek <skotek@nvidia.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
mergify
Bot
added
multi-modality
|
/ci run |
|
✅ Triggered Buildkite CI #94461 for commit |
|
✅ @natsala13, CI is now available for this PR.
|
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #94461. |
|
Post-merge CI coverage audit found that the changed regression file was not executed by the PR build. In exact build #94461, the H200 Multimodal Models Standard 4 lane did run, but its command is This is a test-wiring gap, not evidence that the source fix is wrong. Please add the file/cases to an automatically executed lane (for example, by applying the intended marker if Standard 4 is the correct home) and rerun that exact lane so the regression is actually gated. |
Signed-off-by: Shaun Kotek <skotek@nvidia.com> Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
…roject#61401) Signed-off-by: Shaun Kotek <skotek@nvidia.com> Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com> Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
…roject#61401) Signed-off-by: Shaun Kotek <skotek@nvidia.com> Co-authored-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
Continuing @shaunkotek's work from #59664; we are taking it from here in this PR. This branch rebases Shaun's change onto upstream main without changing the patch. The original description and validation claims are copied below; tests were not rerun as part of the rebase.
Overview
Nemotron Super vision checkpoints include
vision_projector.vision_final_layernorm.{weight,bias}, but the native loader silently discards these tensors and omits the layer from feature extraction. Load and apply the final vision LayerNorm.Claims
Validation
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.