Skip to content

fix: support canonical Nemotron 3.5 Super checkpoint - #34

Merged
TomerBN-Nvidia merged 1 commit into
super_vl_rl_v0.25.1from
tbn/nemotron-3.5-super-minimal-compat
Aug 6, 2026
Merged

TomerBN-Nvidia merged 1 commit into
super_vl_rl_v0.25.1from
tbn/nemotron-3.5-super-minimal-compat

Conversation

@TomerBN-Nvidia

@TomerBN-Nvidia TomerBN-Nvidia commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Why

Enable the canonical nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth checkpoint directly on the custom release branch, without editing checkpoint metadata or passing --hf-overrides.

The checkpoint advertises NemotronH_Omni_Reasoning_V3, uses a flat Transformers RadioConfig, and stores dynamic image-processing bounds in preprocessor_config.json while processor_config.json has no nested image processor.

What changed

  • Register the checkpoint's real architecture and reuse the Nano Nemotron VL implementation.
  • Adapt only the known RADIO-H/16 signature (1280, 32, 16, 5120) used by this checkpoint.
  • Preserve its explicit CLS/register/summary-token layout and native vision/projector weight names.
  • Recover preprocessor_config.json when Transformers returns an empty image-processor config, preserving the checkpoint's 1024..13312 dynamic patch bounds.
  • Build Nemotron-H MTP layers from the draft model config, accept language_model.mtp.*, and set the Nano Nemotron image token ID for multimodal drafting.

Relationship to #33

This is the narrower replacement for #33 on the canonical release line. #33 was merged only into the older BTK preparation branch; it is not present on super_vl_rl_v0.25.1. Its generic flat-RADIO conversion did not register the checkpoint's real architecture and did not preserve the dynamic processor bounds, so the r7 serving test required architecture and processor overrides.

Rather than revert the obsolete preparation branch, this PR is rebased directly onto super_vl_rl_v0.25.1 after merged PR #37.

Validation

  • HSG job 5916941: mounted the exact pre-fix: support Nemotron 3.5 Super Omni serving and MTP #33 + this PR production tree, plus PR fix: unpack FA4 auxiliary outputs #37's FA4 fix, into r7.
    • one 4xGB200 node, TP4+EP
    • no --hf-overrides
    • resolved NemotronH_Omni_Reasoning_V3 directly
    • default FLASH_ATTN VIT backend
    • dynamic encoder profiling enabled
    • 3/3 image requests returned HTTP 200
    • Slurm completed 0:0, final status PASS
  • Python syntax compilation passed for all changed files.
  • Pre-commit passed for all changed files using the repository's Python 3.12 environment, including ruff, mypy, SPDX, forbidden-import, configuration, and attention-backend checks.

Duplicate-work check

No open upstream PR was found for NemotronH_Omni_Reasoning_V3. Upstream vllm-project#39767 concerns missing legacy layer-scale tensors and is intentionally not included.

AI assistance

OpenAI Codex assisted with investigation, implementation, rebasing, and validation. The human maintainer requested this comparison and replacement and must understand the changed lines before merge.

@TomerBN-Nvidia
TomerBN-Nvidia force-pushed the tbn/nemotron-3.5-super-minimal-compat branch from da16be6 to a19a890 Compare August 3, 2026 13:06
Load the canonical Nemotron 3.5 Super multimodal and MTP config, map its native RADIO checkpoint names, and preserve its dynamic-resolution preprocessing metadata.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: tbarnatan <tbarnatan@nvidia.com>
@TomerBN-Nvidia
TomerBN-Nvidia force-pushed the tbn/nemotron-3.5-super-minimal-compat branch from a19a890 to 436aefb Compare August 6, 2026 09:19
@TomerBN-Nvidia
TomerBN-Nvidia changed the base branch from tbn/super-vl-bf16-btk-prep to super_vl_rl_v0.25.1 August 6, 2026 09:19
@TomerBN-Nvidia
TomerBN-Nvidia marked this pull request as ready for review August 6, 2026 09:20
@TomerBN-Nvidia
TomerBN-Nvidia merged commit b31cc13 into super_vl_rl_v0.25.1 Aug 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant