feat(video): enable canonical Nemotron Omni training for V2 MoE checkpoints - #5304
Merged
Conversation
yaoyu-33
added
area:model
ehosseiniasl
force-pushed
the
ehsan/video-rl-gym-async-grpo
branch
from
August 4, 2026 23:22
e3dd310 to
49f70a7
Compare
This was referenced Aug 4, 2026
ehosseiniasl
force-pushed
the
ehsan/video-rl-gym-async-grpo
branch
from
August 5, 2026 14:29
49f70a7 to
b10cfed
Compare
Route V2-labeled Nano Omni MoE configs through the canonical expanded-sequence bridge while preserving dense V2 behavior. Keep MCore causal masks out of media token-validity filtering to avoid broadcasted placeholder counts in dense training. Signed-off-by: Ehsan Hosseini Asl <ehsan.hosseiniasl@gmail.com>
ehosseiniasl
force-pushed
the
ehsan/video-rl-gym-async-grpo
branch
from
August 7, 2026 14:51
b10cfed to
bb0d604
Compare
ehosseiniasl
changed the title
3 tasks
cuichenx
approved these changes
Aug 7, 2026
cuichenx
left a comment
cuichenx
left a comment
Contributor
There was a problem hiding this comment.
LGTM Thanks for rebasing
Contributor
|
/ok to test bb0d604 |
cuichenx
added
ready-to-merge
yaoyu-33
removed
the
waiting-on-customer
yaoyu-33
added
full-test-suite
needs-more-tests
Contributor
|
LGTM too. Thanks for the fix. |
aroshanghias-nvd
removed
the
needs-more-tests
ananthsub
pushed a commit
to NVIDIA-NeMo/Gym
that referenced
this pull request
Aug 19, 2026
## Summary Add video-input support to the NeMo Gym Responses API and vLLM model backend for multimodal RL training. - Accept `input_video` and `video_url` content parts. - Convert Responses API video inputs into vLLM-compatible chat content. - Reject video parts that do not contain a valid URL. - Prefer prompt and generation token IDs returned by the generation request. - Preserve multimodal processor arguments when falling back to the tokenize endpoint. - Validate that generation token IDs and log probabilities have matching lengths. - Propagate token IDs, log probabilities, and optional training metadata through Gym responses. - Ensure component processes load their owning Gym checkout instead of a stale container installation. ## Motivation Video GRPO requires Gym to preserve video content and the exact prompt and generation tokenization used by vLLM. Using token IDs produced by a separate or differently configured tokenization request can cause rollout and policy log probabilities to reference different token sequences, resulting in incorrect training data and elevated TMPE. ## Validation - Added coverage for `input_video` and `video_url` conversion. - Added coverage for missing video URLs. - Added coverage for native vLLM prompt and generation token IDs. - Added coverage for the tokenize fallback and multimodal processor arguments. - Added coverage for token-ID and log-probability propagation. - Added coverage ensuring component processes prefer the owning Gym checkout. - Exercised through downstream synchronous and asynchronous NeMo RL video GRPO training. ## Dependencies None. ## Related integrations - NVIDIA-NeMo/Megatron-Bridge#5304 `feat(video): enable canonical Nemotron Omni training for V2 MoE checkpoints` - NVIDIA-NeMo/RL#3500 `feat(video): add Gym support for sync and async GRPO` --------- Signed-off-by: Ehsan Hosseini Asl <ehsan.hosseiniasl@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enable canonical Nemotron Omni video training for Nano Omni MoE checkpoints serialized with the historical
NemotronH_Nano_VL_V2architecture label.NemotronOmniModelProvider.NemotronVLModelbehavior and legacy mappings for dense V2 checkpoints.Motivation
The Nano Omni checkpoint used for video RL is serialized with the historical
NemotronH_Nano_VL_V2architecture label. However, its MoE architecture and processor-expanded multimodalsequence require the canonical
NemotronOmniModel.During media insertion, only a 2-D padding or token-validity mask may filter placeholders. MCores 4-D causal mask remains unchanged for language-model attention; treating it as a validity
mask produces incorrect visual-feature counts.
This change selects the canonical model and mappings for MoE-shaped V2 checkpoints while preserving backward compatibility for dense V2 checkpoints.
Validation
NemotronOmniModelProvider.Dependencies
The canonical Nemotron Omni implementation from MBridge #5007 is already present in the target branch.
This PR does not modify the Megatron-LM submodule pointer.
Downstream integrations
feat(video): support video inputs in vLLM responsesfeat(video): add Gym support for sync and async GRPO