feat(minf-mm-sc/tq): Support SC/TQ multimodal inference. - #3921

Closed

cspades wants to merge 18 commits into

mainfrom

cye/rl_mllm_omni_mm_nocolgym_rebase_pr3727

Commits on Aug 31, 2026

ci: bump Megatron-Bridge to b11414c7, emerging-optimizers to v0.3.0

kajalj22
authored and
cspades
committed
85d7601
Fix CUDA IMA from using MambaMixer with NRL offloading.
cspades
committed
1235709
Support Megatron-LLM inference backend for multimodal RL.
cspades
committed
63eda8b
Post-rebase housekeeping.
cspades
committed
089e6f5
Pass pre-expanded media tokens to MLLM.
cspades
committed
ff243c0
Remove scripts from branch.
cspades
committed
3ac9345
Deprecate media_cache_key.
cspades
committed
9bbb32e
Preserve bounded image normalization after rebase.
cspades
committed
99e9ccd
Align multimodal engine state with main lifecycle.
cspades
committed
1fce161
More rebase fixes.
cspades
committed
f1318c3
Retreat the model prompt config back to the wrapper. Users can just create a new wrapper for a new config.
cspades
committed
c6c3004
Add tests and lint.
cspades
committed
9c5886b
Address Claude and Fable reviews
cspades
committed
dfb5487
Address more review feedback.
cspades
committed
66f96ea
Add video_model_type to RL mcore gen config, and lint.
cspades
committed
f755067
Support multimodal Megatron-Inference with NeMo-RL TransferQueue (V2).
cspades
committed
6d6eabd
WAR fix for multimodal request checkpoint embedding recompute.
cspades
committed
42526bb

Commits on Sep 1, 2026

Address review feedback and incorporate code from #2957.
cspades
committed
963d5e3