feat(modelopt): support real NVFP4 QAT rollout for MoE and Mamba by HollowMan6 · Pull Request #2983 · NVIDIA-NeMo/RL

HollowMan6 · 2026-06-29T03:01:17Z

What does this PR do ?

Add ModelOpt NVFP4 real-quant vLLM reload support for W4A16 fused-MoE & w4a4 rollout weights.

Issues

List issues that this PR closes (syntax):

Resolves Support QAT for low precision RL #1750

Summary

Add Nano3-specific/Qwen3 real-quant ignore profile so Mamba, attention, gates/routers, shared experts, norms, and selected sensitive layers stay in BF16.
Wire Megatron real-quant refit through export_hf_weights_modelopt(..., quant_mode="w4a16_nvfp4").
Add Nano3/Qwen3 W4A16 & W4A4 real-quant recipe.

Usage

Run the Nano3 W4A16 real-quant rollout recipe directly:

uv run --no-sync examples/run_grpo.py \
  --config examples/configs/recipes/llm/grpo-nanov3-30ba3b-4n4g-megatron-qa-nvfp4-w4a16-real.yaml

Or enable real-quant rollout on an existing Megatron + vLLM recipe:

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

copy-pr-bot · 2026-06-29T03:01:21Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

HollowMan6 · 2026-07-17T21:14:59Z

/ok to test 6a982e5

github-actions · 2026-07-17T21:19:30Z

✅ Submodule Fast-Forward Check Results

Check based on commit: 6a982e5 (PR #2983 from real_quant)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

terrykong

@sharonyu-115 to review

github-actions · 2026-07-18T06:30:17Z

✅ Submodule Fast-Forward Check Results

Check based on commit: 62c766b (PR #2983 from real_quant)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

terrykong

Nice feature — this closes out the real-quant rollout story from #1750, and the switch from the broad vLLM monkeypatch to registering ModelOpt NVFP4 through vLLM's public quantization registry (plus the new design doc) is a real improvement over the earlier revision.

This review was done by a multi-agent pass (6 agents) at head 62c766b. Summary of what was checked and found clean, plus a few informational notes:

Existing review threads: all 12 inline threads from @mxinO and @zhangyimi were verified as resolved at the current head (nano3 quant configs net-reverted, monkeypatch removed, backend selection moved into get_forward_loop_func(), rank-0 stale-cache warning added, etc.).
Verified clean: every vLLM 0.20.0 internal symbol used by vllm_modelopt.py/vllm_quant_backend.py was checked against the pinned wheel (incl. the retain-MoE-kernel-across-layerwise-reload design and the manual a1_gscale/a2_gscale copy-back — both correct); the Megatron-Bridge submodule bump (fast-forward, required for native w4a16_nvfp4 export) and modelopt 0.46.0.dev86 bump were verified required; 174 unit tests from the PR pass locally; pre-commit run --all-files passes at the PR head.
Super-120B recipes: the two 16-node real-quant recipes are registered as BF16/real-quant comparison smokes with no convergence claim — fine as-is; consider adding an evidence-table row in docs/guides/quantization-aware-rl.md once those runs complete.
Sibling PR heads-up: open PR #3012 (Simulated KV cache QARL) touches 6 of the same files, including vllm_quant_backend.py and megatron_quant_policy_worker.py — worth coordinating merge order to keep rebases cheap.

Inline comments: 5 for a dead-negative-assertion pattern in the new recipe scripts (with click-applyable fixes) and 2 for stale quant-config filenames in the guide.

Generated by Claude Code

HollowMan6 · 2026-07-19T03:46:07Z

/ok to test 81c44aa

github-actions · 2026-07-19T03:46:40Z

✅ Submodule Fast-Forward Check Results

Check based on commit: 81c44aa (PR #2983 from real_quant)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

zhangyimi · 2026-07-19T17:01:39Z

I reviewed the W4A4-related changes, and the implementation looks good to me. The training curves also look healthy. LGTM aside from the one remaining comment.

Signed-off-by: Hollow Man <hollowman@opensuse.org>

github-actions · 2026-07-20T06:48:47Z

✅ Submodule Fast-Forward Check Results

Check based on commit: 66c5545 (PR #2983 from real_quant)

✅ Submodules that are properly updated:

Megatron-Bridge: ✅ PR branch is ahead of main branch (fast-forward)

All submodule changes look good! ✨

HollowMan6 · 2026-07-20T06:51:15Z

/ok to test 66c5545

HollowMan6 requested a review from mxinO June 29, 2026 03:01

github-actions Bot added the Documentation Improvements or additions to documentation label Jun 29, 2026

HollowMan6 added CI:L1 Run doctests, unit tests, and functional tests Feature labels Jun 29, 2026

mxinO reviewed Jun 30, 2026

View reviewed changes

HollowMan6 requested a review from mxinO June 30, 2026 08:06

zhangyimi reviewed Jul 1, 2026

View reviewed changes

Comment thread examples/configs/recipes/llm/grpo-nanov3-30ba3b-4n4g-megatron-qa-nvfp4-w4a16-real.yaml

zhangyimi reviewed Jul 1, 2026

View reviewed changes

Comment thread nemo_rl/modelopt/models/generation/vllm_modelopt_patch.py Outdated

HollowMan6 force-pushed the real_quant branch 10 times, most recently from 93c66ac to be1d918 Compare July 7, 2026 21:51

HollowMan6 force-pushed the real_quant branch from be1d918 to f35d75a Compare July 11, 2026 08:46

HollowMan6 marked this pull request as ready for review July 11, 2026 08:46

HollowMan6 requested a review from a team as a code owner July 11, 2026 08:46

Copilot AI review requested due to automatic review settings July 11, 2026 08:46

HollowMan6 requested review from a team as code owners July 11, 2026 08:46

Copilot AI reviewed Jul 11, 2026

HollowMan6 force-pushed the real_quant branch from f35d75a to 76f200e Compare July 11, 2026 09:03

HollowMan6 force-pushed the real_quant branch from 8dda05f to 4cca5bf Compare July 17, 2026 20:23

NVIDIA-NeMo deleted a comment from cursor Bot Jul 17, 2026

HollowMan6 force-pushed the real_quant branch from 4cca5bf to 6a982e5 Compare July 17, 2026 21:14

NVIDIA-NeMo deleted a comment from cursor Bot Jul 17, 2026

NVIDIA-NeMo deleted a comment from github-actions Bot Jul 17, 2026

terrykong requested a review from sharonyu-115 July 18, 2026 01:06

terrykong reviewed Jul 18, 2026

View reviewed changes

HollowMan6 force-pushed the real_quant branch from 6a982e5 to 62c766b Compare July 18, 2026 06:29

terrykong reviewed Jul 18, 2026

View reviewed changes

zhangyimi reviewed Jul 19, 2026

View reviewed changes

Comment thread nemo_rl/modelopt/models/generation/vllm_modelopt.py

HollowMan6 added 9 commits July 19, 2026 11:50

feat(modelopt): support real w4a16 NVFP4 QAT rollout for MoE and Mamba

40145c8

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Add w4a4 real quant support

9c9e2d2

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Fix test cases

2b2a24b

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Clean up

9341ff8

Signed-off-by: Hollow Man <hollowman@opensuse.org>

fix ci

2ca1c1c

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Clean up and add docs

c923bdc

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Address review comments

7e760f3

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Migrate setup.py to use latest mbridge api

de8cdc5

Signed-off-by: Hollow Man <hollowman@opensuse.org>

support additional moe backend

66c5545

Signed-off-by: Hollow Man <hollowman@opensuse.org>

terrykong approved these changes Jul 20, 2026

View reviewed changes

terrykong mentioned this pull request Jul 22, 2026

build: bump vllm 0.20.0 → 0.25.1 #3280

Merged

Uh oh!

Conversation

HollowMan6 commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do ?

Issues

Summary

Usage

Before your PR is "Ready for review"

Additional Information

Uh oh!

copy-pr-bot Bot commented Jun 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

HollowMan6 commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

✅ Submodule Fast-Forward Check Results

✅ Submodules that are properly updated:

Uh oh!

terrykong left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 18, 2026

✅ Submodule Fast-Forward Check Results

✅ Submodules that are properly updated:

Uh oh!

terrykong left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HollowMan6 commented Jul 19, 2026

Uh oh!

github-actions Bot commented Jul 19, 2026

✅ Submodule Fast-Forward Check Results

✅ Submodules that are properly updated:

Uh oh!

Uh oh!

zhangyimi commented Jul 19, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026

✅ Submodule Fast-Forward Check Results

✅ Submodules that are properly updated:

Uh oh!

HollowMan6 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

HollowMan6 commented Jun 29, 2026 •

edited

Loading