feat: Add Nemotron‑3 Nano 30B A3B GRPO nightly tests (FSDP2, +LoRA)

[RayenTian]

What does this PR do ?

• ❗Load Lora ckpt failed:
  • Root cause: the forward of NemotronHMamba2Mixer module may use cuda_kernels_forward, and this will lead to out_proj lora has no gradient. This can also explain why the accuracy curve of lore and w/o lora have a slight mismatch.
  • Solution: exclude *out_proj* modules in lora layers.
  • More tests are undergoing.
• ❗ Use nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 as tokenizer and nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 as model.
  • Because base model doesn't provide default chat template. And if we use chat template of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16, it's too long to write in yaml directly, then an additional file needs to be added to store the chat template.

Convergence Test

Different configs have different accuracy, choose dim=128, alpha=512 as the recipe config.

SFT

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

Summary by CodeRabbit

• Documentation

  • Added LoRA configuration guidance for GRPO training with DTensor backend support and inference strategies.

• New Features

  • Added GRPO configuration examples for NVIDIA Nemotron-3 Nano-30B models with and without LoRA.

• Tests

  • Added test suite entries for new GRPO configurations with corresponding automated validation scripts.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds comprehensive GRPO (Generative Reward Process Optimization) support for the Nanov3 model, including documentation of DTensor LoRA configuration, two YAML recipe variants (with and without LoRA), corresponding test scripts with metric validation, and test suite registration.

Changes

Cohort / File(s)	Summary
Documentation docs/guides/grpo.md	Added LoRA Configuration section describing DTensor backend setup, merge-weight approach for generation, training-inference parity considerations, and reference to example YAML recipe.
Configuration Recipes examples/configs/recipes/llm/grpo-nanov3-30BA3B-2n8g-fsdp2.yaml, examples/configs/recipes/llm/grpo-nanov3-30BA3B-2n8g-fsdp2-lora.yaml	Two YAML configuration files for GRPO Nanov3 experiments. LoRA variant includes DTensor LoRA configuration (dim 128, alpha 256); both define model, batch sizes, sequence length, generation parameters (tensor_parallel_size 4, gpu_memory_utilization 0.7), logging, and cluster allocation (8 GPUs per node, 2 nodes).
Test Scripts tests/test_suites/llm/grpo-nanov3-30BA3B-2n8g-fsdp2.sh, tests/test_suites/llm/grpo-nanov3-30BA3B-2n8g-fsdp2-lora.sh	New test scripts that execute GRPO experiments, convert TensorBoard logs to JSON, and validate metrics including training loss, reward values, and timing thresholds. Scripts include conditional cleanup of checkpoint directories on success.
Test Suite Registration tests/test_suites/nightly.txt	Added two nightly test entries for Nanov3 LoRA and standard GRPO configurations under the Nano-v3 heading.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• cp: feat: Megatron SFT LoRA (1629) into r0.5.0 #1741: Adds DTensor LoRA configuration support referenced in this PR's documentation and recipes.
• chore: add nanov3 lora sft recipe to doc #1860: Documents Nanov3 LoRA SFT recipe referenced alongside GRPO LoRA configuration details.

Suggested labels

documentation

Suggested reviewers

• yuki-97
• terrykong

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	PR introduces major changes (new model configs, LoRA support, nightly tests) affecting numerics/convergence but PR description lacks test results, baseline metrics, performance numbers, and regression analysis.	Update PR description with baseline metrics, test results, convergence validation, and justification of metric thresholds for the Nano v3 GRPO configuration.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The pull request title accurately summarizes the main changes: adding GRPO nightly tests for Nemotron-3 Nano 30B A3B with FSDP2 and LoRA support, which aligns with all file additions (test configs, test scripts, documentation, and nightly test registry).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ruit/nanov3_grpo_recipe_dtensor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[terrykong]

awesome!

can you:

1. retroactively add lora to the news section in the front page readme? some users did not know we had this supported. maybe add an item that lora sft is supported in both backends, and dtensor grpo supported and to see our nano v3 recipe (and mention mcore coming soon)
2. do you think we'd see more interesting reward curves if we did base for model_name but the reasoning model for the tokenizer_name?

[RayenTian]

awesome!

can you:

1. retroactively add lora to the news section in the front page readme? some users did not know we had this supported. maybe add an item that lora sft is supported in both backends, and dtensor grpo supported and to see our nano v3 recipe (and mention mcore coming soon)
2. do you think we'd see more interesting reward curves if we did base for model_name but the reasoning model for the tokenizer_name?

Done for both 1 and 2. Attatched new running curve to PR doc.