Update Qwen3 235B A22B MXFP8 GB200/300 recipe and resolve NaN grad norm

[dingqingy-nv]

What does this PR do ?

• Update Qwen3 235B A22B MXFP8 GB200/300 recipe for better performance.
• Use no VP mapping that resolves NaN grad.

Additional Information

• Related to [Bug] Qwen3 235B A22B MXFP8 NaN grad norm #2096

Summary by CodeRabbit

• Chores
  • Updated QWEN3_235B model training configurations with adjusted parallelism settings and optimization parameters for GB300 and GB200 accelerator types.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR updates QWEN3 model pretraining workload configurations, adjusting expert model parallelism from 16 to 32, removing virtual pipeline model parallelism settings, and adding CUDA graph optimization scopes for attention and mixture-of-experts operations.

Changes

Cohort / File(s)	Summary
QWEN3 Workload Configuration scripts/performance/configs/qwen/qwen3_workload_base_configs.py	Updated GB300 FP8 CS V2 config: increased expert_model_parallel_size to 32, removed virtual_pipeline_model_parallel_size=12, and added cuda_graph_scope for attn/moe operations. Added expert_model_parallel_size=32 to GB200 FP8 CS V2 config.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~4 minutes

Possibly related PRs

• cp: Dsv3 Recipe Update (2152) into r0.3.0 #2186: Modifies the same model configuration parameters (expert_model_parallel_size, virtual_pipeline_model_parallel_size, cuda_graph_scope) across workload config files.

Suggested reviewers

• erhoo82

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	PR contains major changes affecting numerical stability and performance but provides no test results, performance benchmarks, or numerical validation to support claims.	Add detailed test results including convergence validation showing NaN gradient norm resolution, before-and-after performance metrics with hardware/configuration details, and evidence of no negative impact on training quality.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly addresses the main changes: updating Qwen3 235B A22B MXFP8 recipe for GB200/300 and resolving NaN gradient norm issues, which aligns with the actual modifications to the configuration file.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Important

Action Needed: IP Allowlist Update

If your organization protects your Git platform with IP whitelisting, please add the new CodeRabbit IP address to your allowlist:

• ✨ 136.113.208.247/32 (new)
• 34.170.211.100/32
• 35.222.179.152/32

Reviews will stop working after February 8, 2026 if the new IP is not added to your allowlist.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[erhoo82]

/ok to test 0a8447a