[perf,recipe] Fix Qwen3 30B A3B B200 perf config to use hybridep+flex dispatcher

[tomlifu]

What does this PR do ?

This PR fixes Qwen3 30B A3B B200 perf config to use hybridep+flex dispatcher

Changelog

The B200 workload configs for qwen3_30b_a3b were missing several performance-critical settings compared to other GPU variants (GB200, GB300, B300, H100):

1. moe_flex_dispatcher_backend was not set to "hybridep" (inherited "deepep" from BASE_QWEN3_30B_A3B_CONFIG), preventing the optimized hybrid EP dispatcher from being used.

2. moe_token_dispatcher_type was hardcoded to "alltoall" in qwen3_30b_a3b_pretrain_config_b200(), which silently disables the flex dispatcher backend even when the override is passed via CLI. Changed to "flex" to match all other GPU variants.

3. micro_batch_size=4 was not set, leaving the field unset.

4. "attn" was missing from cuda_graph_scope, losing CUDA graph coverage for attention kernels.

Validated on 8x B200 single-node with mbs=4, gbs=512, seq=4096:

• BF16: ~546 TFLOP/s/GPU
• FP8_MX: ~555 TFLOP/s/GPU

GitHub Actions CI

See the CI sectionin the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

• Related to # (issue)

Summary by CodeRabbit

• Chores
  • Updated Qwen model training configurations for improved performance and efficiency.
  • Adjusted batch processing parameters and token distribution strategies to optimize training throughput.
  • Extended computation optimization scope to include additional model components, enhancing training performance.
  • Standardized core parameters across training configuration variants for consistency.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

Configuration updates to QWEN3 training parameters on GB200 hardware, including changes to MOE token dispatcher type from "alltoall" to "flex" and updates to micro-batch size, flex dispatcher backend, and CUDA graph scope settings across three B200 FP8 workload configurations.

Changes

Cohort / File(s)	Summary
MOE Token Dispatcher Update scripts/performance/configs/qwen/qwen3_llm_pretrain.py	Changed MOE token dispatcher type from "alltoall" to "flex" in the GB200 pretrain configuration.
Workload Configuration Updates scripts/performance/configs/qwen/qwen3_workload_base_configs.py	Updated three B200 FP8 configs (V1, CS_V1, MX_V1) to use micro_batch_size=4, moe_flex_dispatcher_backend="hybridep", and extended cuda_graph_scope to include "attn". Converted MX_V1 from alias to explicit config definition.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• Update Qwen3 30B H100 Base Configs with HybridEP #2477: Modifies MoE token dispatching and dispatcher backend settings in the same qwen configuration files, switching dispatcher type to "flex" and updating moe_flex_dispatcher_backend to "hybridep".
• cp: Update Qwen3 235B A22B MXFP8 GB200/300 recipe and resolve NaN grad norm (2209) into r0.3.0 #2210: Updates QWEN3 training config constants in the same file with related changes to cuda_graph_scope and MOE-related settings.
• DSV3 NVFP4 recipe on GB300 #2076: Modifies MOE dispatcher backend settings across pretrain workload configs with moe_flex_dispatcher_backend set to "hybridep".

Suggested labels

performance, performance/optimize

Suggested reviewers

• ko3n1g
• yaoyu-33
• erhoo82

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	PR includes after-deployment performance numbers (BF16: ~546 TFLOP/s/GPU, FP8_MX: ~555 TFLOP/s/GPU) but lacks before-deployment baseline for comparison.	Add baseline performance numbers from original configuration to demonstrate that dispatcher backend and type changes actually improve performance.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately and specifically summarizes the main change—fixing B200 performance config to use hybridep+flex dispatcher for Qwen3 30B A3B.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

scripts/performance/configs/qwen/qwen3_workload_base_configs.py (1)

411-418: B200_FP8_MX_V1 content is identical to B200_FP8_CS_V1 — and differs from GB200_FP8_MX_V1.

Two things worth noting:

1. Duplication: the new explicit replace(BASE_QWEN3_30B_A3B_CONFIG, ...) body is byte-for-byte identical to QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_CS_V1 (lines 401–408). B300 and GB300 handle this case with a simple alias (MX = CS). If CS and MX are meant to stay in sync here, an alias would avoid silent divergence.

   ♻️ Optional: revert to alias pattern (consistent with B300/GB300)

   -QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_MX_V1 = replace(
   -    BASE_QWEN3_30B_A3B_CONFIG,
   -    num_gpus=8,
   -    micro_batch_size=4,
   -    moe_flex_dispatcher_backend="hybridep",
   -    cuda_graph_impl="transformer_engine",
   -    cuda_graph_scope=["attn", "moe_router", "moe_preprocess"],
   -)
   +QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_MX_V1 = QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_CS_V1

2. Asymmetry with GB200 FP8 MX: QWEN3_30B_A3B_PRETRAIN_CONFIG_GB200_FP8_MX_V1 (line 364) uses cuda_graph_scope=["moe_router", "moe_preprocess"] — it intentionally excludes "attn". The new B200 FP8 MX includes "attn". Please confirm this divergence from the GB200 FP8 MX pattern is deliberate (e.g., due to different MX kernel support on B200 standalone vs. the GB200 NVLink array).

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@scripts/performance/configs/qwen/qwen3_workload_base_configs.py` around lines
411 - 418, The new QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_MX_V1 is byte-for-byte
identical to QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_CS_V1, creating duplication
and risking silent drift; either make B200_FP8_MX_V1 an alias of B200_FP8_CS_V1
(like the B300/GB300 pattern) or deliberately keep the explicit replace but
document why; also verify whether the cuda_graph_scope for B200_FP8_MX_V1 should
include "attn" (it currently does) or match
QWEN3_30B_A3B_PRETRAIN_CONFIG_GB200_FP8_MX_V1 which omits "attn"—adjust the
cuda_graph_scope accordingly or add a comment explaining the intentional
divergence from GB200_FP8_MX_V1.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@scripts/performance/configs/qwen/qwen3_workload_base_configs.py`:
- Around line 411-418: The new QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_MX_V1 is
byte-for-byte identical to QWEN3_30B_A3B_PRETRAIN_CONFIG_B200_FP8_CS_V1,
creating duplication and risking silent drift; either make B200_FP8_MX_V1 an
alias of B200_FP8_CS_V1 (like the B300/GB300 pattern) or deliberately keep the
explicit replace but document why; also verify whether the cuda_graph_scope for
B200_FP8_MX_V1 should include "attn" (it currently does) or match
QWEN3_30B_A3B_PRETRAIN_CONFIG_GB200_FP8_MX_V1 which omits "attn"—adjust the
cuda_graph_scope accordingly or add a comment explaining the intentional
divergence from GB200_FP8_MX_V1.

---

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4a64507 and f96789d.

📒 Files selected for processing (2)

• scripts/performance/configs/qwen/qwen3_llm_pretrain.py
• scripts/performance/configs/qwen/qwen3_workload_base_configs.py

[erhoo82]

/ok to test d622b37

[ko3n1g]

Did we test this internally and update golden values?

Blocking, but @malay-nagda can override

[malay-nagda]

Did we test this internally and update golden values?

Blocking, but @malay-nagda can override

yes. created a PR for that too.