perf: enable HybridEP on H100/B200 x86 systems

[seonjinn]

Summary

• Make the same DeepEP HybridEP dependency available on Linux x86_64 and aarch64.
• Pin every dependency group and the lockfile to 17cfb817bccec3a9c247013360cc550c2bac441e.

Multi-node requirement

HYBRID_EP_MULTINODE defaults to 0 and is evaluated when DeepEP is built. Multi-node execution therefore requires a fresh wheel or actor environment built with HYBRID_EP_MULTINODE=1; recipe variables cannot change an existing single-node binary.

For H100/B200 x86, 8 means ranks per NVLink domain, not maximum EP size. The EP group must be divisible by 8. The H100 runs used the DOCA/NCCL path without a DeepEP source or DMA-BUF patch.

Validation

Model	H100 topology	EP	Result	W&B
Qwen3-30B-A3B	4 × 8 GPUs	8	20/20	run
Qwen3-235B-A22B	16 × 8 GPUs	16	20/20	run
Nemotron 3 Super 120B-A12B	32 × 8 GPUs	32	20/20	run

An exact, unpatched 17cfb817 wheel also completed 20/20 steps for Qwen3-30B-A3B on 4 × 8 B200 GPUs; the DMA-BUF workaround was not required.

These runs included #2964 and #3438. Nemotron 3 Super also used the Megatron-LM fixes from NVIDIA/Megatron-LM#5542 and NVIDIA/Megatron-LM#6114, so the results establish compatibility of the pin rather than isolation of this dependency-only diff.

Boundary and testing

• uv lock --check passed with 550 packages resolved.
• The 59/59 B200-hosted test from perf: enable HybridEP in H100/B200 performance recipes #3438 covers resolved recipes, not DeepEP runtime.
• The larger B200 Nemotron 3 Super/EP32 path remains unresolved; this PR does not vendor or rely on the experimental DMA-BUF workaround.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[seonjinn]

/ok to test 79f66b3

[seonjinn]

/ok to test 799351f

[seonjinn]

/ok to test a9aaa39

[seonjinn]

/ok to test 2e516bc

[seonjinn]

/ok to test d59c125

[yuki-97]

besides the two inline comments about test and patch, will need to rebase main since there's a conflict.

[yuki-97]

Would help to link an H100 run of one existing DeepEP recipe on pin 17cfb817 in the PR body — e.g. tests/test_suites/llm/grpo-qwen3.5-35ba3b-2n8g-automodel-ep16.sh.

None of the L0 / fast-L1 tiers turn on DeepEP, so their success here does not cover the actual path this bump touches.

[yuki-97]

I remember previously grpo-qwen3.5-35ba3b-2n8g-automodel-ep16 had some issue before bump deep_ep to 29d31c095796f3c8ece47ee9cdcc167051bbeed9 in #2224, so it's good to have a test on this.

[yuki-97]

On GB200, two things would help close the "Validation boundary" gap:

1. Could you link the B200 validation from perf: enable HybridEP in H100/B200 performance recipes #3438 (job 507030, 59/59 on head d68c6fa8) in the PR body? Since that PR depends on this bump, that job is effectively the runtime evidence for 17cfb817 on B200.

2. Is the local DMA-BUF fix still required on top of 17cfb817 (i.e. was job 507030 also patched)? If so, could you upload it so the shipped pin actually matches what B200 needs at runtime — otherwise people may meet issues.

[seonjinn]

Thanks — I re-checked the provenance.

For H100, multi-node runtime validation used the exact, unpatched DeepEP pin 17cfb817 and a wheel built with HYBRID_EP_MULTINODE=1 before the actor environment was created. The NeMo-RL validation stack also included #2964 and the matching recipe enablement from #3438, so this is runtime compatibility evidence for the pin rather than an isolation test of this dependency-only diff.

Model	H100 80 GB topology	HybridEP result	E2E TPS/GPU vs matched AllToAll	W&B
Qwen3-30B-A3B	4 nodes × 8 GPUs	20/20 steps	+0.60%	run
Qwen3-235B-A22B	16 nodes × 8 GPUs	20/20 steps	+4.34%	run
Nemotron 3 Super 120B-A12B	32 nodes × 8 GPUs	20/20 steps	+8.74%	run

W&B history was re-checked and contains train steps 1–20 for all three H100 runs. The multi-node flag is a build-time requirement at this pin; #3436 selects the source revision but does not itself rebuild an existing single-node DeepEP binary.

Job 507030 was a B200-hosted configuration test, not a DeepEP runtime test. It ran tests/unit/tools/test_hybridep_default_8g_recipes.py and passed 59/59; it did not build, import, or execute DeepEP. It was therefore not patched, and I do not think it should be presented as runtime evidence for 17cfb817. It is valid resolved-recipe coverage for #3438.

Separately, a 4-node × 8-GPU B200 Qwen3-30B-A3B run used an exact, unpatched wheel built from 17cfb817 and completed 20/20 steps with exit code 0. That establishes that the DMA-BUF fallback is not required for this validated B200 path.

The local DMA-BUF fallback was an experimental workaround for ibv_reg_mr errno=14 observed in a larger Super/EP32 configuration. The patched Super attempts did not establish a completed result because they later failed in other runtime paths, so there is not enough evidence to vendor/upload that workaround or make this dependency bump rely on it. The validation boundary should distinguish the 59/59 recipe test, the exact-pin Qwen runtime pass, and the still-unresolved large-EP corner case.

[seonjinn]

/ok to test 98d4b53

[seonjinn]

/ok to test adb76f7

[seonjinn]

Contributor self-review refresh for 919a6d102dc223cf3e19341e8a04eab30870aa8d after the latest main and lockfile refresh.

The DeepEP pin is consistent across x86 and aarch64 resolution paths, and lock/config checks pass. Two runtime evidence items remain before merge readiness: link or rerun the H100 recipe validation, and confirm whether the B200 DMA-BUF workaround is still required on the exact pinned stack.

[seonjinn]

/ok to test 0f35695