Skip to content

perf: enable HybridEP on H100/B200 x86 systems #3436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
seonjinn wants to merge 15 commits into
base: main
Choose a base branch
from
Open

perf: enable HybridEP on H100/B200 x86 systems #3436

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
e0a23cc
build: use HybridEP on x86 and aarch64
seonjinn Jul 31, 2026
589375e
test: validate cross-platform HybridEP lock source
seonjinn Jul 31, 2026
79f66b3
refactor: unify HybridEP dependency across platforms
seonjinn Jul 31, 2026
799351f
style: format HybridEP dependency test
seonjinn Jul 31, 2026
6ba59a8
Merge remote-tracking branch 'origin/main' into sna/hybridep-x86-runt…
seonjinn Aug 3, 2026
a9aaa39
build: pin HybridEP to 17cfb817
seonjinn Aug 3, 2026
2e516bc
Merge origin/main into sna/hybridep-x86-runtime-20260730
seonjinn Aug 9, 2026
1261ded
test: remove redundant HybridEP dependency test
seonjinn Aug 11, 2026
d59c125
Merge remote-tracking branch 'origin/main' into sna/hybridep-x86-runt…
seonjinn Aug 11, 2026
34e8acf
Merge remote-tracking branch 'origin/main' into sna/hybridep-x86-runt…
seonjinn Aug 12, 2026
98d4b53
build: refresh lock after main merge
seonjinn Aug 12, 2026
adb76f7
Merge remote-tracking branch 'origin/main' into sna/hybridep-x86-runt…
seonjinn Aug 13, 2026
04b1cf7
Merge remote-tracking branch 'origin/main' into sna/pr3436-pin-fix-20…
seonjinn Aug 13, 2026
919a6d1
Merge branch 'main' into sna/hybridep-x86-runtime-20260730
seonjinn Aug 18, 2026
0f35695
Merge remote-tracking branch 'origin/main' into review/pr3436-main-sy…
seonjinn Aug 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,17 +132,15 @@ automodel = [
"causal-conv1d",
"nv-grouped-gemm",
"transformer-engine[pytorch,core_cu13] @ git+https://github.com/NVIDIA/TransformerEngine.git@v2.14.1",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@29d31c095796f3c8ece47ee9cdcc167051bbeed9 ; platform_machine == 'x86_64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@a48493600c4886c1b297aaa78db0e1ebc2d8dd6c ; platform_machine == 'aarch64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@17cfb817bccec3a9c247013360cc550c2bac441e",

@yuki-97 yuki-97 Aug 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would help to link an H100 run of one existing DeepEP recipe on pin 17cfb817 in the PR body — e.g. tests/test_suites/llm/grpo-qwen3.5-35ba3b-2n8g-automodel-ep16.sh.

None of the L0 / fast-L1 tiers turn on DeepEP, so their success here does not cover the actual path this bump touches.

@yuki-97 yuki-97 Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember previously grpo-qwen3.5-35ba3b-2n8g-automodel-ep16 had some issue before bump deep_ep to 29d31c095796f3c8ece47ee9cdcc167051bbeed9 in #2224, so it's good to have a test on this.

]
vllm = [
"cuda-python",
"deep_gemm @ git+https://github.com/deepseek-ai/DeepGEMM.git@67fc64863d43521080bf2005e6528d0fceee9510",
# deep_ep also needs libibverbs-dev
# sudo apt-get update
# sudo apt-get install libibverbs-dev
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@29d31c095796f3c8ece47ee9cdcc167051bbeed9 ; platform_machine == 'x86_64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@a48493600c4886c1b297aaa78db0e1ebc2d8dd6c ; platform_machine == 'aarch64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@17cfb817bccec3a9c247013360cc550c2bac441e",
# Default wheels on GitHub are cu130. See v0.25.1 release assets:
# https://github.com/vllm-project/vllm/releases/tag/v0.25.1
"vllm @ https://github.com/vllm-project/vllm/releases/download/v0.25.1/vllm-0.25.1-cp38-abi3-manylinux_2_28_aarch64.whl ; sys_platform == 'linux' and platform_machine == 'aarch64'",
Expand Down Expand Up @@ -193,8 +191,7 @@ mcore = [
# silently falls back to the PyPI releases instead of the pinned git revs.
"mamba-ssm",
"causal-conv1d",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@29d31c095796f3c8ece47ee9cdcc167051bbeed9 ; platform_machine == 'x86_64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@a48493600c4886c1b297aaa78db0e1ebc2d8dd6c ; platform_machine == 'aarch64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@17cfb817bccec3a9c247013360cc550c2bac441e",
"flashinfer-python==0.6.8.post1",
"flashinfer-cubin==0.6.8.post1",
"flashinfer-jit-cache==0.6.8.post1",
Expand Down Expand Up @@ -367,8 +364,7 @@ override-dependencies = [
"llguidance>=1.3.0,<1.4.0",
# Override setuptools range in other dependencies to address CVE GHSA-58pv-8j8x-9vj2
"setuptools>=80.10.2",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@29d31c095796f3c8ece47ee9cdcc167051bbeed9 ; platform_machine == 'x86_64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@a48493600c4886c1b297aaa78db0e1ebc2d8dd6c ; platform_machine == 'aarch64'",
"deep_ep @ git+https://github.com/deepseek-ai/DeepEP.git@17cfb817bccec3a9c247013360cc550c2bac441e",

@yuki-97 yuki-97 Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On GB200, two things would help close the "Validation boundary" gap:

  1. Could you link the B200 validation from perf: enable HybridEP in H100/B200 performance recipes #3438 (job 507030, 59/59 on head d68c6fa8) in the PR body? Since that PR depends on this bump, that job is effectively the runtime evidence for 17cfb817 on B200.

  2. Is the local DMA-BUF fix still required on top of 17cfb817 (i.e. was job 507030 also patched)? If so, could you upload it so the shipped pin actually matches what B200 needs at runtime — otherwise people may meet issues.

@seonjinn seonjinn Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — I re-checked the provenance.

For H100, multi-node runtime validation used the exact, unpatched DeepEP pin 17cfb817 and a wheel built with HYBRID_EP_MULTINODE=1 before the actor environment was created. The NeMo-RL validation stack also included #2964 and the matching recipe enablement from #3438, so this is runtime compatibility evidence for the pin rather than an isolation test of this dependency-only diff.

Model H100 80 GB topology HybridEP result E2E TPS/GPU vs matched AllToAll W&B
Qwen3-30B-A3B 4 nodes × 8 GPUs 20/20 steps +0.60% run
Qwen3-235B-A22B 16 nodes × 8 GPUs 20/20 steps +4.34% run
Nemotron 3 Super 120B-A12B 32 nodes × 8 GPUs 20/20 steps +8.74% run

W&B history was re-checked and contains train steps 1–20 for all three H100 runs. The multi-node flag is a build-time requirement at this pin; #3436 selects the source revision but does not itself rebuild an existing single-node DeepEP binary.

Job 507030 was a B200-hosted configuration test, not a DeepEP runtime test. It ran tests/unit/tools/test_hybridep_default_8g_recipes.py and passed 59/59; it did not build, import, or execute DeepEP. It was therefore not patched, and I do not think it should be presented as runtime evidence for 17cfb817. It is valid resolved-recipe coverage for #3438.

Separately, a 4-node × 8-GPU B200 Qwen3-30B-A3B run used an exact, unpatched wheel built from 17cfb817 and completed 20/20 steps with exit code 0. That establishes that the DMA-BUF fallback is not required for this validated B200 path.

The local DMA-BUF fallback was an experimental workaround for ibv_reg_mr errno=14 observed in a larger Super/EP32 configuration. The patched Super attempts did not establish a completed result because they later failed in other runtime paths, so there is not enough evidence to vendor/upload that workaround or make this dependency bump rely on it. The validation boundary should distinguish the 59/59 recipe test, the exact-pin Qwen runtime pass, and the still-unresolved large-EP corner case.

# Relax megatron-core workspace member's opentelemetry-api ceiling (<1.34) for protobuf 6.x compat with ray
"opentelemetry-api>=1.33.1",
# Override sglang's xgrammar==0.1.32 to address CVE GHSA-7rgv-gqhr-fxg3.
Expand Down Expand Up @@ -552,7 +548,7 @@ requires-dist = ["torch", "packaging", "ninja", "causal-conv1d"]
[[tool.uv.dependency-metadata]]
name = "deep_ep"
# This version has to match the version in the commit/rev/tag used
version = "v1.2.1+29d31c0"
version = "1.2.1+17cfb81"
requires-dist = ["torch", "packaging", "ninja"]

[[tool.uv.dependency-metadata]]
Expand Down
Loading
Loading