[Bugfix][Distributed] Keep cross-node MNNVL groups off CUDA IPC all-reduce

[SiluPanda]

Purpose

Addresses #52778.

Reporter-provided multi-node GPU validation is available below; same-node GPU controls and model-output evaluation remain pending. Local validation uses CPU tests only; this PR does not claim submitter-run GPU tests.

Cross-node MNNVL-capable groups remain enabled for custom all-gather/reduce-scatter, but cannot exchange node-local CUDA IPC handles. should_custom_ar() currently admits TP=2 cross-node groups; CUDA-graph teardown can subsequently exchange their IPC handles and terminate the process inside cudaIpcOpenMemHandle.

This patch:

• declines node-local custom all-reduce when mnnvl_only=True, allowing the existing caller dispatch to try another backend;
• skips IPC graph registration for those groups, whose MNNVL collectives use pre-registered symmetric memory;
• preserves same-node custom all-reduce and the existing MNNVL all-gather/reduce-scatter eligibility;
• adds regression coverage in the existing custom-all-reduce test suite.

No CUDA kernels or arithmetic implementations are changed.

Why this is not duplicate work

Before starting, #52778 had no assignee or visible work claim. Issue/timeline checks and open-PR searches for the issue number, cross-node IPC, MNNVL custom allreduce, mnnvl_only, and register_graph_buffers found no open implementation of this specific fix. The issue-number and mnnvl_only searches were repeated before opening this PR.

Related changes address different mechanisms:

• Merged [Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability #53253 prevents unsupported cross-node groups from reaching MNNVL rendezvous; its description explicitly distinguishes CustomAllreduce admits cross-node TP=2 groups and dies in cudaIpcOpenMemHandle (custom_all_reduce.cuh:143 'invalid resource handle') on MNNVL GB200 NVL72 #52778. Supported Blackwell groups still reach the paths changed here.
• Open [Bugfix][Distributed] Gate MNNVL allreduce workspace creation on fabric support #53377 concerns FlashInfer fusion-workspace capability checks.
• Open [Bugfix] Fix IPC handle leak in CustomAllreduce during CUDA graph memory profiling #46515 concerns IPC-handle lifetime rather than cross-node eligibility.
• Open [Core][Distributed] Size custom all-reduce buffers at init so batch-invariant mode can use them #50505 changes buffer sizing for batch invariance rather than node locality.

Test Plan

.venv/bin/python -m pytest tests/distributed/test_custom_all_reduce.py -v
.venv/bin/python -m pytest \
  tests/distributed/test_custom_all_gather_reduce_scatter.py \
  tests/distributed/test_comm_ops.py -k 'not multi_process' -v
.venv/bin/pre-commit run --files \
  vllm/distributed/device_communicators/custom_all_reduce.py \
  tests/distributed/test_custom_all_reduce.py
git diff --check

Test Result

Executed locally on Apple Silicon macOS with Python 3.12.11 and PyTorch 2.13.0:

Check	Result
New regression tests with unpatched production code	3 failed, 13 passed, 4 GPU tests skipped
Patched custom-all-reduce suite	16 passed, 4 GPU tests skipped
Adjacent communication suites	19 passed, 1 GPU test skipped, 10 distributed tests deselected
Applicable pre-commit hooks, including Ruff and mypy	Passed
Diff whitespace check	Passed

The three baseline failures cover eager/captured cross-node all-reduce dispatch and cross-node IPC graph registration. Positive controls cover same-node registration and MNNVL collective eligibility.

These tests exercise real production Python methods and CPU tensors, with CUDA operations and process-group calls mocked at the boundaries. They do not execute CUDA IPC, NCCL, or MNNVL kernels. The precompiled editable install was unavailable on this host; the Python tests ran through normal source imports after installing dependencies with uv.

Outstanding validation

• Human-run relevant tests and final line-by-line review under the contribution policy.
• Reporter-validated matched unpatched/patched startup, CUDA-graph capture, and serving benchmarks on two MNNVL-capable GB200 NVL72 nodes with TP=2.
• Same-node TP=2 control and GPU all-gather/reduce-scatter regression run.
• Model-output/evaluation results for the serving-backend change.

External GPU validation: the issue reporter confirmed the fix on September 8, 2026. See the full validation report. On two GB200 NVL72 nodes with one GPU per node, cross-node TP=2, and custom all-reduce left enabled, the unpatched control aborts with custom_all_reduce.cuh:143. Applying this patch as the only change allows CUDA-graph capture, server startup, and serving benchmarks at concurrency 1/16/64/128 to complete. The reporter also confirmed that MNNVL setup remains active.

The submitter still has no GPU environment. Same-node GPU controls, GPU all-gather/reduce-scatter regression runs, model-output accuracy evaluation, and the remaining human-review requirements have not been reported as complete.

AI assistance

OpenAI Codex assisted with issue triage, investigation, patch preparation, and execution of the local CPU tests and lint checks. The patch was approved for publication by the submitter. The outstanding checklist is intentionally incomplete. External GPU results above are attributed to the issue reporter; they do not attest to submitter-run GPU tests, model-output accuracy evaluation, or completion of the remaining human-review requirements. The commit includes AI attribution and sign-off.

Main synchronization — September 23, 2026

Merged upstream main (0f2a15c9) without rewriting the original validated commit. Resolved the test-file insertion conflict by retaining both the PR's CPU fixture/regressions and upstream's new SM100 fused-mHC GPU regression.

• Production file is byte-for-byte unchanged from the prior PR head; all existing PR test/helper functions and both new upstream mHC functions are AST-identical to their respective originals.
• Custom-all-reduce suite: 16 passed, 5 GPU tests skipped.
• Adjacent communication suites with -k 'not multi_process': 20 passed, 1 GPU test skipped, 10 distributed tests deselected.
• Changed-file pre-commit checks (including Ruff, mypy and test tethering), scoped whitespace checks, and independent resolution review passed.
• No GPU/model evaluation was rerun for this synchronization. Prior hardware results remain attributed to the reporter's original patch validation.

CI status (original PR head)

The failing pre-run-check is the repository's CI authorization gate, not a lint/test result. It requires one of verified, ready, or ready-run-all-tests, or at least four merged PRs from the author; the log reports one merged PR and no qualifying label. The actual pre-commit job was therefore skipped. GitHub's ready-for-review status is distinct from those authorization labels. Local changed-file pre-commit checks passed as reported above; upstream CI remains unexecuted behind that gate. No CI policy or authorization controls were modified.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[janbernloehr]

TLDR; We have validated that this patch resolves the abort.

Setup

• 2 nodes, GB200 NVL72, MNNVL fabric, 1 GPU used per node, driver 580.178.04, compute capability 10.0
• vllm/vllm-openai:nightly as the base, with vLLM pinned to 311b3513af33bc29b4acb2fde2e9313e5e9966a0 (reports itself as 0.27.2rc1.dev150+g311b3513a)
• Model nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8, served with --quantization modelopt, --tensor-parallel-size 2, max model length 4096, FP8 KV cache
• Ray-based multi-node serving, one rank per node, so the TP=2 group spans two nodes
• Custom all-reduce left at its default: disable_custom_all_reduce=False. Nothing was disabled to make the run pass.

vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 \
  --quantization modelopt \
  --tensor-parallel-size 2 \
  --max-model-len 4096 \
  --kv-cache-dtype fp8_e4m3

The essential ingredient is that the two TP ranks land on different MNNVL-capable nodes. A same-node TP=2 run does not reproduce it.

Unpatched behaviour

The server dies during startup inside the Ray workers. It gets through torch.compile, the warmup run, and

Profiling CUDA graph memory: PIECEWISE=83 (largest=1024), FULL=83 (largest=1024)

and then both ranks die with no capture progress and no Graph capturing finished:

Failed: Cuda error /workspace/csrc/custom_all_reduce.cuh:143 'invalid resource handle'

This matches your reading of the mechanism: should_custom_ar() admits the cross-node TP=2 group via the world_size == 2 shortcut, and the CUDA-graph path then exchanges CUDA IPC handles for peers that are not on this node.

With your patch

Same commit, same base image, patch applied as the only difference:

Capturing CUDA graphs (decode, FULL): 100%|██████████| 83/83
Graph capturing finished in 46 secs, took 2.10 GiB
Application startup complete.

Startup completes, CUDA-graph capture completes, and a four-point concurrency sweep (1 / 16 / 64 / 128) runs to completion and reports metrics. Zero occurrences of custom_all_reduce.cuh:143 anywhere in the log, versus two in the unpatched control (one per rank).

Controls

We pinned the exact aborting commit and ran candidate patches as sibling container layers over it, so every arm shares one base image and one vLLM wheel and the applied source patch is the only difference.

Two further data points on the same baseline:

1. Gating only the world_size == 2 shortcut on not self.mnnvl_only is also sufficient to stop the abort. Your patch is a strict superset of that gate — it rejects mnnvl_only groups before that shortcut is reached, covers groups admitted via fully_connected, and independently guards the registration call. Both arms pass, and their throughput lands within about 4% of each other in both directions across all four concurrency levels, so there is no measurable cost to the broader gate.
2. Merged PR [Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability #53253 applied over the same aborting commit does not fix this — the workload aborts identically to the control. That is consistent with your "Why this is not duplicate work" section, and worth keeping in mind if anyone tries to close CustomAllreduce admits cross-node TP=2 groups and dies in cudaIpcOpenMemHandle (custom_all_reduce.cuh:143 'invalid resource handle') on MNNVL GB200 NVL72 #52778 as already handled.

Worth stressing, because it speaks to the shape of the fix:

Initialized FlashInfer Allreduce norm fusion workspace with backend=mnnvl

appears on both workers in both arms — unpatched and patched alike, and in the unpatched arm it appears before the abort. Your patch leaves MNNVL allreduce setup completely untouched; it only stops the CUDA-IPC custom all-reduce from claiming a group it cannot serve. This is a correctness restoration, not a disablement.

[SiluPanda]

@tlrmchlsmth @jeejeelee, could you review this cross-node MNNVL/CUDA-IPC fix? The reporter validated it on two GB200 NVL72 nodes: the unpatched control aborts, while the patch completes CUDA-graph capture, startup, and all four serving-concurrency runs with MNNVL enabled. The PR description now includes these results and the remaining validation gaps.

[SiluPanda]

TLDR; We have validated that this patch resolves the abort.

Setup

• 2 nodes, GB200 NVL72, MNNVL fabric, 1 GPU used per node, driver 580.178.04, compute capability 10.0

• vllm/vllm-openai:nightly as the base, with vLLM pinned to 311b3513af33bc29b4acb2fde2e9313e5e9966a0 (reports itself as 0.27.2rc1.dev150+g311b3513a)

• Model nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8, served with --quantization modelopt, --tensor-parallel-size 2, max model length 4096, FP8 KV cache

• Ray-based multi-node serving, one rank per node, so the TP=2 group spans two nodes

• Custom all-reduce left at its default: disable_custom_all_reduce=False. Nothing was disabled to make the run pass.

vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 \

  --quantization modelopt \

  --tensor-parallel-size 2 \

  --max-model-len 4096 \

  --kv-cache-dtype fp8_e4m3

The essential ingredient is that the two TP ranks land on different MNNVL-capable nodes. A same-node TP=2 run does not reproduce it.

Unpatched behaviour

The server dies during startup inside the Ray workers. It gets through torch.compile, the warmup run, and

Profiling CUDA graph memory: PIECEWISE=83 (largest=1024), FULL=83 (largest=1024)

and then both ranks die with no capture progress and no Graph capturing finished:

Failed: Cuda error /workspace/csrc/custom_all_reduce.cuh:143 'invalid resource handle'

This matches your reading of the mechanism: should_custom_ar() admits the cross-node TP=2 group via the world_size == 2 shortcut, and the CUDA-graph path then exchanges CUDA IPC handles for peers that are not on this node.

With your patch

Same commit, same base image, patch applied as the only difference:

Capturing CUDA graphs (decode, FULL): 100%|██████████| 83/83

Graph capturing finished in 46 secs, took 2.10 GiB

Application startup complete.

Startup completes, CUDA-graph capture completes, and a four-point concurrency sweep (1 / 16 / 64 / 128) runs to completion and reports metrics. Zero occurrences of custom_all_reduce.cuh:143 anywhere in the log, versus two in the unpatched control (one per rank).

Controls

We pinned the exact aborting commit and ran candidate patches as sibling container layers over it, so every arm shares one base image and one vLLM wheel and the applied source patch is the only difference.

Two further data points on the same baseline:

1. Gating only the world_size == 2 shortcut on not self.mnnvl_only is also sufficient to stop the abort. Your patch is a strict superset of that gate — it rejects mnnvl_only groups before that shortcut is reached, covers groups admitted via fully_connected, and independently guards the registration call. Both arms pass, and their throughput lands within about 4% of each other in both directions across all four concurrency levels, so there is no measurable cost to the broader gate.

2. Merged PR [Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability #53253 applied over the same aborting commit does not fix this — the workload aborts identically to the control. That is consistent with your "Why this is not duplicate work" section, and worth keeping in mind if anyone tries to close CustomAllreduce admits cross-node TP=2 groups and dies in cudaIpcOpenMemHandle (custom_all_reduce.cuh:143 'invalid resource handle') on MNNVL GB200 NVL72 #52778 as already handled.

Worth stressing, because it speaks to the shape of the fix:

Initialized FlashInfer Allreduce norm fusion workspace with backend=mnnvl

appears on both workers in both arms — unpatched and patched alike, and in the unpatched arm it appears before the abort. Your patch leaves MNNVL allreduce setup completely untouched; it only stops the CUDA-IPC custom all-reduce from claiming a group it cannot serve. This is a correctness restoration, not a disablement.

Thank you so much @janbernloehr for your help in validating the change! Appreciate your time and work.

[SiluPanda]

@tlrmchlsmth @jeejeelee, gentle follow-up: the reporter has confirmed the fix with matched failing-unpatched/passing-patched GB200 runs; could you review this PR when you have a chance? Thank you!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @SiluPanda.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[SiluPanda]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @SiluPanda.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Pr is rebased and updated now, no more merge conflicts

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @SiluPanda.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[SiluPanda]

@tlrmchlsmth @jeejeelee, gentle follow-up: the latest merge conflict is resolved and 36 CPU tests pass; with the reporter's GB200 validation above, could you review or advise if anything else is needed? Thank you!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @SiluPanda.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[SiluPanda]

@tlrmchlsmth @jeejeelee, could you review the reporter-validated GB200 fix when convenient? The branch still needs rebasing and its CI requirements cleared before merge. Thank you!