[MatthewBonanni]

Purpose

Fixes #64327. Unblocks #63801.

FlashInfer's MNNVL fused all-reduce + residual + RMSNorm has a cluster-barrier race (flashinfer-ai/flashinfer#7782). The vLLM default single-node backend has been MNNVL since #61989. With HiSparse running host-memory gathers on its copy stream, the race fires and the engine dies with CUDA error: unspecified launch failure (Xid 13 CTA Not Present).

With HiSparse enabled and VLLM_FLASHINFER_ALLREDUCE_BACKEND=auto, backend selection now:

• single node: uses trtllm;
• multi-node: disables FlashInfer all-reduce, since trtllm doesn't support multi-node groups.

An explicit VLLM_FLASHINFER_ALLREDUCE_BACKEND setting is still respected. This is temporary: the check should be removed once vLLM picks up a FlashInfer release with #7782.

Test Plan

pytest tests/distributed/test_comm_ops.py -k "flashinfer_auto_backend or workspace_failure or standalone_workspace"

Test Result

Server check that the new path runs: 4× H100 (driver 580.126.09), main + #63801 (Python side; kernels from the main precompiled wheel) + this PR, GLM-5.3 config with dummy weights cut to 5 layers, HiSparse with spec decode. Same build, same load; the only difference is the backend env var.

VLLM_USE_V2_MODEL_RUNNER=1 VLLM_FLASHINFER_ALLREDUCE_BACKEND={mnnvl,auto} \
vllm serve zai-org/GLM-5.3 -tp 4 --enforce-eager --load-format dummy \
  --hf-overrides '{"num_hidden_layers": 5, "first_k_dense_replace": 5, "indexer_types": ["full", "full", "full", "shared", "shared"]}' \
  --kv-cache-dtype fp8_ds_mla --kv-cache-memory-bytes 4G \
  --max-model-len 67584 --max-num-batched-tokens 8192 --max-num-seqs 32 --no-enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --kv-transfer-config '{"kv_connector":"HiSparseConnector","kv_role":"kv_both","kv_connector_extra_config":{"host_pool_gib":16}}'

# 6 waves, seeds 1..6
vllm bench serve --dataset-name random --random-input-len 52000 --random-output-len 1024 \
  --random-range-ratio 0.25 --num-prompts 32 --max-concurrency 32 --ignore-eos --seed $seed

VLLM_FLASHINFER_ALLREDUCE_BACKEND	Fused AR workspace	Result	Output tok/s (mean of 6 waves)
mnnvl (stock behavior)	backend=mnnvl	6/6 waves, 0 failed requests	499
auto (this PR)	backend=trtllm, logs Using flashinfer allreduce backend trtllm: ...	6/6 waves, 0 failed requests	552

HiSparse was reading heavily from host memory (8.9 TB host-to-device, ~50% hot-buffer miss rate), but this reduced server didn't hit the race on H100 even with mnnvl. The crash evidence is the B200 result below and the standalone repro in flashinfer-ai/flashinfer#7782, which faults within seconds on this same box.

10 passed. The two new HiSparse cases fail without the change.

Server-level evidence that trtllm avoids the crash (from #64327 / flashinfer#7782, set by hand via VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm): GLM-5.3-NVFP4 TP8 on 8× B200 with HiSparse went from a crash on wave 1 in 7/7 runs (MNNVL) to clean for 4/4 waves (TRTLLM), at about 3% lower decode throughput.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[MatthewBonanni]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99298 for commit c28205c626eb.

ℹ️ This PR is 12 commits behind upstream main. Merged upstream main into this branch as c28205c626eb before starting CI. Pull the branch before pushing more changes.