[WIP][DSv4.1] Opt in to mHC coefficient overlap on SM90

[xijiaat]

Overview

Draft: add an experimental, explicit opt-in for DeepSeek-V4.1 mHC coefficient overlap on Hopper (SM90), reusing the stream path introduced by #57603. Hopper remains disabled by default; GPU validation is pending.

Claims

• KernelConfig.enable_mhc_overlap=None preserves SM100 automatic selection; True additionally permits SM90, and False disables overlap.
• Existing DeepGEMM, shape, ubatching, FULL CUDA graph and small-batch restrictions remain. SM90 uses the ordinary TP collective and cannot select the SM100-only fused all-reduce.
• No speedup, numerical equivalence, model quality or production-readiness claim is made in this draft.

Validation

CPU only, using this branch's Python source on Linux, Python 3.12.3 and pytest 9.1.1, in an isolated container with no GPU devices (2 CPU cores / 4 GiB). Installed dependencies were inherited from a vLLM v0.28.0 image; this was not a fresh main wheel build.

CUDA_VISIBLE_DEVICES= NVIDIA_VISIBLE_DEVICES=void VLLM_TARGET_DEVICE=cpu \
  PYTHONPATH=$PWD PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
  .venv/bin/python -m pytest --confcutdir=tests/kernels/mhc -o addopts= \
  -q tests/kernels/mhc/test_mhc_dispatch.py

14 passed, covering opt-in/default/off selection, unsupported architectures, missing DeepGEMM, shape/ubatching fallback and exclusion of the SM100 collective. These mock capability checks validate dispatch only. Source-only version metadata and torch deprecation warnings were emitted.

.venv/bin/pre-commit run --files vllm/config/kernel.py \
  vllm/models/deepseek_v41/nvidia/ops/mhc.py \
  tests/kernels/mhc/test_mhc_dispatch.py tests/kernels/mhc/test_mhc_kernels.py
git diff --check

All applicable hooks pass, including mypy, config validation and Buildkite test tethering. The new dispatch tests are in the existing mHC suite. The replay test now permits SM90 but has not been run on GPU; the existing dedicated NVIDIA mHC CI job uses B200.

Before marking ready:

• Verify that the tested DeepGEMM dependency contains the SM90 register-lifetime repair tracked by [BugFix][sm90] tf32 gemm race condition deepseek-ai/DeepGEMM#448 / [Bugfix] Package the SM90 DeepGEMM mHC register lifetime fix #59129; this draft does not implement that fix.
• Run the mHC replay test and independent numerical references on Hopper, including stream ordering and repeated graph replay.
• Verify eager/piecewise/large-batch fallback, TP modes and an SM100 regression run.
• Run DeepSeek-V4.1 end-to-end model evaluation and matched baseline/opt-in serving measurements, including VLLM_GPU_SYNC_CHECK=error.
• Reassess the existing 16-token threshold on Hopper and complete human line-by-line review. Human review is not asserted.

Details

The existing eligibility gate excludes SM90. This change makes Hopper experimental and opt-in via --kernel-config '{"enable_mhc_overlap": true}'; omitting it preserves defaults. This is a proposed serving configuration, not a GPU-validated launch recipe. The existing configuration hash includes the new field.

This reuses vLLM's TF32 prenorm coefficient path. It does not port the compensated-BF16 projection from sgl-project/sglang#41251. Kernel arithmetic, the existing stream schedule and the GB200-derived 16-token threshold are unchanged. The all-reduce architecture guard is needed because its caller can now also create an mHC stream on SM90.

Duplicate-work checks on 2026-10-09 covered issue #57448 and open/closed PRs matching mHC, overlap, SM90 and Hopper. No equivalent Hopper coefficient-overlap PR was found: #57603 targets SM100; #59802 targets ROCm; #59129 packages the separate DeepGEMM correctness repair. This draft depends on confirming that repair rather than duplicating it.

AI assistance (OpenAI Codex) was used for investigation, implementation, tests and this description. This is WIP awaiting GPU resources; model evaluations and performance results are not yet available.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.