[mgoin]

Overview

Part of #65560.

Deprecate VLLM_MARLIN_USE_ATOMIC_ADD (now a no-op) and stop building the bf16 Marlin kernels for 9.0+PTX, which only existed for the native bf16x2 atomicAdd used by that opt-in path. Independent of #65540 and #65542.

Claims

• The Marlin dense atomic-add reduction gives no end-to-end benefit, so it is removed from the Python paths. Setting VLLM_MARLIN_USE_ATOMIC_ADD now logs a one-time deprecation warning (removal in v0.35.0) instead of changing kernels.
• bf16 Marlin kernels are built with the same arch list as fp16. Multi-arch builds lose the sm_90 SASS and compute_90 PTX for every bf16 Marlin kernel.
• Removes the startup log that told users to set VLLM_MARLIN_USE_ATOMIC_ADD=1, and drops the setting from the MiniMax-M3 H200 recipe. Marlin MoE always passes use_atomic_add=False, so it was a no-op there.

Validation

GB10 (SM121), CUDA 13.0.

Kernel microbenchmark (GPTQ INT4, group 128, CUDA-graph timing, ops.marlin_gemm with use_atomic_add on vs off) on the only shapes where the path could fire (n < 2048, k ≥ 2048):

k × n	m=1	m=16	m=64
8192 × 1280 (bf16)	8.8 → 8.1 µs	11.4 → 10.6	25.8 → 25.5
5120 × 1024 (bf16)	6.4 → 5.7	8.0 → 7.1	17.0 → 16.3
4096 × 768 (bf16)	5.9 → 3.6	7.1 → 4.7	13.6 → 10.9
14336 × 1024 (bf16)	10.8 → 10.1	14.3 → 13.4	33.3 → 32.9

fp16 is the same within 0.1–0.3 µs. Atomic adds accumulate in the output dtype, with max relative error vs the fp32 reduce of about 5e-3 for bf16 and 7e-4 for fp16, and are run-to-run nondeterministic.

End to end, batch-1 decode on Qwen/Qwen2.5-1.5B-Instruct-GPTQ-Int4, whose down_proj (k=8960, n=1536) qualifies (vllm bench latency --input-len 128 --output-len 512, 2 runs each):

	avg latency
global reduce (default)	3.21 s / 3.46 s
VLLM_MARLIN_USE_ATOMIC_ADD=1	3.49 s / 3.47 s
--linear-backend humming	3.50 s / 3.51 s

No measurable difference.

Tests: test_marlin_gemm.py -k test_marlin_gemm 522 passed, test_marlin_tile_padding.py 98 passed, test_moe.py -k marlin 305 passed, plus the new test_deprecated_marlin_atomic_add_env_warns. GSM8K (100 q) on Qwen/Qwen2.5-3B-Instruct-GPTQ-Int4 with the env set: 0.61, with the warning logged once:

WARNING [arg_utils.py] VLLM_MARLIN_USE_ATOMIC_ADD is deprecated, has no effect, and will be removed in v0.35.0.

A configure with TORCH_CUDA_ARCH_LIST="8.0;9.0;12.0" gives the bf16 and fp16 Marlin kernels the same targets: sm_80, compute_80, sm_120f.

Details

• Usage: the env var appears in community DGX Spark / RTX Pro 6000 recipes, most likely because of the removed startup log. Warning instead of raising keeps those configs starting. One DGX Spark writeup credits it with fixing incorrect output on SM121. On current main the default (non-atomic) path passes the Marlin kernel tests and GSM8K on SM121, so I couldn't reproduce that.
• On Hopper, bf16 Marlin now JITs from compute_80 PTX instead of shipping sm_90 SASS. That only matters where Marlin is still picked on SM90; Humming and Machete outrank it there.
• The use_atomic_add argument on the marlin_gemm op and the kernel's atomic branch are left as-is (always False from Python), to avoid conflicting with [Kernel] Marlin: remove 8-bit (INT8/FP8) activation support, rely on Humming #65542, which rewrites those op signatures. A follow-up removes them after [Kernel] Marlin: remove 8-bit (INT8/FP8) activation support, rely on Humming #65542 lands.

Not a duplicate: no open PR removes this path. #48569 (extend Marlin atomic-add to fused MoE) goes the other way and would be superseded. #38669 touches the Marlin repack PTX arch list on H100 but not the bf16 GEMM targets.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Recipe preview: https://vllm-recipes-git-preview-vllm-pr-65558-inferact-inc.vercel.app

[yewentao256]

LGTM, thanks for the work!

[yewentao256]

Suggested change

	def test_deprecated_marlin_atomic_add_env_warns(monkeypatch, caplog_vllm):
	monkeypatch.setenv("VLLM_MARLIN_USE_ATOMIC_ADD", "1")
	with caplog_vllm.at_level(logging.WARNING, logger="vllm"):
	EngineArgs().create_engine_config()
	assert any(
	"VLLM_MARLIN_USE_ATOMIC_ADD is deprecated" in m for m in caplog_vllm.messages
	)

I don't think we need a specific unit test for this update

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mgoin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork