[Build] Fix CUDA arch detection producing kernel-less builds on SM121

[ayush1399]

Purpose

On GB10 / DGX Spark (compute capability 12.1, CUDA 13), this shows up on the default source-build path from the docs, uv pip install -e . with TORCH_CUDA_ARCH_LIST unset. No special flags or custom arch list needed. The build looks fine but has no CUTLASS kernels in it. The .so only contains sm_75 cubins. Serving any FP8 model then fails at startup:

RuntimeError: cutlass_scaled_mm, csrc/.../scaled_mm_entry.cu:265,
NotImplementedError: No compiled cutlass_scaled_mm for a compute capability
less than CUDA device capability: 121

The cause is an old line in torch's vendored select_compute_arch.cmake:

string(REPLACE "2.1" "2.1(2.0)" compute_capabilities "${compute_capabilities}")

It was written for Fermi . The string 12.1 contains 2.1, so torch rewrites it to 12.1(2.0) and emits -gencode arch=compute_20,code=sm_121.

vLLM's extract_unique_cuda_archs_ascending() reads the arch=compute_* part of that flag, so it sees arch 2.0. Nothing in CUDA_SUPPORTED_ARCHS matches 2.0, so every arch-gated kernel is skipped. The rest of the sources build at nvcc's default sm_75, and the build reports success.

There is an upstream fix in progress (pytorch/pytorch#185993), but every torch release up to 2.11 has the bug so there should be a fix in vLLM as well.

Changes

• cmake/utils.cmake: read the code=sm_* part of each gencode flag instead of arch=compute_*, since that is the real SASS target. Fall back to arch=compute_* for PTX-only entries. This gets 12.1 back, and cuda_archs_loose_intersection() maps it to the 12.0f family target.
• CMakeLists.txt: if no supported arch matches, stop with an error instead of building a binary with no kernels.

Not a duplicate

• [Build] Fix CUDA arch coverage checks and scoped kernel feature flags #47149 scopes kernel feature macros per source file and warns when a prebuilt wheel lacks the device's arch. It does not touch arch autodetection, and it does not catch the empty-arch case at configure time.
• [Bugfix] Add 12.1 to CUDA_SUPPORTED_ARCHS for CUDA-13 builds (sm_121a/GB10) #43003 (closed) added 12.1 to the supported arch list. Review there showed the 12.0f family fallback already handles SM121. The bug here happens before that logic runs.
• [Build] Add SM121 (DGX Spark / GB10) to published build targets #38484 changes which archs the published wheels target, not source builds.

AI assistance disclosure

AI assistance was used for root-cause analysis, and drafting this description. The code changes were made and tested by me, reviewed with AI.

Test Plan

.venv/bin/python -m pytest tests/test_cmake_utils.py -v

On GB10: rebuild with TORCH_CUDA_ARCH_LIST unset, check the compiled archs with cuobjdump --list-elf, then serve the FP8 model from the issue.

Test Result

GB10 (aarch64, CUDA 13.0.88, torch 2.11.0+cu130), uv pip install -e ., no TORCH_CUDA_ARCH_LIST set.

Before:

-- Autodetected CUDA architecture(s):  12.1(2.0)
-- Added CUDA NVCC flags for: -gencode;arch=compute_20,code=sm_121
-- CUDA target architectures: 2.0
-- CUDA supported target architectures:
-- Not building scaled_mm_c3x_120 as no compatible archs found ...

_C_stable_libtorch.abi3.so has only sm_75 cubins. vllm serve RedHatAI/gemma-4-31B-it-FP8-block crashes in profile_run.

After:

-- CUDA target architectures: 12.1
-- CUDA supported target architectures: 12.0
-- Building scaled_mm_c3x_sm120 for archs: 12.0f

The rebuild works and the same model serves fine (Application startup complete).

I am attaching the full vllm serve startup logs for both cases, the crash before the fix and the clean startup after it.

FP8_SM121_bug_startup_logs.md
FP8_SM121_bugfix_startup_logs.md

tests/test_cmake_utils.py::test_exact_family_arch_precedes_generic_family_fallback PASSED
tests/test_cmake_utils.py::test_extract_archs_prefers_sass_target_over_corrupted_virtual_arch PASSED

2 passed in 0.26s

This only changes the build system and touches no numerics, so I did not run evals.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. No documentation update is required for this build-only fix.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Harry-Chen]

Good catch! This reminds me of the story of why Windows 9 was not released 🤦

[Harry-Chen]

@ayush1399 Please finish the DCO requirement before we can get this merged.

[ayush1399]

@Harry-Chen updated my commit with a signed-off by for the DCO requirement, lmk if there's anything else I might have missed