[Build] Fix CUDA arch coverage checks and scoped kernel feature flags

[Harry-Chen]

Purpose

This PR fixes several CUDA architecture consistency problems found after the
CUDA arch cleanup in #45277 and the Thor workaround discussion in #46339.

The main goal is to make vLLM's CUDA architecture handling more honest at each
layer:

• CMake should only apply optional kernel feature macros to sources that use
  those kernels.
• Runtime support checks should match the architectures that were actually
  compiled into the wheel.
• Users should get an explicit warning when a pre-built wheel does not cover
  one of their visible CUDA devices, instead of discovering it later through a
  less actionable no kernel image is available for execution on the device
  error.
• The documentation should explain that release wheels intentionally use a
  smaller architecture list than the full source build support matrix.

Build and CMake changes

Scope optional CUDA feature macros to the sources that need them

This PR adds set_compile_definitions_for_srcs() in cmake/utils.cmake and
uses it to keep optional kernel feature macros source-local instead of placing
them in broad target or global compile flags.

The scoped definitions include:

• VLLM_ENABLE_COOPERATIVE_TOPK
• ENABLE_SCALED_MM_SM90
• ENABLE_SCALED_MM_SM100
• ENABLE_CUTLASS_MOE_SM90
• ENABLE_CUTLASS_MOE_SM10X_OR_SM11X
• ENABLE_NVFP4

This avoids changing the compile command for unrelated CUDA sources when only a
component-specific feature gate changes. In practice, it should reduce needless
rebuilds and avoid invalidating more compiler cache entries than necessary.

The cooperative top-k PR in #46339 is still useful, but this PR tightens the
implementation so that the feature macro is attached to the cooperative top-k
sources only, instead of being distributed through VLLM_GPU_FLAGS.

Fix Thor / SM11x CUTLASS grouped MoE coverage

The CUTLASS grouped MoE path had an SM10x-only naming and support gate even
though the build-side architecture filter can include the Thor family on newer
CUDA toolchains. This PR:

• renames the grouped MoE macro from ENABLE_CUTLASS_MOE_SM100 to
  ENABLE_CUTLASS_MOE_SM10X_OR_SM11X;
• updates the C++ support/dispatch check in
  csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu to cover
  version_num >= 100 && version_num < 120;
• updates the Python guard in vllm/_custom_ops.py so
  cutlass_group_gemm_supported() can query SM101/SM110 as supported when the
  compiled C++ path is present;
• keeps SM12x excluded from this grouped MoE path unless a real SM12x dispatch
  is added later.

This is intended to let CUDA 12.9 and CUDA 13 builds use the grouped MoE kernel
on Thor-family devices without claiming support for the SM12x path.

Remove stale CUTLASS MLA macro plumbing

ENABLE_CUTLASS_MLA no longer had an in-tree consumer. The relevant git
history is:

• the macro was introduced with the old CUTLASS MLA path in [NVIDIA] Support Cutlass MLA for Blackwell GPUs #16032;
• the old consumer was removed by Remove old cutlass mla #23961;
• the CMake definition remained after the consumer disappeared.

This PR removes that stale definition instead of continuing to propagate an
unused build macro.

Clean up repeated CUDA source-list handling

The stable CUDA CMake block now uses more specific source-list variable names
for related kernels instead of repeatedly reusing generic names like SRCS.

This is mostly a readability and maintenance fix: the generated source lists are
still passed through the same per-source gencode helpers, but the CMake file now
makes it clearer which feature each source group belongs to.

Runtime CUDA architecture warning

This PR exports the final filtered CUDA architecture list from the stable
libtorch extension:

• CMake records the final CUDA_ARCHS value after vLLM's filtering logic has
  run.
• The exported metadata preserves the semantic architecture suffix:
  • CUDA 13+ family targets are reported as 10.0f, 11.0f, 12.0f, etc.
  • CUDA 12.8/12.9 arch-specific targets are reported as 10.0a, 10.1a,
    10.3a, 12.0a, 12.1a, etc.
• C++ exposes the raw comma-separated string through
  torch.ops._C.get_compiled_cuda_archs().
• Python splits and interprets that string in vllm/platforms/cuda.py.

On NVML-backed CUDA startup, vLLM now checks all visible CUDA devices against
the compiled architecture list and logs a warning when the wheel does not cover
one of those devices.

This intentionally runs in the NVML-backed warning path so that the existing
"do not initialize CUDA during platform probing" behavior is preserved. The
non-NVML path does not perform this check because it would require CUDA runtime
initialization earlier than before.

The warning is especially useful for pre-built wheels where the release or build
pipeline may use a smaller architecture list than a local source build. For
example:

• CUDA 12.9 wheels are arch-specific and may omit newer targets such as
  sm_103 or sm_121.
• CUDA 13 wheels can use family targets such as sm_100f, sm_110f, and
  sm_120f, which changes what it means for a device to be covered.

Documentation updates

The CUDA installation documentation now calls out that pre-built wheel
architecture coverage comes from the release/build pipeline, not just from
CMakeLists.txt.

The troubleshooting guide now includes a CUDA architecture coverage section for
two common symptoms:

• a startup warning saying the current CUDA device is not covered by the wheel;
• a CUDA runtime failure like no kernel image is available for execution on the device.

The suggested remediation is to use a wheel whose CUDA architecture list covers
the device, use a CUDA 13 wheel where family targets are available, or build
from source with a suitable TORCH_CUDA_ARCH_LIST.

Related PRs

• Follow-up to the CUDA architecture cleanup in [Build] Fix CUDA arch build coverage gaps #45277.
• Fixes the Thor grouped MoE direction discussed around [Bugfix] Re-enable FP8 MoE on NVIDIA Thor #46339.
• Removes stale ENABLE_CUTLASS_MLA plumbing left after Remove old cutlass mla #23961 removed the old
  CUTLASS MLA consumer originally introduced by [NVIDIA] Support Cutlass MLA for Blackwell GPUs #16032.
• Related but not duplicating Add SM120 for NVFP4 tensor core support #41738, which focuses on SM120 NVFP4 coverage.

Duplicate-work check

I checked open PRs before preparing this description:

• gh pr list --repo vllm-project/vllm --state open --search "CUDA arch wheel warning Thor CUTLASS in:body"
  returned no open PRs.
• gh pr list --repo vllm-project/vllm --state open --search "VLLM_GPU_FLAGS ENABLE_CUTLASS_MOE in:body"
  returned no open PRs.
• gh pr list --repo vllm-project/vllm --state open --search "compiled cuda archs no kernel image in:body"
  returned Add SM120 for NVFP4 tensor core support #41738, which is related to SM120 NVFP4 coverage but does not cover
  the source-local macro scoping, Thor grouped MoE support gate, runtime
  compiled-arch warning, or documentation changes in this PR.

No issue number was provided for an issue-specific
gh issue view <issue_number> --comments duplicate-work check.

Test Plan

Will be tested by the CI and release pipeline. Some real-device tests are also needed.

Test Result

• CI pipeline: https://buildkite.com/vllm/ci/builds/75451
• Release pipeline: https://buildkite.com/vllm/release-v2/builds/3348
  • Artifacts: https://wheels.vllm.ai/d492d1e697613dbb73b7268284196d3330ccc2b4/

AI assistance disclosure

This PR was prepared with AI assistance from OpenAI Codex. The human submitter
is responsible for reviewing every changed line and validating the final PR.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results.
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

Documentation preview: https://vllm--47149.org.readthedocs.build/en/47149/

[Harry-Chen]

@claude review once

[Copilot]

Pull request overview

This PR tightens CUDA architecture handling across build, runtime, and docs so that (1) optional CUDA kernel feature macros are scoped to only the sources that need them, (2) runtime support checks align with what the wheel actually compiled, and (3) users receive an actionable startup warning when their visible GPU arch isn’t covered by the installed wheel.

Changes:

• Add a compiled-arch export from the stable libtorch extension and a CUDA startup warning that compares visible devices against the compiled arch list.
• Fix/rename the CUTLASS grouped MoE SM10x/SM11x feature gate (Thor/Blackwell) across CMake, C++, and Python.
• Refactor CUDA CMake source lists/flags to use per-source compile definitions, and document wheel arch coverage expectations in installation + troubleshooting docs.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vllm/platforms/cuda.py	Add compiled-arch parsing + NVML-path warning for uncovered visible CUDA devices.
vllm/_custom_ops.py	Update Python-side CUTLASS grouped GEMM capability guard for Thor/SM11x.
tests/kernels/quantization/test_cutlass_group_gemm_support.py	Add unit test ensuring Python guard allows Thor SM101/SM110 and excludes SM120.
tests/cuda/test_cuda_arch_warnings.py	Add tests for compiled-arch coverage logic and warning emission.
requirements/cuda.txt	Bump PyNvVideoCodec pin used in CUDA requirements set.
docs/usage/troubleshooting.md	Document the new “CUDA architecture not covered by the wheel” warning/error remediation.
docs/getting_started/installation/gpu.cuda.inc.md	Add installation-time warning about pre-built wheel arch coverage (CUDA 12.9 vs 13 family targets).
csrc/libtorch_stable/torch_bindings.cpp	Expose torch.ops._C.get_compiled_cuda_archs() from the stable extension.
csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu	Rename/extend grouped MoE SM10x/SM11x feature gate and align dispatch/support checks.
csrc/libtorch_stable/ops.h	Add declaration for get_compiled_cuda_archs() in the stable ops header.
CMakeLists.txt	Compute/export compiled CUDA arch metadata; scope optional kernel macros via per-source definitions; cleanup source list handling.
cmake/utils.cmake	Add set_compile_definitions_for_srcs() helper for source-local feature macros.

Comments suppressed due to low confidence (1)

vllm/_custom_ops.py:902

• torch.ops._C.cutlass_group_gemm_supported is always defined in the stable extension, but its implementation is omitted under USE_ROCM (see csrc/libtorch_stable/torch_bindings.cpp). On ROCm builds this call will raise RuntimeError (not AttributeError), so this wrapper can unexpectedly crash instead of returning False.

    if cuda_device_capability < 90 or cuda_device_capability >= 120:
        return False
    try:
        return torch.ops._C.cutlass_group_gemm_supported(cuda_device_capability)
    except AttributeError:

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Harry-Chen]

@DarkLight1337 Could you try the artifacts from https://wheels.vllm.ai/d492d1e697613dbb73b7268284196d3330ccc2b4/ to check whether it is working on Thor with the adjustments in this PR? Thanks!

[DarkLight1337]

You mean installing directly from the wheel while checking out to this branch? (i.e. no building from source)

[Harry-Chen]

You mean installing directly from the wheel while checking out to this branch? (i.e. no building from source)

I actually do not know the usability of the wheels. If they used to be usable, then you can try them, otherwise you can try compiling from source (but if we already have 11.0f in our arch list, what is the difference?)

[DarkLight1337]

Let me just build from source on this branch then, just to be sure. It will take a while, I'll update you tomorrow

[Harry-Chen]

Let me just build from source on this branch then, just to be sure. It will take a while, I'll update you tomorrow

Sure, you can also try the wheel to see if we still have gaps with a manual build and how we can improve.

[DarkLight1337]

Building from source works

[DarkLight1337]

The wheel works as well

[Harry-Chen]

This is cherry-picked from #47139, which is already merged.

[JaheimLee]

Can #47139 (comment) be fixed here?

[Harry-Chen]

Can #47139 (comment) be fixed here?

It is a separate issue, more related to #48540.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Harry-Chen.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork