[Build] Fix cuda packaging build error - #29580
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to fix CUDA packaging pipeline build failures when compiling for sm_52 (“52-real”), introduced by PR #29451, by ensuring certain half2-based helper signatures are available even when the half2-intrinsic implementations are not.
Changes:
- Removes/relaxes
__CUDA_ARCH__ >= 530gating around some half-specific helper type definitions. - Refactors half helper functions (
PackNatural,DotAccum,HorizontalAdd) so their signatures are always present while their bodies are conditionally compiled.
jambayk
approved these changes
Jul 7, 2026
jambayk
enabled auto-merge (squash)
July 7, 2026 00:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes the CUDA packaging pipeline build error (with CUDA arch
52-real) that was introduced by #29451.Root cause
The small-M batched GEMV path added in #29451 defines
halfhelpers for the batched kernel — theAcc2<half>accumulator trait and thePackNatural/DotAccum/HorizontalAddoverloads — inside a single#if (!defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 530)guard that compiles the entire declaration out for archs belowsm_53.However,
MatMulFloat4BatchedKernel<half>is launched from host code and is therefore instantiated for every target architecture, includingsm_52. During thesm_52device pass thosehalfhelpers no longer exist, sonvccreported (100 errors, e.g.):incomplete type "Acc2<half>" is not allowedno suitable user-defined conversion from "WPack<half>" to "const WPack<nv_bfloat16>" exists(the compiler fell back to thenv_bfloat16overloads)Fix
Mirror the existing
nv_bfloat16convention in the same file: always define the type and function signatures, and gate only the arch-specific (half2-intrinsic) bodies. NowAcc2<half>is always a complete type and thehalfoverloads always exist, soMatMulFloat4BatchedKernel<half>compiles forsm_52. On archs belowsm_53the bodies compile to no-ops (returning zeros), which matches the already-shippednv_bfloat16behavior for archs belowsm_80.Motivation and Context
The CUDA packaging pipeline builds with
CMAKE_CUDA_ARCHITECTURES=52-real;61-real;75-real;86-real;89-real;90-virtual, so thesm_52device pass is required and the pipeline was broken by #29451.