Skip to content

[Build] Fix cuda packaging build error - #29580

Merged
jambayk merged 1 commit into
mainfrom
tlwu/20260706/fix_cuda_build
Jul 7, 2026
Merged

[Build] Fix cuda packaging build error#29580
jambayk merged 1 commit into
mainfrom
tlwu/20260706/fix_cuda_build

Conversation

@tianleiwu

@tianleiwu tianleiwu commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes the CUDA packaging pipeline build error (with CUDA arch 52-real) that was introduced by #29451.

Root cause

The small-M batched GEMV path added in #29451 defines half helpers for the batched kernel — the Acc2<half> accumulator trait and the PackNatural / DotAccum / HorizontalAdd overloads — inside a single #if (!defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 530) guard that compiles the entire declaration out for archs below sm_53.

However, MatMulFloat4BatchedKernel<half> is launched from host code and is therefore instantiated for every target architecture, including sm_52. During the sm_52 device pass those half helpers no longer exist, so nvcc reported (100 errors, e.g.):

  • incomplete type "Acc2<half>" is not allowed
  • no suitable user-defined conversion from "WPack<half>" to "const WPack<nv_bfloat16>" exists (the compiler fell back to the nv_bfloat16 overloads)

Fix

Mirror the existing nv_bfloat16 convention in the same file: always define the type and function signatures, and gate only the arch-specific (half2-intrinsic) bodies. Now Acc2<half> is always a complete type and the half overloads always exist, so MatMulFloat4BatchedKernel<half> compiles for sm_52. On archs below sm_53 the bodies compile to no-ops (returning zeros), which matches the already-shipped nv_bfloat16 behavior for archs below sm_80.

Motivation and Context

The CUDA packaging pipeline builds with CMAKE_CUDA_ARCHITECTURES=52-real;61-real;75-real;86-real;89-real;90-virtual, so the sm_52 device pass is required and the pipeline was broken by #29451.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to fix CUDA packaging pipeline build failures when compiling for sm_52 (“52-real”), introduced by PR #29451, by ensuring certain half2-based helper signatures are available even when the half2-intrinsic implementations are not.

Changes:

  • Removes/relaxes __CUDA_ARCH__ >= 530 gating around some half-specific helper type definitions.
  • Refactors half helper functions (PackNatural, DotAccum, HorizontalAdd) so their signatures are always present while their bodies are conditionally compiled.

Comment thread onnxruntime/contrib_ops/cuda/quantization/matmul_4bits.cu
Comment thread onnxruntime/contrib_ops/cuda/quantization/matmul_4bits.cu
Comment thread onnxruntime/contrib_ops/cuda/quantization/matmul_4bits.cu
@jambayk
jambayk enabled auto-merge (squash) July 7, 2026 00:08
@jambayk
jambayk merged commit 2125ec4 into main Jul 7, 2026
92 of 95 checks passed
@jambayk
jambayk deleted the tlwu/20260706/fix_cuda_build branch July 7, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants