Skip to content

fix: guard oversized shared memory driver enums with CUDA >= 13.4 #4377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jimmyzho merged 2 commits into from
Aug 18, 2026
Merged

fix: guard oversized shared memory driver enums with CUDA >= 13.4 #4377

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fea687f
fix: guard oversized shared memory driver enums with CUDA >= 13.4
JiaxuanBai Aug 6, 2026
ab45d03
Merge branch 'main' into fix_enums
jimmyzho Aug 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions include/flashinfer/trtllm/fmha/fmhaKernels.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,9 @@ class TllmGenFmhaKernel {
}

void setupKernelSmem(CUfunction func, KernelMeta const& kernelMeta) const {
#if CUDA_VERSION >= 13030
// CU_FUNC_ATTRIBUTE_SHARED_MEMORY_MODE and the ALLOW_OVERSIZED enum value require
// CUDA >= 13.4; CUDA 13.3 only ships the launch-attribute half of this API.
#if CUDA_VERSION >= 13040
if (isRubinOversized(mSM, kernelMeta.mSharedMemBytes)) {
cuErrCheck(cuFuncSetAttribute(func, CU_FUNC_ATTRIBUTE_SHARED_MEMORY_MODE,
CU_SHARED_MEMORY_MODE_ALLOW_OVERSIZED_SHARED_MEMORY));
Expand All @@ -147,7 +149,7 @@ class TllmGenFmhaKernel {
void appendOversizedSmemLaunchAttr(CUlaunchAttribute* launch_attribute,
CUlaunchConfig& launch_config,
KernelMeta const& kernelMeta) const {
#if CUDA_VERSION >= 13030
#if CUDA_VERSION >= 13040
if (isRubinOversized(mSM, kernelMeta.mSharedMemBytes)) {
IKL_LOG_DEBUG(
"TRTLLM-Gen launch info: using oversized shared memory for kernel %s (smem=%u bytes)",
Expand All @@ -164,7 +166,7 @@ class TllmGenFmhaKernel {
CUdevice device;
cuErrCheck(cuCtxGetDevice(&device));
int smem_bytes = 0;
#if CUDA_VERSION >= 13030
#if CUDA_VERSION >= 13040
if (smArch == kSM_107) {
cuErrCheck(cuDeviceGetAttribute(
&smem_bytes, CU_DEVICE_ATTRIBUTE_MAX_OVERSIZED_SHARED_MEMORY_PER_BLOCK, device));
Expand Down
Loading