fix: guard oversized shared memory driver enums with CUDA >= 13.4 by JiaxuanBai · Pull Request #4377 · flashinfer-ai/flashinfer

JiaxuanBai · 2026-08-06T09:38:16Z

CU_FUNC_ATTRIBUTE_SHARED_MEMORY_MODE,
CU_SHARED_MEMORY_MODE_ALLOW_OVERSIZED_SHARED_MEMORY and CU_DEVICE_ATTRIBUTE_MAX_OVERSIZED_SHARED_MEMORY_PER_BLOCK were introduced in CUDA 13.4; CUDA 13.3 only ships the launch-attribute half of the shared-memory-mode API (CU_LAUNCH_ATTRIBUTE_SHARED_MEMORY_MODE, CUsharedMemoryMode values 0-2). The #if CUDA_VERSION >= 13030 guards added in #4122 / relanded in #4280 therefore break the trtllm-gen fmha_gen JIT build on CUDA 13.3 (the current released toolkit) with
fmhaKernels.cuh(136/157/169): error: identifier ... is undefined
on any GPU arch, e.g. via trtllm_batch_decode_with_kv_cache_mla on SM100.

Bump the three guards to #if CUDA_VERSION >= 13040. No CUDA 13.3 fallback is needed: the 13.3 driver does not support the oversized mode, so falling through to the existing MAX_DYNAMIC_SHARED_SIZE_BYTES path is correct.

References:

CUDA 13.3 Driver API (symbols absent): https://docs.nvidia.com/cuda/archive/13.3.0/cuda-driver-api/group__CUDA__TYPES.html
CUDA 13.4 developer preview Driver API (symbols present): https://docs.nvidia.com/cuda/developer-preview/13.4/pdf/CUDA_Driver_API.pdf

Verified with CUDA 13.3 (nvcc V13.3.73) on SM100:
tests/attention/test_trtllm_gen_mla.py::test_trtllm_batch_decode_mla (trtllm-gen backend) fails to build before, passes after.

📌 Description

🔍 Related Issues

🚀 Pull Request Checklist

Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete.

✅ Pre-commit Checks

I have installed pre-commit by running pip install pre-commit (or used your preferred method).
I have installed the hooks with pre-commit install.
I have run the hooks manually with pre-commit run --all-files and fixed any reported issues.

If you are unsure about how to set up pre-commit, see the pre-commit documentation.

🧪 Tests

Tests have been added or updated as needed.
All tests are passing (unittest, etc.).

Reviewer Notes

Summary by CodeRabbit

Bug Fixes
- Updated CUDA compatibility checks for oversized shared-memory support.
- Ensured related functionality is available only with CUDA 13.4 or newer, improving compatibility with supported CUDA environments.

CU_FUNC_ATTRIBUTE_SHARED_MEMORY_MODE, CU_SHARED_MEMORY_MODE_ALLOW_OVERSIZED_SHARED_MEMORY and CU_DEVICE_ATTRIBUTE_MAX_OVERSIZED_SHARED_MEMORY_PER_BLOCK were introduced in CUDA 13.4; CUDA 13.3 only ships the launch-attribute half of the shared-memory-mode API (CU_LAUNCH_ATTRIBUTE_SHARED_MEMORY_MODE, CUsharedMemoryMode values 0-2). The `#if CUDA_VERSION >= 13030` guards added in flashinfer-ai#4122 / relanded in flashinfer-ai#4280 therefore break the trtllm-gen fmha_gen JIT build on CUDA 13.3 (the current released toolkit) with fmhaKernels.cuh(136/157/169): error: identifier ... is undefined on any GPU arch, e.g. via trtllm_batch_decode_with_kv_cache_mla on SM100. Bump the three guards to `#if CUDA_VERSION >= 13040`. No CUDA 13.3 fallback is needed: the 13.3 driver does not support the oversized mode, so falling through to the existing MAX_DYNAMIC_SHARED_SIZE_BYTES path is correct. References: - CUDA 13.3 Driver API (symbols absent): https://docs.nvidia.com/cuda/archive/13.3.0/cuda-driver-api/group__CUDA__TYPES.html - CUDA 13.4 developer preview Driver API (symbols present): https://docs.nvidia.com/cuda/developer-preview/13.4/pdf/CUDA_Driver_API.pdf Verified with CUDA 13.3 (nvcc V13.3.73) on SM100: tests/attention/test_trtllm_gen_mla.py::test_trtllm_batch_decode_mla (trtllm-gen backend) fails to build before, passes after. AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

coderabbitai · 2026-08-06T09:38:38Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8e9dae96-1c63-4c78-8f9b-e213d1e40b43

📥 Commits

Reviewing files that changed from the base of the PR and between 4581d63 and ab45d03.

📒 Files selected for processing (1)

include/flashinfer/trtllm/fmha/fmhaKernels.cuh

🚧 Files skipped from review as they are similar to previous changes (1)

include/flashinfer/trtllm/fmha/fmhaKernels.cuh

Included review availability: Your plan includes up to 8 reviews per rolling hour; 7 remain after this review.

📝 Walkthrough

Walkthrough

The FMHA kernel raises oversized shared-memory API guards from CUDA 13.3 to CUDA 13.4 for setup, launch attributes, and device capability queries.

Changes

CUDA shared-memory guards

Layer / File(s)	Summary
Raise oversized shared-memory guards `include/flashinfer/trtllm/fmha/fmhaKernels.cuh`	The kernel setup, launch-attribute construction, and device capability query now require CUDA 13.4.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to ab45d

The change limits oversized shared-memory driver enums to CUDA 13.4 and preserves the existing path on CUDA 13.3, with no actionable merge-blocking risk remaining beyond normal checks and review.

Suggested reviewers: sricketts

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the CUDA version guard fix for oversized shared-memory driver enums.
Description check	✅ Passed	The description explains the issue, affected CUDA versions, fix, related issue, and verification results, although checklist items remain unchecked.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

JiaxuanBai · 2026-08-07T04:51:04Z

Hi @jimmyzho @aleozlx ! Could you please review this PR? Thanks!

JiaxuanBai · 2026-08-13T01:55:52Z

Hi @yzh119 , could you help take a look at this PR? Thanks.

jimmyzho

LGTM

jimmyzho · 2026-08-18T06:42:48Z

/bot run tests/attention

flashinfer-bot · 2026-08-18T06:43:51Z

GitLab MR !1256 has been created, and the CI pipeline #63246609 is currently running. I'll report back once the pipeline job completes.

flashinfer-bot · 2026-08-18T09:46:02Z

[FAILED] Pipeline #63246609 — 15/16 executed test jobs passed

Compared with nightly #63077496 (different CI configuration).

Unit Tests

GPU	CUDA 12.9	CUDA 13.0	Notes
B300	✅ Pass	✅ Pass	—
GB200	✅ Pass	✅ Pass	—
GB300	✅ Pass	✅ Pass	—
H100	✅ Pass	✅ Pass	—
RTX Pro 6000 Blackwell	✅ Pass	✅ Pass	—

✅ Pass · 🟡 Old failure · ❌ New failure · ⏱ Test timeout · ⚠️ Infrastructure · ❔ Unknown or unclassified · — Not run

Multi-GPU and Multi-Node Tests — 5/6 passed

GPU	CUDA 12.9	CUDA 13.0	Notes
B300 (multi-GPU)	✅ Pass	✅ Pass	—
GB200 (multi-node)	⚠️ Infra	✅ Pass	Infrastructure: `test infrastructure interrupted the job` (1 job; CUDA 12.9)
GB300 (multi-node)	✅ Pass	✅ Pass	—

Failure details

Timeouts, infrastructure, or incomplete jobs

Infrastructure: test infrastructure interrupted the job — GB200 / CUDA 12.9 / multi-node
- Jobs: multi_node_test_gb200: [cu129]

coderabbitai · 2026-08-18T16:19:53Z

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

jimmyzho · 2026-08-18T16:26:57Z

@flashinfer-bot run tests/attention

…ashinfer-ai#4377) flashinfer-ai#4375 CU_FUNC_ATTRIBUTE_SHARED_MEMORY_MODE, CU_SHARED_MEMORY_MODE_ALLOW_OVERSIZED_SHARED_MEMORY and CU_DEVICE_ATTRIBUTE_MAX_OVERSIZED_SHARED_MEMORY_PER_BLOCK were introduced in CUDA 13.4; CUDA 13.3 only ships the launch-attribute half of the shared-memory-mode API (CU_LAUNCH_ATTRIBUTE_SHARED_MEMORY_MODE, CUsharedMemoryMode values 0-2). The `#if CUDA_VERSION >= 13030` guards added in flashinfer-ai#4122 / relanded in flashinfer-ai#4280 therefore break the trtllm-gen fmha_gen JIT build on CUDA 13.3 (the current released toolkit) with fmhaKernels.cuh(136/157/169): error: identifier ... is undefined on any GPU arch, e.g. via trtllm_batch_decode_with_kv_cache_mla on SM100. Bump the three guards to `#if CUDA_VERSION >= 13040`. No CUDA 13.3 fallback is needed: the 13.3 driver does not support the oversized mode, so falling through to the existing MAX_DYNAMIC_SHARED_SIZE_BYTES path is correct. References: - CUDA 13.3 Driver API (symbols absent): https://docs.nvidia.com/cuda/archive/13.3.0/cuda-driver-api/group__CUDA__TYPES.html - CUDA 13.4 developer preview Driver API (symbols present): https://docs.nvidia.com/cuda/developer-preview/13.4/pdf/CUDA_Driver_API.pdf Verified with CUDA 13.3 (nvcc V13.3.73) on SM100: tests/attention/test_trtllm_gen_mla.py::test_trtllm_batch_decode_mla (trtllm-gen backend) fails to build before, passes after.  ## 📌 Description  ## 🔍 Related Issues  ## 🚀 Pull Request Checklist Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete. ### ✅ Pre-commit Checks - [ ] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [ ] I have installed the hooks with `pre-commit install`. - [ ] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. > If you are unsure about how to set up `pre-commit`, see [the pre-commit documentation](https://pre-commit.com/). ## 🧪 Tests - [ ] Tests have been added or updated as needed. - [ ] All tests are passing (`unittest`, etc.). ## Reviewer Notes   ## Summary by CodeRabbit * **Bug Fixes** * Updated CUDA compatibility checks for oversized shared-memory support. * Ensured related functionality is available only with CUDA 13.4 or newer, improving compatibility with supported CUDA environments.  Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Jimmy Zhou <79552142+jimmyzho@users.noreply.github.com>

JiaxuanBai requested review from Aneureka, aleozlx, bkryu, cyx-6, dhiraj113, feih-nv, jiahanc, jimmyzho, kahyunnam, nv-yunzheq, qiching, qsang-nv, saltyminty, samuellees, sricketts, yongwww, yyihuang and yzh119 as code owners August 6, 2026 09:38

jimmyzho approved these changes Aug 18, 2026

View reviewed changes

jimmyzho added the run-ci label Aug 18, 2026

Merge branch 'main' into fix_enums

ab45d03

jimmyzho enabled auto-merge (squash) August 18, 2026 16:23

jimmyzho added run-ci and removed run-ci labels Aug 18, 2026

flashinfer-bot added run-ci and removed run-ci labels Aug 18, 2026

jimmyzho merged commit 92ef090 into flashinfer-ai:main Aug 18, 2026
30 of 47 checks passed

lishapira mentioned this pull request Aug 30, 2026

CI: add vLLM + NIXL EP test to the EP CI job ai-dynamo/nixl#2154

Merged

Navigation Menu

Uh oh!

fix: guard oversized shared memory driver enums with CUDA >= 13.4 - #4377

fix: guard oversized shared memory driver enums with CUDA >= 13.4#4377
jimmyzho merged 2 commits into
flashinfer-ai:mainfrom
JiaxuanBai:fix_enums

JiaxuanBai commented Aug 6, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 6, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

JiaxuanBai commented Aug 7, 2026 •

edited

Loading

Uh oh!

JiaxuanBai commented Aug 13, 2026

Uh oh!

jimmyzho left a comment

Uh oh!

jimmyzho commented Aug 18, 2026

Uh oh!

flashinfer-bot commented Aug 18, 2026

Uh oh!

flashinfer-bot commented Aug 18, 2026

Timeouts, infrastructure, or incomplete jobs

Uh oh!

coderabbitai Bot commented Aug 18, 2026

Uh oh!

jimmyzho commented Aug 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

JiaxuanBai commented Aug 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📌 Description

🔍 Related Issues

🚀 Pull Request Checklist

✅ Pre-commit Checks

🧪 Tests

Reviewer Notes

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

JiaxuanBai commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JiaxuanBai commented Aug 13, 2026

Uh oh!

jimmyzho left a comment

Choose a reason for hiding this comment

Uh oh!

jimmyzho commented Aug 18, 2026

Uh oh!

flashinfer-bot commented Aug 18, 2026

Uh oh!

flashinfer-bot commented Aug 18, 2026

Unit Tests

Multi-GPU and Multi-Node Tests — 5/6 passed

Timeouts, infrastructure, or incomplete jobs

Uh oh!

coderabbitai Bot commented Aug 18, 2026

Uh oh!

jimmyzho commented Aug 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JiaxuanBai commented Aug 6, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 6, 2026 •

edited

Loading

JiaxuanBai commented Aug 7, 2026 •

edited

Loading