Skip to content

Use single mma warp group for short q_len in FA to optimize decoding performance #18985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fridge003 merged 4 commits into from
Feb 23, 2026
Merged

Use single mma warp group for short q_len in FA to optimize decoding performance #18985

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4022f07
upd
Qiaolin-Yu Feb 18, 2026
6b11984
Merge branch 'main' into fa
Qiaolin-Yu Feb 18, 2026
32bafa4
upd
Qiaolin-Yu Feb 20, 2026
f0cc447
Merge branch 'main' into fa
Qiaolin-Yu Feb 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions sgl-kernel/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ FetchContent_Populate(repo-flashinfer)
FetchContent_Declare(
repo-flash-attention
GIT_REPOSITORY https://github.com/sgl-project/sgl-attn
GIT_TAG f866ec34002250e74c8bbcbcffa0e1ae71300b2d
GIT_TAG cc75c5c5979a607ad20a6828635646f9841acf01
GIT_SHALLOW OFF
)
FetchContent_Populate(repo-flash-attention)
Expand Down Expand Up @@ -464,21 +464,21 @@ if (SGL_KERNEL_ENABLE_FA3)
endif()

file(GLOB FA3_BF16_GEN_SRCS
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimall_bf16*_sm90.cu")
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9]*_bf16*_sm90.cu")
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The glob pattern [0-9]* can match an empty string, which might lead to unintended files being included if, for example, a file named flash_fwd_hdim_bf16_sm90.cu exists. To ensure that you only match files with at least one digit in the head dimension, it's safer to use [0-9][0-9]*.

        "${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9][0-9]*_bf16*_sm90.cu")

file(GLOB FA3_BF16_GEN_SRCS_
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimdiff_bf16*_sm90.cu")
list(APPEND FA3_BF16_GEN_SRCS ${FA3_BF16_GEN_SRCS_})

# FP16 source files
# FP16 source files - use individual hdim files instead of hdimall to avoid ptxas crash
file(GLOB FA3_FP16_GEN_SRCS
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimall_fp16*_sm90.cu")
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9]*_fp16*_sm90.cu")
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For consistency and to prevent accidentally matching files without a head dimension, it's better to use [0-9][0-9]* instead of [0-9]*. This ensures at least one digit is present.

        "${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9][0-9]*_fp16*_sm90.cu")

file(GLOB FA3_FP16_GEN_SRCS_
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimdiff_fp16*_sm90.cu")
list(APPEND FA3_FP16_GEN_SRCS ${FA3_FP16_GEN_SRCS_})

# FP8 source files
file(GLOB FA3_FP8_GEN_SRCS
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimall_e4m3*_sm90.cu")
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9]*_e4m3*_sm90.cu")
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For consistency and to prevent accidentally matching files without a head dimension, it's better to use [0-9][0-9]* instead of [0-9]*. This ensures at least one digit is present.

        "${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdim[0-9][0-9]*_e4m3*_sm90.cu")

file(GLOB FA3_FP8_GEN_SRCS_
"${repo-flash-attention_SOURCE_DIR}/hopper/instantiations/flash_fwd_hdimdiff_e4m3*_sm90.cu")
list(APPEND FA3_FP8_GEN_SRCS ${FA3_FP8_GEN_SRCS_})
Expand Down
Loading