Skip to content

[ROCm][Kernel][V1] Enable AMD Radeon GPU Custom Paged Attention on v1 #17004

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 21, 2025
Merged

[ROCm][Kernel][V1] Enable AMD Radeon GPU Custom Paged Attention on v1 #17004

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4f6bd68
Enable custom paged attention kernel for Navi3x
hyoon1 Feb 22, 2025
7c49487
add navi4x support for custom paged attention kernel
hyoon1 Mar 5, 2025
7af074d
Enable AMD Radeon GPU Custom Paged Attention on v1
hyoon1 Apr 21, 2025
b877a0d
Merge branch 'upstream_main' into cpa_v1_navi
hyoon1 May 2, 2025
6c1ec10
remove unnecessary arguments
hyoon1 May 2, 2025
87c7c3e
chage gpu naming convention
hyoon1 May 2, 2025
b5fdeb5
cache is_navi_gpu result
hyoon1 May 5, 2025
12a4b93
Merge branch 'upstream_main' into cpa_v1_navi
hyoon1 May 8, 2025
95d92da
fix vheloop to ensure minimum value of 1
hyoon1 May 8, 2025
e6831c9
Merge branch 'upstream_main' into cpa_v1_navi
hyoon1 May 8, 2025
543681b
Merge branch 'upstream_main' into cpa_v1_navi
hyoon1 May 13, 2025
79a4f26
Merge branch 'upstream_main' into cpa_v1_navi
hyoon1 May 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion benchmarks/kernels/benchmark_paged_attention.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def main(
if version == "v2":
if current_platform.is_rocm():
global PARTITION_SIZE
if not args.custom_paged_attn:
if not args.custom_paged_attn and not current_platform.is_navi():
PARTITION_SIZE = 1024
else:
PARTITION_SIZE = PARTITION_SIZE_ROCM
Expand Down Expand Up @@ -166,6 +166,7 @@ def run_cuda_benchmark(num_iters: int, profile: bool = False) -> float:
scale,
block_tables,
seq_lens,
None,
block_size,
max_seq_len,
alibi_slopes,
Expand Down
Loading