Skip to content

GQA-4 example#833

Merged
aska-0096 merged 1 commit into
kabafrom
GQA
Aug 8, 2023
Merged

GQA-4 example#833
aska-0096 merged 1 commit into
kabafrom
GQA

Conversation

@aska-0096
Copy link
Copy Markdown
Contributor

  • Number of Query Group is static template argument
  • Number of Head need to be multiple of Number of Query Group

make example_grouped_query_attention_forward_wmma_fp16
./bin/example_grouped_query_attention_forward_wmma_fp16

@aska-0096 aska-0096 merged commit 3ba0f0d into kaba Aug 8, 2023
@illsilin illsilin deleted the GQA branch December 14, 2023 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant