Skip to content

Cuda fa mma 5 #330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 2, 2025
Merged

Cuda fa mma 5 #330

merged 4 commits into from
Feb 2, 2025

Conversation

@Nexesenex
Copy link
Owner

No description provided.

@Nexesenex Nexesenex merged commit a8fb3b7 into Nexesenex:lcpp_pr_cuda_mma Feb 2, 2025
6 of 7 checks passed
Nexesenex added a commit that referenced this pull request May 14, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
9999fa2 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex pushed a commit that referenced this pull request May 24, 2025
@ikawrakow
Allow q8_0 KV cache for head size 256 (#330)
1bbb143 
* Allow q8_0 KV cache for head size 256

* We need also these

---------

Co-authored-by: Iwan Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
6a1397b 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
2fccedc 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
6fe2ef5 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 9, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
4804234 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 9, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
f90af19 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 10, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
c28404f 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 11, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
37ece27 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 12, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
6beb466 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 12, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
fc1b8f4 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 13, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
da82c6e 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 14, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
8aa3fc8 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
31e5439 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
c9379ea 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
cbfda8d 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
a1eff3e 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
758e9c8 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 15, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
7206d44 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Aug 18, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
984383a 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 5, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
47732f6 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 7, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
4639a5c 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 7, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
0e4d0e4 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 9, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
7cee4e5 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 9, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
528e105 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
abd01e6 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
37904d7 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
9f73f3f 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
42c9079 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 12, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
7f1c819 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 13, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
86c9a69 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 13, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
1132c9f 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 16, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
30f3f7a 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 16, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
309197e 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 18, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
88f66b5 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 19, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
9c8fb20 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 20, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
9946197 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
c86a854 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
0af82d6 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
816cece 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
4e07034 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
03fdb2d 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
c849518 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 23, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
fae97ea 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 24, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
d00f03b 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
0bb6648 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
99b99d5 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
deba739 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 27, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
c5126e8 
Co-Authored-By: Kawrakow <[email protected]>
Nexesenex added a commit that referenced this pull request Oct 28, 2025
@Nexesenex @ikawrakow
Allow q8_0 KV cache for head size 256 #330
8f46ee1 
Co-Authored-By: Kawrakow <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

devops examples ggml Nvidia GPU python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Nexesenex @ericcurtin @ochafik @JohannesGaessler