CUDA: fix kernel selection logic for tile FA - #19686
Conversation
|
@JohannesGaessler CI seems to dislike something: |
2026-02-17T14:06:08.1940247Z [MUL_MAT] ERR = 0.000589984 > 0.000500000 MUL_MAT(type_a=q5_1,type_b=f32,m=16,n=1,k=256,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1): �[1;31mFAIL�[0mThe failing test is a matrix multiplication which is 100% unaffected by the changes in this PR. Since we are using random seeds in the tests it can happen from time to time that the small differences between the CUDA and CPU backends randomly exceed the tolerance we set, particularly if the input matrices are small and the differences are less likely to average out. |
ggerganov
left a comment
There was a problem hiding this comment.
Adding a comment/reference of the DV limit would be useful.
Aight, just throwing out an idea: maybe it would be a good idea to vary the permitted NMSE divergence according to tensor size, to avoid false positives like this? |
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
* CUDA: fix kernel selection logic for tile FA * add comment
Fixes #19652 .
The problem is that the kernel selection logic is slightly wrong so the host code runs into an abort.