Eval bug: Q4_K_M with vulkan generates garbage/repetitive output #16961
Description
Name and Version
version: 6933 (fcfce04)
built with cc (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0 for x86_64-linux-gnu
Operating systems
Linux
GGML backends
Vulkan
Hardware
i5-1135G7
Models
ggml-org/gemma-3-1b-it-Q4_K_M.gguf
Problem description & steps to reproduce
When I run llama-cli, any prompt gives inappropriate output. Often a short sequence is repeated endlessly. Generated text is unrelated to the prompt.
Note:
- the same model works fine on CPU => vulkan backend problem
- using Q8_0 or f16 models works fine with vulkan backend => Q4 specific problem
- I have the feeling that it depends on previous vulkan run. It may be an init problem.
First Bad Commit
No response
Relevant log output
Examples of truncated generation:
> go
부드럽게 바꾸기 100% 화이트보드
(필요한 경우)
1. 이름
2. 연락처
3. 이메일
4. 프로필 사진
> go
hubulong 2023 + 2024 to a complex system. This would involve many layers and complex interactions. The project would aim to create a truly dynamic system. The complexity is a core element of the project, ensuring the system isn't static.
> go
April 1st 2024 2024 to 2025 [2025] to 2026 - 2027 - 2028 2029 - 2030 2031 - 2032 to 2033 - 2034 - 2035 - 2036 - 2037 - 2038 - 2039 - 2040 - 2041 - 2042 - 2043 - 2044 - 2045 - 2046 - 2047 - 2048 - 2049 - 2050 - 2051 - 2052 - 2053 - 2054 - 2055 - 2056 - 2057 - 2058 - 2059 - 2060 - 2061 - 2062 - 2063 - 2064 - 2065 - 2066 - 2067 - 2068 - 2069
> go
부활 13th 5th 4th 3rd 2nd 1st 0th 9th 8th 7th 6th 5th 4th 3rd 2nd 1st 0th 9th 8th 7th 6th 5th 4th 3rd 2nd 1st 0th 9th 8th 7th 6th 5th 4th 3rd 2nd 1st 0th 9th 8th 7th
> go
부드러운 2780809699
and 36300356589
and 29314271987
and 54752771472
and 38856176395
and 23827691788
> go
’er-ish’s’ to ‘er’s’ + ‘er’s’ and ‘er’s’ to ‘er’s’ + ‘er’s’ and so on to 9 + 1 to 9 + 1.
This is a complex expression and it's not a standard programming concept. It's a method of generating a long string of characters.
Let's try a simple example:
Input: "hello"
Output: "helloworld"
Input: "world"
Output: "helloworld"
Input: "hello"
Output: "helloworld"
> go
부드러운 1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1
> go
부침과 같은 것과 같은 것과 같은 것
와 같은 것.
**설명:**
이것은 매우 복잡하고 난해한 질문이며, 그에 대한 답을 얻는 것은 매우 어려운 일입니다. 특히 복잡한 수학적 개념과 개념에 대한 이해를 필요로 합니다.
**이 질문의 핵심은 다음과 같습니다.**
> go
ensical 92/112/112-2023-9.
9.26. 3.23. 14.21. 13.53. 17.17. 16.20. 13.93. 16.61. 15.11. 17.30. 14.73. 14.13. 14.12. 14.13. 14.13. 14.13. 14.13. 14.13. 14.13. 14.13. 14.13. 14.13. 14.13.
> go
부드럽게 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95% + 95%