Misc. bug: Problems with official jinja templates (Gemma 2, Llama 3.2, Qwen 2.5) #11866
Description
Name and Version
llama-cli --version
version: 4713 (a4f011e)
built with MSVC 19.42.34436.0 for x64
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-server
Command line
1. llama-server -ngl 99 -m gemma-2-2b-it-Q8_0.gguf --jinja --chat-template-file gemma2.jinja -c 8192
2. llama-server -ngl 99 -m Llama-3.2-3B-Instruct-Q8_0.gguf --jinja --chat-template-file llama3.2.jinja -c 8192
3. llama-server -ngl 99 -m Qwen2.5-1.5B-Instruct-Q8_0.gguf --jinja --chat-template-file qwen2.5.jinja -c 8192Problem description & steps to reproduce
Extracting official chat templates from chat_template field in tokenizer_config.json (Gemma 2, Llama 3.2, Qwen 2.5), storing them in files, and then trying to use them with llama-server results in errors.
- Gemma 2:
parse: error parsing grammar: expecting name atafter each message. - Llama 3.2: server doesn't start.
- Qwen 2.5:
parse: error parsing grammar: expecting name atafter each message.
@ochafik Could you look into this? It would be nice to have jinja implementation fully working with official templates, at least for major models.
First Bad Commit
No response