`server`: fix tool-call of DeepSeek R1 Qwen, return reasoning_content (Command 7RB & DeepSeek R1) unless `--reasoning-format none`#11607

Merged

ochafik merged 94 commits intoggml-org:masterfrom

ochafik:r1-toolcall

Feb 13, 2025

Commits on Feb 3, 2025

minja: enhance backfill of templates w/o tools description (use example tool call delta!)
ochafik
committed
pass vocab to common_chat_params_init
ochafik
committed
DeepSeek R1: parse thoughts / return in separate field in API (non streamed mode)
ochafik
committed
Avoid double bos w/ jinja
ochafik
committed
server/oai: ensure content is null when there are tool calls
ochafik
committed
update logs
ochafik
committed
rename tests
ochafik
committed
tool-call: allow `--jinja --chat-template chatml`
ochafik
committed
tool-call: fix command-r7b parsing when response is multiline
ochafik
committed
tool-calls: add DeepSeek R1 Qwen 7B to server test_hello_world
ochafik
committed
tell DS R1 not to overthink (weather test)
ochafik
committed
add deepseek models to server tool call section in readme
ochafik
committed
tool-call: allow `--jinja --chat-template chatml`
ochafik
committed
Update test_tool_call.py
ochafik
committed
minimize diffs
ochafik
committed
fix typo
Olivier Chafik
committed
fix double bos issue (drop bos/eos tokens from jinja template)
Olivier Chafik
committed
fix bad merge
Olivier Chafik
committed
fix build / rm diff
Olivier Chafik
committed
Merge branch 'jinja-chatml' into r1-toolcall
Olivier Chafik
committed
add missing try catch around jinja parsing to default to chatml
Olivier Chafik
committed
Merge branch 'jinja-chatml' into r1-toolcall
Olivier Chafik
committed
tool-calls: r1: add missing <｜tool▁calls▁end｜> to grammar!
Olivier Chafik
committed
tweak delta logic
Olivier Chafik
committed
tool-calls: accommodate variety of wrong tool call opening tags both Qwen 32B and 7B distills like to spit out
Olivier Chafik
committed
Simplify default chatml logic
Olivier Chafik
committed
Merge branch 'jinja-chatml' into r1-toolcall
Olivier Chafik
committed
tool-calls: add deepseek r1 template + accommodate broken official template slightly better
Olivier Chafik
committed
rm unneeded vocab
Olivier Chafik
committed
simpler hacky fixes for original broken template (+ fix minja example syntax polyfill)
Olivier Chafik
committed
sync: minja https://github.com/google/minja/pull/46
Olivier Chafik
committed
Merge branch 'master' into r1-toolcall
Olivier Chafik
committed
actually we want eos_token in the template to infer tool call examples, explicitly skipped in new template options
Olivier Chafik
committed
update to minja's new api
Olivier Chafik
committed

Commits on Feb 4, 2025

sync: minja
Olivier Chafik
committed
simplify hack to fix original template's backfill from minja
Olivier Chafik
committed
tool-call: r1: add one more trigger approx "<｜tool calls begin｜>"
Olivier Chafik
committed
r1: fix inadvertent newline in grammar before <｜tool▁call▁end｜>
ochafik
committed
tool-call: r1: fix grammar
ochafik
committed
move trigger_words init inside non-llguidance branch
ochafik
committed
fix / test parsing of r1 parser
ochafik
committed
Fix / test models/templates/llama-cpp-deepseek-r1.jinja
ochafik
committed
update test_calc_result
ochafik
committed
fix test_calc_result
ochafik
committed
fix spaces
ochafik
committed
`sync`: minja
ochafik
committed
Update test-chat.cpp
ochafik
committed
fix mistral chat test: need empty tokens
ochafik
committed
Update chat.cpp
ochafik
committed
Merge branch 'sync-minja-4' into r1-toolcall
ochafik
committed
server: check that content is null when we get tool_calls
ochafik
committed
tool-call: ensure we don't return content when there are tool calls / warn
ochafik
committed
fix mistral expectation
ochafik
committed
ensure deepseek r1 thoughts parsed even w/o tool calls
ochafik
committed
fix test-chat
ochafik
committed
Update chat.cpp
ochafik
committed
Merge branch 'sync-minja-4' into r1-toolcall
ochafik
committed
Fix r1 grammar since we made <｜tool▁calls▁begin｜> optional (triggering on just <｜tool▁call▁begin｜> for 7B's sake)
ochafik
committed
r1: revert making <｜tool▁calls▁begin｜> optional as somehow sampling triggers us on "<｜tool▁call▁begin｜><", which is already invalid per the grammar
ochafik
committed
return thoughts in reasoning_content field
Olivier Chafik
committed
update readme section about common model tool call formats
Olivier Chafik
committed
Merge branch 'master' into r1-toolcall
Olivier Chafik
committed
Update test_tool_call.py
Olivier Chafik
committed

Commits on Feb 5, 2025

Merge branch 'master' into r1-toolcall
ochafik
authored
--think to force any model to return reasoning_content (or just parse <think> for deepseek r1)
ochafik
committed
Merge branch 'r1-toolcall' of github.com:ochafik/llama.cpp into r1-toolcall
ochafik
committed
fix test_thoughts
ochafik
committed
fix compiler warning about parens
Olivier Chafik
committed
align Command R7B w/ --think / reasoning_content behaviour
Olivier Chafik
committed
Update README.md
Olivier Chafik
committed
fix --think arg env
Olivier Chafik
committed
disable some failing chatml tests
Olivier Chafik
committed
Update README.md
Olivier Chafik
committed
use existing string_strip
Olivier Chafik
committed
revert tool example backfill change - command 7rb just needs the right template
Olivier Chafik
committed

Commits on Feb 8, 2025

Merge remote-tracking branch 'origin/master' into r1-toolcall
ochafik
committed
Use --reasoning-format, remove forced thinking for now
ochafik
committed
return reasoning_content before content
ochafik
committed
update model template / format mapping
ochafik
committed
fix test-chat
ochafik
committed

Commits on Feb 9, 2025

rm thoughts from generic parser
ochafik
committed
sync: minja (https://github.com/google/minja/pull/52)
ochafik
committed
tool-calls: allow r1 output to miss <think> opening tag (since latest template update adds it)
ochafik
committed
sync: minja (https://github.com/ggerganov/llama.cpp/pull/11774)
ochafik
committed
rm wrong warning in command-r parser (when normal text)
ochafik
committed
update deepseek r1 templates (+ put update commands in ./scripts/get_chat_template.py's comments)
ochafik
committed
fix server test_tool_calls.py
ochafik
committed
add models/templates/README.md
ochafik
committed
fix test_calc_result & test_thoughts
ochafik
committed
fix test-chat (update delta to latest r1 template change)
ochafik
committed

Commits on Feb 12, 2025

Merge remote-tracking branch 'origin/master' into r1-toolcall
Olivier Chafik
committed

Commits on Feb 13, 2025

prefer json::at to operator[] in chat.cpp
Olivier Chafik
committed
Merge remote-tracking branch 'origin/master' into r1-toolcall
Olivier Chafik
committed
Apply suggestions from code review

ochafik
and
ggerganov
authored