Fixes #15247 | Update chat.cpp to support (at least) qwen3 reasoning + tool_choice = required

[ExtReMLapin]

There was this issue that we would not use tool_choice at required with reasoning because of forced tool call imposed by grammar.

Grammar now allows the model to think first.

Reasoning = big brain

Tool calling = strong arms

Now you can be very smart and very strong

Fixes #15247

[ExtReMLapin]

All am I sure is that it fixes issues with Qwen3 + reasoning (enabled or disabled)+ tool calling.
I fear it also has to be implemented for others reasoning parsers (non hermes)

CC hermes2 contributor : @ochafik

[ExtReMLapin]

Back at the office on tuesday, re-reading the PR i might reconsider the logic around

thinking_forced_open

[ExtReMLapin]

For a future PR, the following functions needs the same kind of patch :

• common_chat_params_init_granite : same as qwen3, but i'm not sure as the thinking tags are not always there
• common_chat_params_init_command_r7b : "<|START_THINKING|>", "<|END_THINKING|>"
• common_chat_params_init_deepseek_r1 : same as qwen3
• common_chat_params_init_gpt_oss ???

ready for review @ggerganov

Not sure exactly who I should ping as ochafik seems to be busy this week

[ExtReMLapin]

Not sure who I should ping

@slaren

[ggerganov]

Same comment as #15019 (comment)

This is a smaller change, so I can take a look and merge this, but prefer if we have someone who would take over this part of the code.

[ExtReMLapin]

Got it, thanks for keeping me updated !

[ExtReMLapin]

If you are being held hostage, blink twice @ochafik

[ochafik]

Thanks @ExtReMLapin ! Looks very promising :-)

Could you add some tests in test-chat.cpp (will need to force git add models/templates/qwen3-something.jinja, see models/templates/README.md )

[ExtReMLapin]

So I added two sets of tests, one in test-chat.cpp (100% ai generated, but tested) and I'm less that meh-ly conviced of their point.

I would have much more liked some kind of grammar check that filter/accepts/prune the reasoning part.

And this is why I added another test inside test_tool_call.py of the server tool, which is in my opinion not the perfect place because chat.cpp is not only used in the server, but it's an e2e test and much more straighforward at checking if things works.

Test results before this PR :

===================================================================================================== short test summary info ======================================================================================================
FAILED unit/test_tool_call.py::test_required_tool_with_reasoning[tool0-unsloth/Qwen3-0.6B-GGUF:Q4_K_M-None-deepseek-CompletionMode.NORMAL] - AssertionError: Expected reasoning content, but got None
FAILED unit/test_tool_call.py::test_required_tool_with_reasoning[tool0-unsloth/Qwen3-0.6B-GGUF:Q4_K_M-None-deepseek-CompletionMode.STREAMED] - AssertionError: Expected reasoning content, but got None
FAILED unit/test_tool_call.py::test_required_tool_with_reasoning[tool1-unsloth/Qwen3-0.6B-GGUF:Q4_K_M-None-deepseek-CompletionMode.NORMAL] - AssertionError: Expected reasoning content, but got None
FAILED unit/test_tool_call.py::test_required_tool_with_reasoning[tool1-unsloth/Qwen3-0.6B-GGUF:Q4_K_M-None-deepseek-CompletionMode.STREAMED] - AssertionError: Expected reasoning content, but got None
================================================================================================ 4 failed, 463 deselected in 5.79s =================================================================================================

[Inferior 1 (process 936362) detached]
terminate called after throwing an instance of 'std::runtime_error'
  what():  Test failed
Abandon (core dumped)

---

After :

unit/test_tool_call.py ....                                                                                                                                                                                                  [100%]

================================================================================================ 4 passed, 463 deselected in 6.07s =================================================================================================

[chat] All tests passed!