Fix tool calls dropped when closed by EOS without an end marker

[rayanhanader]

What

When a model with no tool-call end marker (the Mistral family, where tool_call_end_tokens is empty) emits a tool call, mlx_lm.server returns an empty assistant message with finish_reason: "stop", even though the model generated a valid [TOOL_CALLS] ... sequence and completion_tokens is non-zero.

Why

The state machine transitions normal -> tool on the [TOOL_CALLS] token and buffers everything after it into tool_text. With no end marker the tool call is closed by the EOS token, which transitions tool -> None, so the last token of the generation reports its state as None.

The flush after the accumulation loop in handle_completion is guarded by prev_state == "tool", which is None at that point, so the buffered tool call is never appended. made_tool_call stays False, finish_reason stays "stop", and the message comes back empty. The parser is never reached, which is why there is no Failed to parse tool call warning either.

Fix

Flush whenever tool_text is non-empty. It can only be non-empty when a tool call was opened and not yet flushed, so this is also safe for models that do have an end marker, since their tool_text is reset to "" on the tool -> normal transition.

Test

Added test_tool_call_without_end_marker_is_not_dropped in tests/test_server.py. It mocks generation with a scripted token stream that ends in the EOS to None transition and asserts the tool call is returned with finish_reason: "tool_calls". It fails on main and passes with the fix. The full test_server.py suite passes and pre-commit (black, isort) is clean.

Notes

I could not run the 24B model from the report on a 16 GB machine, so I reproduced the empty-message behavior on Ministral-8B-Instruct-2410-4bit (same empty tool_call_end_tokens). To confirm the report's model is the same case, I checked the tokenizer and chat template of mlx-community/mistralai_Devstral-Small-2-24B-Instruct-2512-MLX-MXFP4: tool_call_end_tokens is empty, the template renders tool calls as [TOOL_CALLS]<name>[ARGS]<json_args> which the current mistral.py parser already handles, and there is no tool-call-id length constraint. So for that model this is purely the flush bug.

Fixes #1307.