support MiniCPM5 XML tool calls - #24743
Open
zkh2016 wants to merge 7 commits into
Open
Conversation
MiniCPM5-2.8B ships a BPE tokenizer.json (LlamaTokenizerFast) plus a stale sentencepiece tokenizer.model with a different vocabulary. The minicpm converter hardcoded _set_vocab_sentencepiece(), so it picked the wrong vocab and the model produced garbage / immediate-EOS despite loading fine. - minicpm set_vocab(): fall back to the BPE path (_set_vocab_gpt2) when there is no sentencepiece tokenizer.model. - register the MiniCPM5 pre-tokenizer hash as the qwen2-style pre-tokenizer. Convert from a directory without tokenizer.model so the BPE path is used. Verified: tokenization now matches transformers, and short/long-context generation is correct on host and on the Hexagon NPU. (cherry picked from commit cd8f6f26b427cfbc570b6c09d4e4efa934efb1e6)
MiniCPM5 emits tool calls as attribute-keyed XML: <function name="NAME"><param name="KEY">VALUE</param>...</function> Two problems prevented llama.cpp from returning OpenAI-style tool_calls: 1. The shipped chat template references an undefined variable (has_tool_sep) in its assistant tool_call branch, so minja throws when the differential auto-parser probes it. Analysis then aborts and the generated grammar is empty. Add a normalized, analyzer-friendly template in models/templates/ that keeps the same <function>/<param> wire format (use via --chat-template-file). 2. Even with a clean template the differential analyzer mis-infers this attribute-keyed format. Pin it explicitly in the workarounds table as TAG_WITH_TAGGED with the right markers, and register the <function>/<param> delimiters as preserved tokens so they survive detokenization (they are special/control tokens) and reach the tool-call parser. Verified end-to-end on MiniCPM5-1B (llama-server --jinja): single-arg, multi-arg, and no-tool requests all produce correct tool_calls / content.
o7si
reviewed
Jun 22, 2026
Contributor
|
This is already taken care of here: #24889 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
support MiniCPM5 BPE tokenizer (fixes garbage generation)
support MiniCPM5 XML tool calls ()
Additional information
Requirements