[Fix] MiniCPM-V 4.6: skip legacy minicpmv conv template, fall back to…#25053
[Fix] MiniCPM-V 4.6: skip legacy minicpmv conv template, fall back to…#25053tc-mb wants to merge 1 commit into
Conversation
… HF jinja so prompt aligns with <|image_pad|>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request modifies the match_minicpm function to return None for the minicpmv4_6 model type, allowing the system to use the Hugging Face jinja chat template. Feedback suggests that the current implementation is too specific to version 4.6 and should be generalized to cover future versions as intended by the code comments.
| # MiniCPM-V 4.6 (and any future versions) ship a real HF jinja chat | ||
| # template that uses <|image_pad|> / <|video_pad|> instead of the legacy | ||
| # (<image>./</image>) placeholder. Returning None here lets the | ||
| # TemplateManager fall back to the HF tokenizer/processor template, | ||
| # which keeps the prompt aligned with the multimodal processor. | ||
| model_type = get_model_type(model_path) | ||
| if model_type == "minicpmv4_6": | ||
| return None |
There was a problem hiding this comment.
The comment mentions that this logic should apply to "any future versions", but the implementation specifically checks only for minicpmv4_6. If future versions (e.g., minicpmv4_7) or variants like minicpmo4_6 are released with HF jinja templates, they will fall through to the regex match on line 1161 and incorrectly return the legacy template. Consider using a more generic check (e.g., checking for a version prefix or a list of types) to align with the comment's intent and ensure future-proofing.
|
#24998 |
1 participant
Summary
match_minicpmmatches theminicpm-vsubstring inMiniCPM-V-4_6paths and forces the legacy v2.6 conv template ((<image>./</image>)), but 4.6's processor andchat_template.jinjause<|image_pad|>/<|video_pad|>— so the prompt and the processor are out of sync and images are never expanded.Returning
Noneformodel_type == "minicpmv4_6"lets the manager fall back to the HF jinja template. 2.6 / 4.0 / 4.5 / o-2.6 untouched. Verified end-to-end onMiniCPM-V-4_6(single-image, multi-image, text-only).