Improve MiroThinker chat template compatibility with the new Jinja template engine by hksdpc255 · Pull Request #1404 · ikawrakow/ik_llama.cpp

hksdpc255 · 2026-03-11T12:21:20Z

At least it will no longer crash when starting llama-server.

Jinja only changes. Safe to merge.

I have read the contributing guidelines
Self-reported review complexity:
- Low
- Medium
- High

firecoperana · 2026-03-11T12:45:12Z

fa8893d Does this fix the crash?

hksdpc255 · 2026-03-11T12:55:44Z

@firecoperana Yes

firecoperana · 2026-03-11T13:11:30Z

My fix is mostly for user who does not need the template when using completions like text completions. It also helps in your case because template is not fully compatible.

CISC · 2026-03-11T14:02:07Z

Where is this template from? It's not the original one.

I'd urge you not to use the pop method as that is technically not allowed in chat templates (it just happens to work in our jinja parser (for now)).

hksdpc255 · 2026-03-12T00:52:23Z

Where is this template from? It's not the original one.

I'd urge you not to use the pop method as that is technically not allowed in chat templates (it just happens to work in our jinja parser (for now)).

@CISC The original template (https://github.com/ikawrakow/ik_llama.cpp/blob/afa6439ac35c5710c05583fd05e6aa866ebd0a02/models/templates/MiroThinker.jinja) crashes due to the line {%- set message.tool_calls = [] %}. The error message is:

common_chat_templates_init: error: Index 1 out of bounds for array of size 0
common_chat_templates_init: failed to initialize chat template
common_chat_templates_init: please consider disabling jinja via --no-jinja, or using another chat template
srv          init: init: chat template parsing error: std::exception
srv          init: init: please consider disabling jinja via --no-jinja, or use a custom chat template via --chat-template
srv          init: init: for example: --no-jinja --chat-template chatml
srv    operator(): operator(): cleaning up before exit...
main: exiting due to model loading error

I also cannot find any reference to tool_calls[0] in the template itself, so this behavior might indicate a bug in the llama.cpp Jinja engine. Changing it to .pop at least prevent it from crash when starting the server.

CISC · 2026-03-12T07:32:53Z

I also cannot find any reference to tool_calls[0] in the template itself, so this behavior might indicate a bug in the llama.cpp Jinja engine. Changing it to .pop at least prevent it from crash when starting the server.

~~That error is not from jinja though, so something else is failing.~~ Edit: it is, but you didn't paste the traceback?

Anyway, I guess you don't know who made this template?

hksdpc255 · 2026-03-12T08:03:24Z

No, I am the person who made the template.

ikawrakow · 2026-03-12T08:07:51Z

@CISC

You want to show us how it should be done with your own PR? Assuming of course this is possible without you being banned from the llama.cpp project.

CISC · 2026-03-12T08:11:51Z

@CISC

You want to show us how it should be done with your own PR? Assuming of course this is possible without you being banned from the llama.cpp project.

Not a problem, however it is tricky as this is modifying the messages object, which is not ok. You will have to make a copy to be compliant.

hksdpc255 · 2026-03-12T08:26:17Z

~~That error is not from jinja though, so something else is failing.~~ Edit: it is, but you didn't paste the traceback?

@CISC llama.cpp itself does not provide any traceback even with -lv 4. Only throw a exception.

hksdpc255 · 2026-03-12T08:30:28Z

I'm trying to keep all the modifications at the top of the chat template. It seems that MiroThinker’s fine-tuned models use different chat templates but share the same MCP-style tool calling. They preserve the system-message and user-message handling from the original chat template.

This way, I can simply copy and paste the top section into each model’s template and it should just work.

CISC · 2026-03-12T08:56:46Z

Assuming of course this is possible without you being banned from the llama.cpp project.

BTW, I'm not sure if you're just trolling or genuinely believe this, either way a bit strange as there is ample evidence to the contrary.

ikawrakow · 2026-03-12T09:08:50Z

@CISC

BTW, I'm not sure if you're just trolling or genuinely believe this, either way a bit strange as there is ample evidence to the contrary.

No, I'm not trolling, and I wouldn't know what evidence you are talking about. Obviously you can prove me wrong by adding a PR here.

CISC · 2026-03-12T09:20:33Z

BTW, I'm not sure if you're just trolling or genuinely believe this, either way a bit strange as there is ample evidence to the contrary.

No, I'm not trolling, and I wouldn't know what evidence you are talking about. Obviously you can prove me wrong by adding a PR here.

@hksdpc255 for one contributed here before contributing PRs that got merged in llama.cpp. Also a quick scan of AUTHORS reveals several other names that were or are still contributing.

Also, I am already a contributor here, both from previous and newly submitted code (though my attributions were stripped before merge (I don't mind BTW)).

CISC · 2026-03-12T09:28:43Z

@hksdpc255 The macros are a nice cleanup, though you are still modifying the object, you can test on Chat Template Editor.

hksdpc255 · 2026-03-12T10:01:41Z

@CISC The Chat Template Editor seems not stable enough to use. It always shows

Chat Template Error
'tool' is undefined

when I enable tools.

I'm currently using https://huggingface.co/spaces/huggingfacejs/chat-template-playground for debugging.

CISC · 2026-03-12T10:08:40Z

@CISC The Chat Template Editor seems not stable enough to use. It always shows
Chat Template Error
'tool' is undefined
when I enable tools.

That would be because you are not passing tool as an argument to your macros, so the error is correct. :)

I'm currently using https://huggingface.co/spaces/huggingfacejs/chat-template-playground for debugging.

I would not use this, it is not fully compliant, and will also overlook immutability issues.

For compatibility reasons. It can be removed when upstream fix the jinja render engine bugs.

CISC · 2026-03-12T10:23:37Z

models/templates/MiroThinker-compat.jinja

+{%- macro function_name() %}
+    {%- if tool.function is defined %}
+        {{- tool.function.name }}
+    {%- elif tool.name is defined and tool.description is defined %}
+        {{- tool.name }}
+    {%- endif %}
+{%- endmacro %}
+{%- macro function_description() %}
+    {%- if tool.function is defined %}
+        {{- tool.function.description }}
+    {%- elif tool.name is defined and tool.description is defined %}
+        {{- tool.description }}
+    {%- endif %}
+{%- endmacro %}
+{%- macro function_parameters() %}
+    {%- if tool.function is defined %}
+        {{- tool.function.parameters }}
+    {%- elif tool.name is defined and tool.description is defined %}
+        {{- tool.parameters }}
+    {%- endif %}
+{%- endmacro %}
+
+{%- macro render_tool(server_name) %}
+    {%- if tool.mt_visited is not defined %}
+        {%- if server_name != ns.last_server %}
+            {{- "\n## Server name: " + server_name + "\n" }}
+            {%- set ns.last_server = server_name %}
+        {%- endif %}
+        {{- "### Tool name: " + function_name() + "\n" }}
+        {{- "Description: " + function_description() + "\n" }}
+        {{- "Input JSON schema: " + (function_parameters() | tojson(ensure_ascii=False)) + "\n" }}
+        {{- "\n" }}
+    {%- endif %}
+{%- endmacro %}
+
+{%- macro render_tool_server() %}
+    {%- if (function_name().split('_sandbox') | length > 1) or function_name().startswith('run_') or (function_name().split('python') | length > 1) %}
+        {{- "tool-python" }}
+    {%- elif function_name().split('_search') | length > 1 %}
+        {{- "search_and_scrape_webpage" }}
+    {%- elif function_name() == 'scrape_and_extract_info' %}


Suggested change

{%- macro function_name() %}

{%- if tool.function is defined %}

{{- tool.function.name }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.name }}

{%- endif %}

{%- endmacro %}

{%- macro function_description() %}

{%- if tool.function is defined %}

{{- tool.function.description }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.description }}

{%- endif %}

{%- endmacro %}

{%- macro function_parameters() %}

{%- if tool.function is defined %}

{{- tool.function.parameters }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.parameters }}

{%- endif %}

{%- endmacro %}

{%- macro render_tool(server_name) %}

{%- if tool.mt_visited is not defined %}

{%- if server_name != ns.last_server %}

{{- "\n## Server name: " + server_name + "\n" }}

{%- set ns.last_server = server_name %}

{%- endif %}

{{- "### Tool name: " + function_name() + "\n" }}

{{- "Description: " + function_description() + "\n" }}

{{- "Input JSON schema: " + (function_parameters() | tojson(ensure_ascii=False)) + "\n" }}

{{- "\n" }}

{%- endif %}

{%- endmacro %}

{%- macro render_tool_server() %}

{%- if (function_name().split('_sandbox') | length > 1) or function_name().startswith('run_') or (function_name().split('python') | length > 1) %}

{{- "tool-python" }}

{%- elif function_name().split('_search') | length > 1 %}

{{- "search_and_scrape_webpage" }}

{%- elif function_name() == 'scrape_and_extract_info' %}

{%- macro function_name(tool) %}

{%- if tool.function is defined %}

{{- tool.function.name }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.name }}

{%- endif %}

{%- endmacro %}

{%- macro function_description(tool) %}

{%- if tool.function is defined %}

{{- tool.function.description }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.description }}

{%- endif %}

{%- endmacro %}

{%- macro function_parameters(tool) %}

{%- if tool.function is defined %}

{{- tool.function.parameters }}

{%- elif tool.name is defined and tool.description is defined %}

{{- tool.parameters }}

{%- endif %}

{%- endmacro %}

{%- macro render_tool(server_name, tool) %}

{%- if tool.mt_visited is not defined %}

{%- if server_name != ns.last_server %}

{{- "\n## Server name: " + server_name + "\n" }}

{%- set ns.last_server = server_name %}

{%- endif %}

{{- "### Tool name: " + function_name(tool) + "\n" }}

{{- "Description: " + function_description(tool) + "\n" }}

{{- "Input JSON schema: " + (function_parameters(tool) | tojson(ensure_ascii=False)) + "\n" }}

{{- "\n" }}

{%- endif %}

{%- endmacro %}

{%- macro render_tool_server(tool) %}

{%- if (function_name(tool).split('_sandbox') | length > 1) or function_name(tool).startswith('run_') or (function_name(tool).split('python') | length > 1) %}

{{- "tool-python" }}

{%- elif function_name(tool).split('_search') | length > 1 %}

{{- "search_and_scrape_webpage" }}

{%- elif function_name(tool) == 'scrape_and_extract_info' %}

And you need to fix all other calls similarly.

But why? Shouldn't a macro capture the context from where it is called?

No. https://jinja.palletsprojects.com/en/stable/templates/#macros

Additionally, for this macro:

{%- macro render_tool(server_name) %} {%- if tool.mt_visited is not defined %} {%- if server_name != ns.last_server %} {{- "\n## Server name: " + server_name + "\n" }} {%- set ns.last_server = server_name %} {%- endif %} {{- "### Tool name: " + function_name() + "\n" }} {{- "Description: " + function_description() + "\n" }} {{- "Input JSON schema: " + (function_parameters() | tojson(ensure_ascii=False)) + "\n" }} {{- "\n" }} {%- endif %} {%- endmacro %}

If I changed it to

{%- macro render_tool(server_name, tool, ns) %} {%- if tool.mt_visited is not defined %} {%- if server_name != ns.last_server %} {{- "\n## Server name: " + server_name + "\n" }} {%- set ns.last_server = server_name %} {%- endif %} {{- "### Tool name: " + function_name() + "\n" }} {{- "Description: " + function_description() + "\n" }} {{- "Input JSON schema: " + (function_parameters() | tojson(ensure_ascii=False)) + "\n" }} {{- "\n" }} {%- endif %} {%- endmacro %}

It seems {%- set ns.last_server = server_name %} will not affect the namespace outside of the macro. At least in llama.cpp.

ns is not within the scope of the macro (it's declared later).

Not just passed-in BTW, all arrays and dicts/objects (except namespace) are immutable.

Seems making it fully compatible with standard Jinja2 would require a huge refactor. I’d rather leave it as is for now unless it breaks again.

Yes, the way this template was written makes it harder, which is why I started by asking where it came from. :)

So why {%- set message.tool_calls = [] %} or {%- set message.tool_calls = None %} will cause crash in llama.cpp?

Not entirely sure, will require some debugging, but both should clearly be caught by your if checks.

hksdpc255 · 2026-03-12T10:46:27Z

MiroThinker-compat.jinja should works for now.

MiroThinker.jinja should works when upstream fix the bug. (Then we can safely remove MiroThinker-compat.jinja)

ikawrakow · 2026-03-13T06:07:28Z

What is the status here? Did the two of you agree?

hksdpc255 · 2026-03-13T06:23:08Z

I believe the version provided in the PR is good enough, though CISC may prefer the approach discussed here: discussion_r2923730444 Both of them is compatable with llama.cpp and ik_llama.cpp(with PR #1376 merged)

The main issue with the template is that it incorrectly assumes the provided tools and messages lists are mutable. In reality, this is only true in llama.cpp’s simplified implementation. Making it fully compatible with a standard Jinja engine would require a substantial refactor.

hksdpc255 · 2026-03-13T06:29:07Z

The version provided in discussion_r2923730444 has fewer violations of standard Jinja2. The version in the PR is more hacky, but it is completely separated from the part I would prefer not to modify. If MiroThinker’s developers release a new version of that section, I can simply copy and paste it without needing to rework my changes.

ikawrakow · 2026-03-13T17:18:38Z

@CISC

Now that has been merged, I can address your comments.

BTW, I'm not sure if you're just trolling or genuinely believe this, either way a bit strange as there is ample evidence to the contrary.

No, I'm not trolling, and I wouldn't know what evidence you are talking about. Obviously you can prove me wrong by adding a PR here.

@hksdpc255 for one contributed here before contributing PRs that got merged in llama.cpp. Also a quick scan of AUTHORS reveals several other names that were or are still contributing.

Also, I am already a contributor here, both from previous and newly submitted code (though my attributions were stripped before merge (I don't mind BTW)).

This project clearly states that it is a fork of llama.cpp and acknowledges all ggml/llama.cpp authors in its LICENSE file. Hence, when someone ports one of your contributions to ik_llama.cpp, you don't get acknowledged separately as ik_llama.cpp author as you are already acknowledged together with all other ggml/llama.cpp authors (and you have done nothing to deserve a separate acknowledgment). You become a real ik_llama.cpp author, acknowledged separately in the ik_llama.cpp AUTHORS file, when you make an actual original contribution to ik_llama.cpp, or you yourself port one of your llama.cpp contributions to ik_llama.cpp and submit a PR here that gets accepted. You have never done this, and doing so is exactly the thing that may get you in trouble with the other llama.cpp maintainers.

Hence, to use your own words, it is you who is trolling here.

CISC · 2026-03-13T19:35:34Z

You become a real ik_llama.cpp author, acknowledged separately in the ik_llama.cpp AUTHORS file, when you make an actual original contribution to ik_llama.cpp,

Well, well, well, talk about showing your true colors... I think you need to dial down that sarcasm a little and look a bit more carefully.

or you yourself port one of your llama.cpp contributions to ik_llama.cpp and submit a PR here that gets accepted. You have never done this, and doing so is exactly the thing that may get you in trouble with the other llama.cpp maintainers.

Sure, it is true that I have never submitted a PR here, but that is simply because I don't use ik_llama.cpp, so it's a bit out of my scope (and free time), I do try to be helpful on topics I know about though, either because it interests me in general, or because it is code I for some strange reason are quite familiar with, hmmm...

I'll repeat this again though; this "trouble" you keep talking about is a pure figment of your imagination.

Hence, to use your own words, it is you who is trolling here.

You sir are quite the comedian.

Improve compatibility with the new Jinja template engine

6804533

hksdpc255 changed the title ~~Improve compatibility with the new Jinja template engine~~ Improve MiroThinker chat template compatibility with the new Jinja template engine Mar 11, 2026

hksdpc255 marked this pull request as draft March 11, 2026 12:26

Refactor MiroThinker chat template using macros

1be0f74

Add MiroThinker-compat chat template

b1d74d1

For compatibility reasons. It can be removed when upstream fix the jinja render engine bugs.

CISC reviewed Mar 12, 2026

View reviewed changes

hksdpc255 marked this pull request as ready for review March 12, 2026 10:46

hksdpc255 mentioned this pull request Mar 12, 2026

Autoparser - complete refactoring of parser architecture #1376

Open

hksdpc255 marked this pull request as draft March 12, 2026 11:13

hksdpc255 marked this pull request as ready for review March 12, 2026 11:18

Add workaround for llama.cpp crashing

795986b

ikawrakow approved these changes Mar 13, 2026

View reviewed changes

ikawrakow merged commit 9b90fd3 into ikawrakow:main Mar 13, 2026

Conversation

hksdpc255 commented Mar 11, 2026

Uh oh!

firecoperana commented Mar 11, 2026

Uh oh!

hksdpc255 commented Mar 11, 2026

Uh oh!

firecoperana commented Mar 11, 2026

Uh oh!

CISC commented Mar 11, 2026

Uh oh!

hksdpc255 commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CISC commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hksdpc255 commented Mar 12, 2026

Uh oh!

ikawrakow commented Mar 12, 2026

Uh oh!

CISC commented Mar 12, 2026

Uh oh!

hksdpc255 commented Mar 12, 2026

Uh oh!

hksdpc255 commented Mar 12, 2026

Uh oh!

CISC commented Mar 12, 2026

Uh oh!

ikawrakow commented Mar 12, 2026

Uh oh!

CISC commented Mar 12, 2026

Uh oh!

CISC commented Mar 12, 2026

Uh oh!

hksdpc255 commented Mar 12, 2026

Uh oh!

CISC commented Mar 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hksdpc255 commented Mar 12, 2026

Uh oh!

ikawrakow commented Mar 13, 2026

Uh oh!

hksdpc255 commented Mar 13, 2026

Uh oh!

hksdpc255 commented Mar 13, 2026

Uh oh!

ikawrakow commented Mar 13, 2026

Uh oh!

CISC commented Mar 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hksdpc255 commented Mar 12, 2026 •

edited

Loading

CISC commented Mar 12, 2026 •

edited

Loading