Fix Gemma 4 MTP on llama-server (Windows)

[leotm]

Overview

Minimally fix Gemma 4 MTP models broken on llama-server.exe (Windows)

Fix: #24443

Additional information

Tested on

• Windows 11
• unsloth/gemma-4-31B-it-qat-GGUF
• local build (unsloth.ai/docs/models/gemma-4/qat#llama.cpp-guide adapted for Win11 CMD, yup - old skool)

NB: E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting) is still shown, which can be addressed separately

NB: I considered addressing a related TODO but this can be addressed separately also

NB: I've not tested other gemma-4 models such as older non-qat ones

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: yes - to understand code related to the issue and get MTP working locally

[eminence]

This model is broken on linux, too. But this patch doesn't seem to fix the issue for me:

❯ ./build/bin/llama-cli -hf unsloth/gemma-4-31B-it-qat-GGUF -fit off

Loading model... |0.00.559.547 Segmentation fault

(stacktrace is from the head commit in this PR)

Thread 1 "llama-cli" received signal SIGSEGV, Segmentation fault.
0x00007ffff5cf7df2 in llama_context::n_ctx (this=0x0) at /nas/achin/devel/llama.cpp/src/llama-context.cpp:719
719         return cparams.n_ctx;
(gdb) bt
#0  0x00007ffff5cf7df2 in llama_context::n_ctx (this=0x0) at /nas/achin/devel/llama.cpp/src/llama-context.cpp:719
#1  0x00007ffff5d025d9 in llama_n_ctx (ctx=0x0) at /nas/achin/devel/llama.cpp/src/llama-context.cpp:3491
#2  0x00007ffff7ebb75f in server_context_impl::load_model (this=0x5555556ba650, params=...) at /nas/achin/devel/llama.cpp/tools/server/server-context.cpp:905
#3  0x00007ffff7e9185a in server_context::load_model (this=0x7fffffffb6b0, params=...) at /nas/achin/devel/llama.cpp/tools/server/server-context.cpp:3566
#4  0x00007ffff7d803b6 in llama_cli (argc=5, argv=0x7fffffffddd8) at /nas/achin/devel/llama.cpp/tools/cli/cli.cpp:406
#5  0x0000555555556239 in main (argc=5, argv=0x7fffffffddd8) at /nas/achin/devel/llama.cpp/tools/cli/main.cpp:4

(gdb) print(cparams)
Cannot access memory at address 0x8

[CISC]

I don't understand what this is supposed to fix?

MTP models do not need to set t_embd.

[ServeurpersoCom]

-> To be tested under Windows
That seems strange to me too, I tested it under Linux and there's no reason it should be different!

[leotm]

weird, earlier today was trying gemma-4-31B-it-qat-GGUF (noting - Jun 9 Update: Added MTP support)
wondering why the token gen speed was slow NTP with recommended settings

set LLAMA_CACHE=C:\Users\leom\.cache\huggingface\hub
.unsloth\llama.cpp\build\bin\Release\llama-server.exe ^
    -hf unsloth/gemma-4-31B-it-qat-GGUF:UD-Q4_K_XL ^
    --temp 1.0 ^
    --top-k 64 ^
    --spec-type draft-mtp ^
    --spec-draft-n-max 4 ^
    -ngl 999 ^
    -fa on ^
    --cache-type-k q8_0 ^
    --cache-type-v q8_0 ^
    --ctx-size 131072

thinking the issue was the only error llama-server.exe displays (a red herring)
E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)

i've re-tested from scratch and MTP strangely seems to be working now via upstream unsloth\llama.cpp

C:\Users\leom>.unsloth\llama.cpp\build\bin\Release\llama-server.exe --version
version: 9585 (d73cd0767)
built with MSVC 19.44.35227.0 for Windows AMD64

thanks for testing this on Linux ^ i think this clarifies the change isn't needed and safe to close

(i'm unable to repro the original issue, which seems to be the model fails to load at all)

[eminence]

i still get a segfault with this unsloth model. i'll open a new issue