server : disable embeddings/pooling on the speculative draft/MTP context

[liminfei-amd]

Overview

MTP (and speculative draft) models fail to load in llama-server while the same
model loads and runs in llama-cli (#24443). The server aborts in load_model
when creating the speculative context.

Root cause

When the server builds the speculative draft context and the MTP context,
it reuses the target context params. Those carry embeddings = true and a
pooling type. The draft/MTP graphs only produce draft logits and have no
embeddings output, so creating their context with embeddings/pooling enabled
fails. llama-cli does not copy these target-side params into the speculative
context, which is why it is unaffected. (This matches the maintainer note on
#24480 that MTP contexts do not need to set the embeddings tensor.)

Fix

In tools/server/server-context.cpp, before llama_init_from_model() for the
draft context and for the MTP context, set:

cparams.embeddings   = false;
cparams.pooling_type = LLAMA_POOLING_TYPE_NONE;

so the speculative context is created for logits only. The target context is
unchanged.

Validation

On an RDNA3.5 (gfx1151) ROCm/Vulkan build:

• cli: loads & runs the MTP model (--spec-type draft-mtp) — unaffected.
• server, unpatched: aborts in load_model (reproduced; gdb stack bottoms at
  server.cpp ctx_server.load_model).
• server, patched: loads, and serves MTP speculative decoding end to end
  (draft-mtp: #gen drafts = 15, #acc drafts = 5, #acc tokens = 8, mean acc len = 1.53), slots release cleanly. Output matches the cli run.
• git apply --check clean on current master.

Scope / related

#24480 (closed, unmerged) addressed a Gemma4-model-specific facet in
src/models/gemma4-assistant.cpp; this PR is the server-side context-params fix
and is independent of it. The issue thread also mentions hardware-specific
variants; this PR fixes the embeddings/pooling load failure that reproduces on
the speculative draft/MTP path, which is the common failure mode in the report.

Requirements

• I have read the contributing guidelines.
• AI usage disclosure: this change was prepared with AI assistance; a human
  reviewed and verified it and can explain every line.

[ggml-gh-bot]

Hi @liminfei-amd, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 5 open PRs.

• AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[AndASM]

60bc886 appears to cause a new break that blocks this fix. With that commit reverted this fix works for me.

That's this pull I think #24980

[liminfei-amd]

Thanks for testing and for pinpointing #24980 @AndASM. I've rebased this onto current master.

Re-verified on master (b3fed31), gfx1151: with this patch llama-server loads the MTP target+draft (gemma-4-12b) and serves completions, no abort. The Gemma4Assistant requires ctx_other to be set line now shows up only as a "normal during memory fitting" warning and the server continues, so the embeddings/pooling fix here is still needed and sufficient to load. If you still see a hard break from #24980 on your setup, a repro would help.

[TheBit]

@liminfei-amd Hi, and thanx for the patch! Can you please take a look at conflicts? I got update today in Unsloth Studio and lost MTP for Qwen3.6-35B on my M5 with warning:

MTP is not available in the installed llama.cpp build, so this model is running without it.

[liminfei-amd]

Hi @TheBit, the conflicts are resolved and the updated fix passes target+MTP testing on gfx1151 and gfx1201.

The Unsloth warning appears unrelated because this PR is not merged. Could you share the Studio version and its bundled llama-server --version?

[TheBit]

Hi @TheBit, the conflicts are resolved and the updated fix passes target+MTP testing on gfx1151 and gfx1201.

The Unsloth warning appears unrelated because this PR is not merged. Could you share the Studio version and its bundled llama-server --version?

@liminfei-amd just opened up my Unsloth, it asked to update llama.cpp, and after it the bug is gone, so yeah, it probably was another issue. 130 TPS (M5, 128), back to normal:

Unsloth Version: v0.1.49-beta (didn't updated Unsloth itself)
Package Version: 2026.7.3
llama.cpp Version: b10043-mix-0ac9dfb (this is what fixed the problem)
Backend: MLX
Python: 3.13.13
Torch: 2.10.0
Transformers: 4.57.6

This is what was before: 36 TPS with that warning:

[sheldonrobinson]

Are there any plans to merge this patch?

[liminfei-amd]

Closing this PR because I can no longer reproduce the reported server failure on current master. In a same-commit baseline/head comparison using the exact Gemma 4 target, MTP draft, and mmproj, both versions initialized speculative decoding and produced the same output and draft counts. I could not isolate a single replacement commit, so this PR is no longer needed; this does not claim that issue #24443 is resolved.