models : dedup qwen35 graphs

[ggerganov]

cont #19597

Use the new struct llm_build_delta_net_base to deduplicate the delta net graphs from Qwen35 models.

TODO:

• Before merging, check that this does not break something after the Qwen 3.5 models are released
• Try to avoid the explicit repeat of Q and K (again, requires the actual models for testing)

[JamePeng]

The model is now online.
https://huggingface.co/Qwen/Qwen3.5-397B-A17B

[ggerganov]

Unless Qwen releases smaller models, I won't be able to test this. So if anyone gives it a try would appreciate feedback.

[ghost]

I tested this with these two models:
https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF/tree/main/UD-Q4_K_XL
https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF/tree/main/MXFP4_MOE

The model works on afa6bfe (current master).
It crashes on cdd792c (this PR).
It works on cc45f2a (parent of cdd792c).

It crashes at this assert: src\models\delta-net-base.cpp:279: GGML_ASSERT(s->ne[0] == S_v && s->ne[1] == S_v && s->ne[2] == H_v && s->ne[3] == n_seqs) failed
The values in the assert are: s->ne[0] = 128 s->ne[1] = 8192 s->ne[2] = 1 s->ne[3] = 4 S_v = 128 H_v = 64 n_seqs = 4

[ggerganov]

@matbrez Thank you for the feedback. Could you try again the latest version of this branch and confirm that it works?

[bunnynode]

@ggerganov The model can load success with recent commit. But generate repeat 0 tokens.

./llama-server
-m /models/unsloth/Qwen3.5-397B-A17B-GGUF/MXFP4_MOE/Qwen3.5-397B-A17B-MXFP4_MOE-00001-of-00006.gguf
--jinja
--alias "qwen35"
--batch-size 2048
--ubatch-size 2048
--cache-type-k f16
--cache-type-v f16
-ngl 999 -fa on --temp 0.7 --top-k 20 --top-p 0.8 --min-p 0 --presence-penalty 1.0 -c 128000 -n 128000 --no-context-shift
--host 0.0.0.0
--port 8076
--offline
--chat-template-kwargs "{"enable_thinking": false}"

git show --stat

commit 3f6ca46 (HEAD -> pr-19660, origin/gg/qwen35-dedup)
Date: Mon Feb 16 09:51:21 2026 +0200

models : dedup qwen35 graphs

src/models/models.h | 58 +---------------------
src/models/qwen35.cpp | 454 ++++++++++++++++++--------------------------------------------------------------------------------------------------------------------------------------------------------
src/models/qwen35moe.cpp | 455 ++++++++++++++++++---------------------------------------------------------------------------------------------------------------------------------------------------------
3 files changed, 96 insertions(+), 871 deletions(-)

[ggerganov]

@bunnynode And to make sure - the master works ok, correct?

[ggerganov]

Please try with 1f5a7e4

[bunnynode]

@ggerganov It works now with 1f5a7e4, on m3 ultra, the tg improved from 19.8t/s(master) to 21.7/s(this pr)

But far slow than mlx-lm (34t/s) , both q4

[CISC]

Very nice.