Add arch support for cohere2-MoE - #24260

Merged

pwilkin merged 21 commits into

ggml-org:masterfrom

michaelw9999:cohere2-moe

Jun 13, 2026

Commits on Jun 7, 2026

Add arch support for cohere2-MoE
michaelw9999
committed
d03b03b
Removed redundant gating_func checks
michaelw9999
committed
14581b8
Changed ffn lookup to prefer prefix_dense_intermediate_size
michaelw9999
committed
b5a360a
Renamed arch to cohere2moe
michaelw9999
committed
df2bd77
Removed redundant lmhead check and chat template changes
michaelw9999
committed
c4b4436
Removed lm_head.weight check from modify tensors, load output tensor not required, fallback to token_embd.weight
michaelw9999
committed
04d20e7
Changed to (routed+shared)*0.5 for shared expert combined avg
michaelw9999
committed
7fdcf06
fixed sliding_window_pattern issue and pattern
michaelw9999
committed
c8e57a2
Fixed transformers crash 'first_k_dense_replace' error
michaelw9999
committed
f5a68ed
Remove comment
michaelw9999
committed
ac49f84

Commits on Jun 10, 2026

Commits on Jun 11, 2026

Fixed remaining todos: cohere2moe renamed, changed swa parsing to use get_key_or_arr, removed extra get_arr use
michaelw9999
committed
2720615
Force metadata usage

michaelw9999
and
CISC
authored
87f2fbb
Remove Cohere2 checkpoint comment

michaelw9999
and
CISC
authored
fc2dbeb
Remove MTP comment

michaelw9999
and
CISC
authored
691fc6c
Regenerate cohere2moe tokenizer hash
michaelw9999
committed
cebf758
Add cohere2moe to Llama Model Saver supported list
michaelw9999
committed
12692d6

Commits on Jun 12, 2026

Check for zerobios tensors and add support for Command to use LayerNorm
michaelw9999
committed
00487aa
Map expert_selection_fn to sigmoid in base.py instead of command.py
michaelw9999
committed
9c38245
use bools for foundnorm/foundnormrms

michaelw9999
and
CISC
authored
d932047