Add Qwen3.5 MoE

[Goekdeniz-Guelmez]

No description provided.

[Goekdeniz-Guelmez]

looks to me that its juts Qwen3 Next with vision PR

[johnmai-dev]

looks to me that its juts Qwen3 Next with vision PR

Although they are largely similar, there are still some subtle differences.

Qwen3_5MoeGatedDeltaNet

https://github.com/bozheng-hit/transformers/blob/be63a26077e10243d3fd8eb8cc074ce314feb04f/src/transformers/models/qwen3_5_moe/modeling_qwen3_5_moe.py#L446-L627

Qwen3NextGatedDeltaNet

https://github.com/huggingface/transformers/blob/7769f660935b5d48b73bf6711d0a78b6f8f98739/src/transformers/models/qwen3_next/modeling_qwen3_next.py#L590-L801

[Goekdeniz-Guelmez]

thanks for that! will get updated.

[johnmai-dev]

thanks for that! will get updated.

You might be able to directly reference this: mlx_lm/models/qwen3_5_text.py → Qwen3_5GatedDeltaNet.

https://github.com/johnmai-dev/mlx-lm/blob/728b8713f6bf73099f04c0091ad1b180dd3231bf/mlx_lm/models/qwen3_5_text.py#L56-L159

#861

[awni]

Closing this in favor of #869.

Thanks for adding this @Goekdeniz-Guelmez I will add you as a co-author on the other PR.

[Goekdeniz-Guelmez]

@awni totally!