MTMD: Fix Qwen3-tts-0.6b

[ServeurpersoCom]

Overview

Fixes Qwen3-TTS-12Hz-0.6B: the loader no longer requires the code predictor input projection, which that checkpoint does not have, and the predictor ffn_down stays in F32 since its FFN intermediate peaks above the F16 range.

The fix spans both sides, C++ and conversion script. The C++ half only needs a rebuild, but the F32 rule lives in the conversion script, so please regenerate the mmproj GGUF files.

Additional information

Qwen3-TTS-12Hz-0.6B failed in two different ways and this fixes both. The mmproj refused to load because the loader required the code predictor input projection, which the reference only builds when the talker and the predictor differ in hidden size, so the 0.6B checkpoint simply has no such tensor. With that out of the way the model then died on the first frame: its code predictor carries a massive activation, a spike about twice what F16 can represent, and since mul_mat casts its input to the weight type an F16 ffn_down turned that spike into infinity, then NaN, then an out of bounds gather. Keeping those five weights in F32 costs roughly 31 MB and makes the model run on CPU and CUDA again. The 1.7B is unaffected either way, it has more than six times the headroom under the F16 ceiling.

Existing mmproj GGUF files need to be regenerated: the fix lives in the conversion script, so a GGUF converted before this change still carries the F16 ffn_down and will still break on the 0.6B. The rule applies to every checkpoint in the family, so regenerate the 1.7B GGUF files as well and keep the published set on one consistent format.

Voice cloning stays speaker-embedding only for now: the helper exposes no reference transcript and builds only the speaker-slot prompt, and more fundamentally the conversion script packs just the decoder half of the speech tokenizer, so there is no codec encoder to turn a reference clip into the codes the in-context cloning mode feeds on.

Follow-up #26254

Testing

Short sample: "the quick brown fox jumped over the lazy dog".

qwen3tts-0.6B-clone-short.wav
qwen3tts-0.6B-tts-short.wav
qwen3tts-1.7B-clone-short.wav
qwen3tts-1.7B-tts-short.wav

Long sample: "Here is what went wrong with the small model. Its code predictor carries one enormous activation, a single spike about twice the largest value a half precision float can hold. When the weights are stored in that format, the multiplication turns the spike into infinity, the next normalization turns infinity into a not a number, and the whole thing collapses into an out of bounds memory read. The fix keeps five small weight tensors in full precision. It costs about thirty megabytes, and it lets the model run everywhere again."

qwen3tts-0.6B-clone-long.wav
qwen3tts-0.6B-tts-long.wav
qwen3tts-1.7B-clone-long.wav
qwen3tts-1.7B-tts-long.wav

The plain TTS clips use the model's own default voice, the cloned ones are conditioned on a 17 s reference recording of a real speaker.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES

[ServeurpersoCom]

I have another small PR to submit for the Vulkan crash: the gen graph hands GET_ROWS an index that is a view at a few bytes offset, which the Vulkan path rejects, and materializing those three index rows fixes it with bit for bit identical output on CPU and CUDA.