[Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder

[ZhanqiuHu]

The speech tokenizer in Qwen3TTSCode2Wav was loaded in bfloat16, causing NaN overflow in the SnakeBeta activation's torch.exp() calls. This produced all-silence PCM output (-32768 for every sample).

Change torch_dtype from bfloat16 to float32 to match the upstream Qwen3-TTS tokenizer config (encoder_config.dtype: "float32") and official examples which default to float32.

[Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder

Purpose

• Fixes [Bug]: Qwen3-TTS voice clone produces all-silence audio (bfloat16 NaN in speech tokenizer) #1663 — speech tokenizer loaded in bfloat16 causes NaN overflow in SnakeBeta activation, producing all-silence PCM output.
• One-line fix: torch_dtype=torch.bfloat16 → torch_dtype=torch.float32 in qwen3_tts_code2wav.py.
• Matches upstream tokenizer config (encoder_config.dtype: "float32" in HuggingFace config.json) and official example.

Test Plan

Serve:

vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-Base \
  --omni --port 8201 \
  --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
  --trust-remote-code --enforce-eager --host 0.0.0.0

Request:

curl -s http://localhost:8201/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    "input": "你好，世界。",
    "task_type": "Base",
    "ref_text": "甚至出现交易几乎停滞的情况。",
    "ref_audio": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone_1.wav",
    "response_format": "pcm"
  }' -o /tmp/tts_output.pcm

Verify:

import struct
data = open("/tmp/tts_output.pcm", "rb").read()
samples = struct.unpack(f"<{len(data)//2}h", data)
print(f"Total: {len(samples)}, Min: {min(samples)}, Max: {max(samples)}, Unique: {len(set(samples))}")

Test Result

A/B tested 8 runs each on NVIDIA B200 with Qwen3-TTS-12Hz-1.7B-Base. Before the fix (bfloat16), all 8 runs produced all-silence PCM (Min: -32768, Max: -32768, Unique: 1). After the fix (float32), all 8 runs produced valid audio with diverse samples. See #1663 for full details.

• Serve Qwen3-TTS-12Hz-1.7B-Base, send Base voice-clone request
• Verify output PCM contains real audio (not all -32768)
• A/B tested 8/8 runs: 100% silence with bf16, 100% correct with fp32

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
• The test results. Please paste the results comparison before and after, or the e2e results.
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
• (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[linyueqian]

@hsliuustc0106 @Gaohan123 I have hit the same silence issue with certain inputs. The fix looks correct to me. SnakeBeta uses torch.exp() which overflows in bf16 for some codebook values. The official HF examples also use bf16 and likely have the same bug. Their standalone tokenizer example defaults to fp32 and works fine.

[hsliuustc0106]

@hsliuustc0106 @Gaohan123 I have hit the same silence issue with certain inputs. The fix looks correct to me. SnakeBeta uses torch.exp() which overflows in bf16 for some codebook values. The official HF examples also use bf16 and likely have the same bug. Their standalone tokenizer example defaults to fp32 and works fine.

@linyueqian can we add some of the tests into our CI?

[linyueqian]

@hsliuustc0106 @Gaohan123 I have hit the same silence issue with certain inputs. The fix looks correct to me. SnakeBeta uses torch.exp() which overflows in bf16 for some codebook values. The official HF examples also use bf16 and likely have the same bug. Their standalone tokenizer example defaults to fp32 and works fine.

@linyueqian can we add some of the tests into our CI?

Agree. @ZhanqiuHu Could you add a CI test for this fix? Serve the model, send the voice clone request using this exact input, and use whisper to verify the output is real audio (not silence). You can mirror the pattern in tests/examples/online_serving/test_qwen3_omni.py which uses convert_audio_file_to_text from tests/conftest.py.

[ZhanqiuHu]

Sounds good👍 I’ll take a look and add CI test later.

[Gaohan123]

Does the fix affect inference speed? As it hardcoded code2wav to FP32

[ZhanqiuHu]

@linyueqian
Added E2E regression tests in tests/e2e/online_serving/test_qwen3_tts_base.py:

1. test_base_voice_clone_not_silence: Sends a Base voice-clone request with PCM format, verifies samples are not all-silence.
2. test_base_voice_clone_whisper_transcription: Sends a Base voice-clone request, transcribes the output with Whisper, and asserts cosine similarity > 0.9 against the expected text (mirrors the pattern in test_qwen3_omni.py)
3. test_base_voice_clone_wav_format: Verifies WAV headers, content-type, and minimum audio size

Test inputs (ref_audio, ref_text, synthesis text) are from the official offline inference example in examples/offline_inference/qwen3_tts/end2end.py (get_base_query).

[hsliuustc0106]

any perf regression from this change?

[ZhanqiuHu]

any perf regression from this change?

I think if the NaN issue is common (which it always occurs in my case), then we should probably use float32 even if it gives overhead. We could also do some accuracy tests on standard benchmarks/datasets to further confirm (if you have any suggestions).

I expected the tokenizer decoder to be relatively lightweight, and here's some perf numbers:

codes = torch.randint(0, 2048, (1, 50, 16), device="cuda")  # 50 frames, 16 quantizers
tok = Qwen3TTSTokenizer.from_pretrained(speech_dir, torch_dtype=dtype, load_feature_extractor=False)
tok.model.to("cuda")
tok.model.decode(audio_codes=codes)

dtype	mean	std	min	max
float32	10.70 ms	0.04 ms	10.66 ms	11.09 ms
bfloat16	10.42 ms	0.04 ms	10.36 ms	10.74 ms

(100 reps, 10 warmup, B200)

[hsliuustc0106]

lgtm