Skip to content

[BUG FIX]: prevent EngineCore crash when Qwen TTS Base task is missing ref_text#2203

Merged
linyueqian merged 1 commit into
vllm-project:mainfrom
teith:fix/qwen3-tts-missing-ref-text-crash
Apr 11, 2026
Merged

[BUG FIX]: prevent EngineCore crash when Qwen TTS Base task is missing ref_text#2203
linyueqian merged 1 commit into
vllm-project:mainfrom
teith:fix/qwen3-tts-missing-ref-text-crash

Conversation

@teith
Copy link
Copy Markdown
Contributor

@teith teith commented Mar 26, 2026

A single malformed request to /v1/audio/speech with task_type=Base and no ref_text kills the entire EngineCore. All subsequent requests — including valid ones — fail with EngineDeadError.

Reproduce:
serve:
vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-Base --omni --trust-remote-code --host 0.0.0.0 --port 8000 --enforce-eager

from openai import OpenAI
client = OpenAI(api_key="none", base_url="http://localhost:8000/v1")
client.audio.speech.create(
    model="Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    voice="clone",
    input="Hello",
    extra_body={
        "task_type": "Base",
        "ref_audio": "https://cdn-media.huggingface.co/speech_samples/sample1.flac",
    },
)
# Engine is now dead. Every subsequent request returns EngineDeadError.

Root cause: _build_prompt_embeds() raises ValueError when ref_text is missing in ICL mode. vLLM v1 treats any worker exception as fatal.

Fix:
serving_speech.py: validate ref_text presence for Base task before the request reaches the engine (returns HTTP 400)
qwen3_tts_talker.py: fall back to x-vector-only mode instead of raising, matching the existing non-ICL codepath (protects offline inference and other entrypoints)

@teith teith requested a review from hsliuustc0106 as a code owner March 26, 2026 00:39
@yenuo26
Copy link
Copy Markdown
Collaborator

yenuo26 commented Mar 26, 2026

Do we need to add test cases for this scenario? @linyueqian

@linyueqian
Copy link
Copy Markdown
Collaborator

Do we need to add test cases for this scenario? @linyueqian

yes, it would be great.

Copy link
Copy Markdown
Collaborator

@linyueqian linyueqian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@linyueqian linyueqian added the ready label to trigger buildkite CI label Mar 26, 2026
@linyueqian
Copy link
Copy Markdown
Collaborator

fix pre-commit please

@linyueqian
Copy link
Copy Markdown
Collaborator

fix dco please

@teith teith force-pushed the fix/qwen3-tts-missing-ref-text-crash branch 2 times, most recently from 152f1b6 to 1316136 Compare March 27, 2026 03:43
Copy link
Copy Markdown
Collaborator

@lishunyang12 lishunyang12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a minor comment, otherwise looks good

Comment thread vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_talker.py
@hsliuustc0106
Copy link
Copy Markdown
Collaborator

fix ci

linyueqian added a commit to teith/vllm-omni that referenced this pull request Apr 11, 2026
Parametrized test covering ref_text=None, "", and whitespace-only
to ensure the serving layer returns HTTP 400 instead of letting a
malformed request crash the EngineCore.

Regression test for vllm-project#2203

Signed-off-by: Yueqian Lin <yueqian.lin@outlook.com>
@linyueqian linyueqian force-pushed the fix/qwen3-tts-missing-ref-text-crash branch 2 times, most recently from 2eabe77 to 85325d8 Compare April 11, 2026 02:16
… TTS Base task

A single malformed request to /v1/audio/speech with task_type=Base
and no ref_text kills the entire EngineCore. All subsequent requests
fail with EngineDeadError.

Root cause: _build_prompt_embeds() raises ValueError when ref_text is
missing in ICL mode. vLLM v1 treats any worker exception as fatal.

Fix:
- serving_speech.py: validate ref_text presence for Base task before
  the request reaches the engine (returns HTTP 400)
- qwen3_tts_talker.py: fall back to x-vector-only mode instead of
  raising, matching the existing non-ICL codepath

Also adds a parametrized regression test covering ref_text=None, "",
and whitespace-only inputs.

Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
@linyueqian linyueqian force-pushed the fix/qwen3-tts-missing-ref-text-crash branch from 85325d8 to a0cd54c Compare April 11, 2026 02:17
@linyueqian linyueqian enabled auto-merge (squash) April 11, 2026 02:17
@linyueqian linyueqian merged commit 001f2e3 into vllm-project:main Apr 11, 2026
7 of 8 checks passed
daixinning pushed a commit to daixinning/vllm-omni that referenced this pull request Apr 13, 2026
…g ref_text (vllm-project#2203)

Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
lengrongfu pushed a commit to lengrongfu/vllm-omni that referenced this pull request May 1, 2026
…g ref_text (vllm-project#2203)

Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
…g ref_text (vllm-project#2203)

Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants