[jbyczkow]

[Bugfix] Fix Qwen2-Audio ValueError on audio clips longer than 30s

Purpose

InputProcessingContext.call_hf_processor() sets
merged_kwargs.setdefault("truncation", False) so the dummy placeholder
text passed to the HF processor isn't cut short — vLLM needs the full
untruncated placeholder sequence to keep multi-modal token positions
aligned with the audio embeddings.

Qwen2AudioProcessor forwards that same truncation kwarg unmodified to
its Whisper-style feature extractor, whose own default is
truncation=True: it truncates/pads every audio clip to the model's
fixed 30s window (3000 mel frames). With truncation=False leaking in
from the text-side default, clips longer than 30s are no longer
truncated, so their extracted input_features keep their native (larger)
frame count. This later fails Qwen2AudioFeatureInputs' fixed-shape
validation:

ValueError: input_features dim[2] expected 3000, got 3495

which crashes the engine core and takes the whole server down
(EngineDeadError).

First introduced by 0b19ebc
("[MM] Simplify _apply_hf_processor_main (#53275)"), which added the
truncation=False default in call_hf_processor(). Not
--enforce-eager-specific — reproduces on both eager and
torch.compile paths, since the bug is in multi-modal preprocessing,
before model execution.

Fix

Qwen2AudioMultiModalProcessor._preprocess_hf_mm_data() now explicitly
sets truncation=True in the kwargs it returns, so it wins over
call_hf_processor()'s setdefault(). The dummy-text truncation this
default was originally protecting against is unaffected here — the
dummy text is tiny (<audio_bos><audio_token><audio_eos>) and never
actually gets truncated either way.

Test Plan

vllm serve Qwen/Qwen2-Audio-7B-Instruct --dtype bfloat16 --tensor-parallel-size 1 \
  --max-model-len 4096 --gpu-memory-utilization 0.85 \
  -cc '{"inductor_compile_config":{"benchmark_combo_kernel":false}}' \
  --port 8001 --trust-remote-code --enforce-eager --limit-mm-per-prompt '{"audio":1}'

Client (any audio clip longer than 30s triggers it; the snippet below
synthesizes one so it's fully reproducible without external data):

import base64, io, requests, numpy as np, soundfile as sf

sr = 16000
duration_s = 35  # > 30s Whisper window
audio = (0.1 * np.sin(2 * np.pi * 440 * np.arange(sr * duration_s) / sr)).astype(np.float32)

buf = io.BytesIO()
sf.write(buf, audio, sr, format="WAV")
b64 = base64.b64encode(buf.getvalue()).decode("ascii")

resp = requests.post(
    "http://localhost:8001/v1/chat/completions",
    json={
        "model": "Qwen/Qwen2-Audio-7B-Instruct",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Transcribe the audio."},
                {"type": "input_audio", "input_audio": {"data": b64, "format": "wav"}},
            ],
        }],
        "max_tokens": 64,
    },
)
print(resp.status_code, resp.text[:500])

Test Result

Before fix: server crashes on the first request with a >30s audio clip:

ValueError: input_features dim[2] expected 3000, got 3500
EngineDeadError: EngineCore encountered an issue.

All subsequent requests get 500 Internal Server Error, server has to
be restarted.

After fix: server processes the same request successfully, returns a
transcription with 200 OK. Confirmed on Intel BMG B70 (XPU) and NVIDIA A100-SXM4-80GB (CUDA).

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[jikunshang]

/ci run

[github-actions]

❌ This PR is 10 commits behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

[jikunshang]

/ci run

[github-actions]

✅ Triggered Buildkite CI #89238 for commit 8a5223a7b4ec.