[Bugfix] Fix cross-request codes.ref leak in Qwen3-TTS make_omni_output

[henryj]

Purpose

Fix cross-request voice leakage in Qwen3-TTS voice clone under concurrency.

make_omni_output collapses every request's codes["ref"] into a single last-writer-wins variable and emits it as a length-1 list. to_payload_element indexes list payloads with element[idx] if idx < len(element) else element[0], so that list is broadcast to every request in the batch: each utterance's first vocoder chunks decode with another request's reference voice as context. Audible result: onset timbre deformation converging over ~1-2 s on every voice-clone utterance at any concurrency > 1; correct at concurrency 1. Silent (no warning, no error).

This PR keeps codes.ref batch-aligned: one entry per request, with an empty placeholder for requests that carry no ref (the downstream numel() > 0 checks already treat an empty tensor as "no ref").

Fixes #4370 (full diagnosis, repro script, and forensics there). Same defect family as #4355, which proposes hardening the element[0] fallback itself.

Test Plan

vLLM Version: 0.22.0

vLLM-Omni Commit: 3fe3aa3 (main)

• New regression test tests/model_executor/models/qwen3_tts/test_qwen3_tts_talker_ref_codes.py: CPU-only, no weights, constructs the talker via __new__ like the existing preprocess tests. Covers per-request refs, the no-ref placeholder, and the no-refs-at-all case.
• Production-scale verification of the same change applied to vllm-omni 0.22.0 site-packages: 278 audiobook segments, 6 registered cloned voices, concurrency 16, one RTX 4090, OpenAI-compatible endpoint.

Test Result

• Regression test assertions verified against the real method body pre/post patch (standalone harness; this dev box cannot install vllm, CI will run the pytest file): on main, codes.ref comes back as a length-1 list holding the last request's ref for every request, including requests that have no ref of their own; on this branch, each request gets its own entry. ruff check and ruff format --check pass on both changed files.
• Production workload (0.22.0 + this change): all 278 utterance onsets clean by ear (headphones A/B), output indistinguishable from concurrency-1 renders and from the official qwen-tts package decoding the same segments on the same GPU. Throughput unchanged within noise (22.4x vs 22.8x realtime at concurrency 16).

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

Reviewed PR #4373: [Bugfix] Fix cross-request codes.ref leak in Qwen3-TTS make_omni_output

[linyueqian]

LGTM. I verified both the defect and the fix independently rather than just reading the diff.

Root cause confirmed: make_omni_output collapsed every request's ref into a single last-writer-wins variable and emitted it as a length-1 list, and to_payload_element broadcasts element[0] to any request index past the end of the list. The prefix cache path has the same fallback in _unwrap_lists, so both paths leaked. With the batch-aligned list, element[idx] picks the right request on both paths, and the ordering assumption is the same one the audio codes concatenation already relies on.

I also checked the two subtle cases:

1. The empty placeholder is safe at all three consumers (_normalize_ref_code, the async-chunk path, and the orchestrator path in stage_input_processors/qwen3_tts.py): each guards on numel().
2. Cross-step accumulation in full-payload mode: the ref is written into the request's persistent intermediate buffer at first prefill, so a request that has a ref re-emits it every step and the placeholder can never clobber a real ref in accumulate_full_payload_output. The placeholder only ever applies to requests that genuinely carry no ref, which is exactly the case main got wrong.

Test results (H20-class node, vllm 0.22.0; these are CPU-only tests):

• New regression test against the PR base without the fix: 2/3 fail, leak reproduces exactly as described.
• On this branch: 3/3 pass, and the full tests/model_executor/models/qwen3_tts/ directory passes (114/114).
• Cherry-picked onto current main (9577ccb): applies cleanly, no new failures.

One note for CI triage: current main already fails 10 tests in test_qwen3_tts_talker_preprocess.py with AttributeError: '_embedding_dtype', introduced by #4283 (those tests build the talker via __new__ and skip __init__). If Buildkite comes back red on that file after the ready label is added, it is pre-existing and not this PR.

[linyueqian]

[minor] The placeholder is 1-D (0,) while real refs are 2-D (N, num_code_groups). Every current consumer only checks numel(), so this works today, but a (0, num_code_groups) placeholder would be shape-consistent if a downstream ever concatenates or reshapes these. Fine to leave as is.

[hsliuustc0106]

Could pre-size this to len(info_dicts) and assign by index rather than appending — len(ref_code_list) == batch size then holds by construction, and both append(torch.empty(0)) sites further down drop out:

ref_code_list: list[torch.Tensor] = [
    torch.empty(0, dtype=torch.long) for _ in info_dicts
]
...
for i, info in enumerate(info_dicts):
    if not isinstance(info, dict):
        continue
    ...
    if isinstance(ref_code, torch.Tensor) and ref_code.numel() > 0:
        ref_code_list[i] = ref_code
        has_ref_code = True

Minor — but it means a future continue added to this loop can't silently desync the list from the batch.