Skip to content

[bugfix] fix audio empty bug for non-async-chunk mode - #4792

Closed
yinpeiqi wants to merge 1 commit into
vllm-project:mainfrom
yinpeiqi:fix-ci
Closed

[bugfix] fix audio empty bug for non-async-chunk mode#4792
yinpeiqi wants to merge 1 commit into
vllm-project:mainfrom
yinpeiqi:fix-ci

Conversation

@yinpeiqi

@yinpeiqi yinpeiqi commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

PLEASE FILL IN THE PR DESCRIPTION HERE.

Fix the audio empty bug raised in https://buildkite.com/vllm/vllm-omni/builds/11897/canvas?sid=019f1495-91f0-4941-b501-6e2b7a2c4310&tab=output

Go back to the previous path that hidden states will go through both inter_stage_output and multimodal_output.

============================= test session starts ==============================
platform linux -- Python 3.12.13, pytest-9.1.0, pluggy-1.6.0 -- /workspace/pqyin/vllm-omni-pr3569/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /workspace/pqyin/vllm-omni-pr3569
configfile: pyproject.toml
plugins: asyncio-1.4.0, mock-3.15.1, typeguard-4.5.2, anyio-4.13.0, hydra-core-1.3.3, cov-7.1.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 1 item

tests/worker/test_gpu_ar_model_runner.py::test_build_omni_output_non_async_chunk_accumulates_full_payload PASSED [100%]

=============================== warnings summary ===============================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: 14 warnings
  /workspace/pqyin/vllm-omni-pr3569/.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

vllm_omni/version.py:55
  /workspace/pqyin/vllm-omni-pr3569/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
   --> vLLM-Omni version 0.22.1.dev113+g8fae1b0bf.d20260615
   --> vLLM version 0.23.0
  This will likely cause compatibility issues.
    warn_if_misaligned_vllm_version()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
--- Running Summary
======================== 1 passed, 17 warnings in 0.44s ========================

Purpose

Test Plan

vLLM Version:

vLLM-Omni Commit:

Test Result

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)

@yinpeiqi
fix ci
ec49df7 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@yinpeiqi

yinpeiqi commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@amy-why-3459 @fake0fan @hsliuustc0106

tzhouam added a commit that referenced this pull request Jun 30, 2026
@tzhouam @claude
[BugFix][Core] Align no-async-chunk fix with PR #4792 + cover NPU run…
10c80e3 
…ners

Converge the connector-starvation fix (d229272) with the canonical PR #4792
("fix audio empty bug for non-async-chunk mode"):

- gpu_ar / gpu_generation runners: instead of conditioning the accumulate on
  pooler_output while leaving pooler_inter=None, set the no-async-chunk branch
  to (pooler_output, pooler_output) / (per_req_payloads, per_req_payloads).
  This feeds the SAME full payload to accumulate_full_payload_output AND now
  also populates the inter_stage_outputs field uniformly, matching #4792.
- npu_ar / npu_generation runners: apply the same one-liner. These were the gap
  in d229272 -- the NPU runners have no accumulate_full_payload_output call, so
  inter_stage_outputs is their only inter-stage transport; with (None, ...) they
  starved in no-async-chunk mode exactly like the GPU path.
- tests/worker/test_gpu_ar_model_runner.py: inter_stage_outputs is now non-None
  in no-async-chunk and mirrors multimodal_outputs (matches #4792's test update).
- Revert the partition non-lossy change from d229272: with the no-async-chunk
  branch no longer routing through partition_flat_payload, that change only
  affected async_chunk (a path that was NOT failing) and diverged from #4792, so
  drop it to match upstream exactly.

GPU-equivalent to the verified d229272 behavior: accumulate still receives the
full payload (Qwen3-TTS + Qwen3-Omni-30B realtime e2e both passed under it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
tzhouam added a commit that referenced this pull request Jun 30, 2026
@tzhouam @claude
[BugFix][Core] Align no-async-chunk fix with PR #4792 + cover NPU run…
044dfee 
…ners

Converge the connector-starvation fix (d229272) with the canonical PR #4792
("fix audio empty bug for non-async-chunk mode"):

- gpu_ar / gpu_generation runners: instead of conditioning the accumulate on
  pooler_output while leaving pooler_inter=None, set the no-async-chunk branch
  to (pooler_output, pooler_output) / (per_req_payloads, per_req_payloads).
  This feeds the SAME full payload to accumulate_full_payload_output AND now
  also populates the inter_stage_outputs field uniformly, matching #4792.
- npu_ar / npu_generation runners: apply the same one-liner. These were the gap
  in d229272 -- the NPU runners have no accumulate_full_payload_output call, so
  inter_stage_outputs is their only inter-stage transport; with (None, ...) they
  starved in no-async-chunk mode exactly like the GPU path.
- tests/worker/test_gpu_ar_model_runner.py: inter_stage_outputs is now non-None
  in no-async-chunk and mirrors multimodal_outputs (matches #4792's test update).
- Revert the partition non-lossy change from d229272: with the no-async-chunk
  branch no longer routing through partition_flat_payload, that change only
  affected async_chunk (a path that was NOT failing) and diverged from #4792, so
  drop it to match upstream exactly.

GPU-equivalent to the verified d229272 behavior: accumulate still receives the
full payload (Qwen3-TTS + Qwen3-Omni-30B realtime e2e both passed under it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
@amy-why-3459 amy-why-3459 mentioned this pull request Jun 30, 2026
Closed
@hsliuustc0106

hsliuustc0106 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

fix conflicts

@hsliuustc0106 hsliuustc0106 added the bug Something isn't working label Jun 30, 2026
@yinpeiqi

yinpeiqi commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

seems this bug already fixed?

@linyueqian linyueqian mentioned this pull request Jul 24, 2026
Merged
@Gaohan123 Gaohan123 mentioned this pull request Sep 22, 2026
Closed
1 task done
@vllm-omni-review-bot vllm-omni-review-bot mentioned this pull request Oct 10, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gcanlin gcanlin Awaiting requested review from gcanlin gcanlin is a code owner
@tzhouam tzhouam Awaiting requested review from tzhouam tzhouam is a code owner
@yenuo26 yenuo26 Awaiting requested review from yenuo26 yenuo26 is a code owner

Assignees

No one assigned

Labels

bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yinpeiqi @hsliuustc0106