[2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async #2677

Merged

hsliuustc0106 merged 18 commits into

vllm-project:mainfrom

natureofnature:pr/switch-connector-flow

May 15, 2026

Commits on May 14, 2026

split commnication refactor pr: 2/5
natureofnature
committed
Fix Buildkite Audio Streaming and Bagel CI regressions.
natureofnature
committed
Fix Qwen3-Omni non-async audio over-generation and O(N^2) accumulator.
natureofnature
committed
Trim per-step overhead in the full-payload accumulator hot path.
natureofnature
committed
Skip empty-cache work and collapse double-branch in _sync_local_stage_payloads.
natureofnature
committed
Drop redundant sender-side all-zero tensor filter in accumulate_full_payload_output.
natureofnature
committed
Replace per-request waiting_queue.remove() loop with bulk remove_requests.
natureofnature
committed
Skip lock + broadcast in recv_full_payload_inputs when TP-trivial and idle.
natureofnature
committed
[Perf] Wire mixin cleanup_finished_request into runner finish hook
natureofnature
committed
[BugFix] Guard cleanup_finished_request by mixin state init
natureofnature
committed
[Perf] Increase _recv_loop no-progress backoff from 1ms to 5ms
natureofnature
committed
[BugFix] cleanup_finished_request must clean both internal and external keys
natureofnature
committed
Reword two comments per reviewer feedback.
natureofnature
committed
Cosmetic cleanup per reviewer feedback.
natureofnature
committed
[BugFix] Route non-async-chunk Stage-1 input through worker connector.
natureofnature
committed
[BugFix] Forward speaker/language metadata in non-async-chunk Stage-1 input.
natureofnature
committed
[BugFix] Trim trailing stop-token row from accumulated thinker output.
natureofnature
committed
Merge branch 'main' into pr/switch-connector-flow
hsliuustc0106
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async #2677

[2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async #2677
hsliuustc0106 merged 18 commits into
vllm-project:mainfrom
natureofnature:pr/switch-connector-flow

Commits on May 14, 2026

split commnication refactor pr: 2/5

Fix Buildkite Audio Streaming and Bagel CI regressions.

Fix Qwen3-Omni non-async audio over-generation and O(N^2) accumulator.

Trim per-step overhead in the full-payload accumulator hot path.

Skip empty-cache work and collapse double-branch in _sync_local_stage_payloads.

Drop redundant sender-side all-zero tensor filter in accumulate_full_payload_output.

Replace per-request waiting_queue.remove() loop with bulk remove_requests.

Skip lock + broadcast in recv_full_payload_inputs when TP-trivial and idle.

[Perf] Wire mixin cleanup_finished_request into runner finish hook

[BugFix] Guard cleanup_finished_request by mixin state init

[Perf] Increase _recv_loop no-progress backoff from 1ms to 5ms

[BugFix] cleanup_finished_request must clean both internal and external keys

Reword two comments per reviewer feedback.

Cosmetic cleanup per reviewer feedback.

[BugFix] Route non-async-chunk Stage-1 input through worker connector.

[BugFix] Forward speaker/language metadata in non-async-chunk Stage-1 input.

[BugFix] Trim trailing stop-token row from accumulated thinker output.

Merge branch 'main' into pr/switch-connector-flow