Skip to content

[CI/Build] Re-enable turn-based server VAD realtime test - #9300

Merged
alex-jw-brooks merged 1 commit into
vllm-project:mainfrom
NickCao:codex/turnbased-server-vad-ci
Oct 26, 2026
Merged

alex-jw-brooks merged 1 commit into
vllm-project:mainfrom
NickCao:codex/turnbased-server-vad-ci

Conversation

@NickCao

@NickCao NickCao commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Turn-based server VAD is implemented, so stop deselecting the Qwen3-Omni multi-turn E2E. Update the test to the Realtime 24 kHz PCM contract and assert explicit interruptible server VAD settings.

Contributor intent (required human answers)

All four questions must be answered by the human contributor before submission.
An agent may reuse explicit human answers with a public-safe source reference;
it must not invent, infer, or prefill missing answers. If an answer is missing
or materially ambiguous, ask the human and wait. Do not publish private
conversation content without authorization.

1. What outcome do you want to achieve?

Re-enable the turn-based server VAD realtime test.

2. What is the highest-leverage variable for that outcome?

Drop the pytest deselect and set proper turn_detection parameters.

3. How will this work move that variable?

Drop the pytest deselect and set proper turn_detection parameters, as well as fix the incompatible sample rate.

4. Whose customer or user needs does this address, and what are those needs?

Developer can now easily test the turn-based server VAD realtime behavior end to end.

Purpose

Previous the turn-based server VAD realtime test was disabled as server vad support was removed during refactoring, re-enable that since this has been reimplemented.

Test Plan

vLLM Version: 0.32.0

vLLM-Omni Commit: 3b4e104

pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and H100" --run-level "advanced_model"

Test Result

25 passed, 2893 deselected, 36 warnings in 1291.99s (0:21:31)

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.

Note

To avoid agent spam, vLLM-Omni limits contributors without write access to 5 open PRs. If this blocks well-intentioned critical work, please contact a committer to request bypass list access.
(anything written below this line will be removed by GitHub Actions)

@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from e840112 to 3b4e104 Compare October 23, 2026 14:22
@NickCao
NickCao marked this pull request as ready for review October 23, 2026 14:50
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-23T14:50:10.888618Z
We couldn’t complete this review. Comment @codex review to try again.
3b4e104 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@vllm-omni-review-bot

Copy link
Copy Markdown

This PR was classified as CI work.

CI owner: @yenuo26 @congw729

Routing: @yenuo26 via CI owner, CODEOWNERS; @congw729 via CODEOWNERS

@NickCao, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

@vllm-omni-review-bot

vllm-omni-review-bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Omni ReviewBot triage note

Automated triage of commit a9c942439f08 produced:

  • Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NickCao — evidence TODO for 3b4e104a5dae7dcc73a6f5350af51a3b6e61352f (base cbe4cc4b7c1126a903553f8e79e44ddfab782cf3).

  • Please attach the verbose result for TestQwen3OmniRealtimeWebSocket::test_server_vad_multi_turn_without_client_commit at this head, with GPU/model/runtime details and the effective session settings plus terminal-event summary for both turns. The supplied 25 passed aggregate does not identify whether this newly enabled test executed.
  • Acceptance: the 24 kHz PCM session completes two distinct automatically triggered responses without a client commit, retaining the existing nonempty-audio, unique-ID and terminal-event assertions. A focused negative control (for example, disabling automatic response) would also establish that the test catches the behavior it is intended to guard.

I inspected the two-file change and related audio/protocol handling, but did not execute the test. This is an evidence request, not a code-defect finding; current-head CUDA readiness is still unverified.

@NickCao

NickCao commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator Author

@NickCao — evidence TODO for 3b4e104a5dae7dcc73a6f5350af51a3b6e61352f (base cbe4cc4b7c1126a903553f8e79e44ddfab782cf3).

* [ ]  Please attach the verbose result for `TestQwen3OmniRealtimeWebSocket::test_server_vad_multi_turn_without_client_commit` at this head, with GPU/model/runtime details and the effective session settings plus terminal-event summary for both turns. The supplied `25 passed` aggregate does not identify whether this newly enabled test executed.

* Acceptance: the 24 kHz PCM session completes two distinct automatically triggered responses without a client commit, retaining the existing nonempty-audio, unique-ID and terminal-event assertions. A focused negative control (for example, disabling automatic response) would also establish that the test catches the behavior it is intended to guard.

I inspected the two-file change and related audio/protocol handling, but did not execute the test. This is an evidence request, not a code-defect finding; current-head CUDA readiness is still unverified.

The full log is at https://gist.github.com/NickCao/8f0f431ec1e45bf0841ab7eeaf3ee6d1:

--- Running test: test_server_vad_multi_turn_without_client_commit[server_vad]
PASSED

@hsliuustc0106 hsliuustc0106 added the high priority high priority issue, needs to be done asap label Oct 24, 2026 — with ChatGPT Codex Connector
@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from 3b4e104 to 0ba262c Compare October 26, 2026 13:26
@NickCao NickCao added the ready label to trigger buildkite CI label Oct 26, 2026
@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from 0ba262c to 0be3109 Compare October 26, 2026 14:08
@NickCao NickCao added ready label to trigger buildkite CI and removed ready label to trigger buildkite CI labels Oct 26, 2026

@alex-jw-brooks alex-jw-brooks left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit, but otherwise lgtm

Comment thread tests/entrypoints/openai_api/test_qwen3_omni_realtime_websocket.py Outdated
@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from 0be3109 to dfad251 Compare October 26, 2026 15:28
@NickCao NickCao added ready label to trigger buildkite CI and removed ready label to trigger buildkite CI labels Oct 26, 2026
@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from dfad251 to 958fed7 Compare October 26, 2026 15:40
@NickCao NickCao removed the ready label to trigger buildkite CI label Oct 26, 2026
Turn-based server VAD is implemented, so stop deselecting the Qwen3-Omni multi-turn E2E. Update the test to the Realtime 24 kHz PCM contract and assert explicit interruptible server VAD settings.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
@NickCao
NickCao force-pushed the codex/turnbased-server-vad-ci branch from 958fed7 to a9c9424 Compare October 26, 2026 15:41
@NickCao NickCao added the ready label to trigger buildkite CI label Oct 26, 2026
@NickCao

NickCao commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Re-tested pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and H100" --run-level "advanced_model" -k test_server_vad_multi_turn_without_client_commit

@alex-jw-brooks
alex-jw-brooks merged commit 6935183 into vllm-project:main Oct 26, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

high priority high priority issue, needs to be done asap ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants