Skip to content

[Test] Add voice or language test case for Qwen3-omni and Qwen-tts#1844

Merged
gcanlin merged 16 commits intovllm-project:mainfrom
yenuo26:voice
Apr 2, 2026
Merged

[Test] Add voice or language test case for Qwen3-omni and Qwen-tts#1844
gcanlin merged 16 commits intovllm-project:mainfrom
yenuo26:voice

Conversation

@yenuo26
Copy link
Copy Markdown
Collaborator

@yenuo26 yenuo26 commented Mar 12, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

1.Add speaker and language test cases for Qwen3-omni
2.Add voice case-sensitive test cases for Qwen-tts

Test Plan

  1. Qwen3-omni
pytest -sv tests/e2e/online_serving/test_qwen3_omni_expansion.py -k "test_speaker or test_language" --run-level advanced_model

2.Qwen-tts

pytest -sv tests/e2e/online_serving/test_qwen3_tts_customvoice_expansion.py::test_voice_003 --run-level advanced_model

Test Result

364379ad-59bd-4909-b3d0-bef9a45d6381
61241c1f-3cc9-4885-9d58-b82079b0071a
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

- Added OpenCC dependency for text conversion in preprocess_text function.
- Introduced functions to estimate voice gender from audio and validate against expected gender in assert_omni_response.
- Refactored audio merging logic into a separate function for clarity.
- Updated test cases to include voice gender specifications and improved parameter handling for model configurations.

Signed-off-by: yenuo26 <410167048@qq.com>
@yenuo26 yenuo26 requested a review from hsliuustc0106 as a code owner March 12, 2026 08:49
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 16e54f14bd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread tests/conftest.py Outdated
Comment on lines +936 to +939
if f0 > 170.0:
half = f0 / 2.0
if 75.0 <= half <= 185.0:
f0 = half
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid unconditional octave halving in gender estimator

This octave-correction branch halves any f0 > 170 when the halved value is in 75..185, which incorrectly maps most genuine female pitches (typically ~180–255 Hz) into the male range before classification. In practice, a correctly estimated female voice around 200–230 Hz gets forced to ~100–115 Hz and then labeled male, so the new female-voice validations can fail even when the model output is correct.

Useful? React with 👍 / 👎.

yenuo26 added 3 commits March 31, 2026 19:39
Signed-off-by: wangyu <410167048@qq.com>
…and Qwen3-omni

Signed-off-by: wangyu <410167048@qq.com>
@yenuo26 yenuo26 changed the title [WIP][Test] Add qwen3-omni voice and language test case [Test] Add voice or language test case for Qwen3-omni and Qwen-tts Mar 31, 2026
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <410167048@qq.com>
@yenuo26
Copy link
Copy Markdown
Collaborator Author

yenuo26 commented Apr 1, 2026

@congw729 please help to add nightly-test label

@congw729 congw729 added the nightly-test label to trigger buildkite nightly test CI label Apr 1, 2026
yenuo26 and others added 6 commits April 1, 2026 12:14
…gging and efficiency.

Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: wangyu <410167048@qq.com>
@congw729 congw729 added ready label to trigger buildkite CI and removed nightly-test label to trigger buildkite nightly test CI labels Apr 1, 2026
@yenuo26 yenuo26 changed the title [Test] Add voice or language test case for Qwen3-omni and Qwen-tts [WIP][Test] Add voice or language test case for Qwen3-omni and Qwen-tts Apr 1, 2026
@yenuo26
Copy link
Copy Markdown
Collaborator Author

yenuo26 commented Apr 1, 2026

success in nightly-test:
fdc4eba9-5a28-478f-879e-a5866287a9c2
success in merge-test:
a1bdc2cc-f9e6-40b9-a667-5a4bdd7f3603
The Bagel failure is a known issue and is being tracked in Issue #2416
Please review whether this PR can be merged. @gcanlin @Gaohan123

yenuo26 added 2 commits April 1, 2026 19:36
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <410167048@qq.com>
@yenuo26 yenuo26 changed the title [WIP][Test] Add voice or language test case for Qwen3-omni and Qwen-tts [Test] Add voice or language test case for Qwen3-omni and Qwen-tts Apr 1, 2026
Copy link
Copy Markdown
Collaborator

@gcanlin gcanlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gcanlin gcanlin merged commit 9c2a576 into vllm-project:main Apr 2, 2026
8 checks passed
vraiti pushed a commit to vraiti/vllm-omni that referenced this pull request Apr 9, 2026
…llm-project#1844)

Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants