Skip to content

[Revert] Revert "[Log] Wire stat loggers into AsyncOmniEngine to match AsyncLL…#2716

Merged
gcanlin merged 1 commit into
vllm-project:mainfrom
amy-why-3459:revert_log_stats
Apr 13, 2026
Merged

[Revert] Revert "[Log] Wire stat loggers into AsyncOmniEngine to match AsyncLL…#2716
gcanlin merged 1 commit into
vllm-project:mainfrom
amy-why-3459:revert_log_stats

Conversation

@amy-why-3459
Copy link
Copy Markdown
Contributor

@amy-why-3459 amy-why-3459 commented Apr 13, 2026

…M (#2551)"

This reverts commit 5d58abb.

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

This PR resulted in a significant performance degradation of the Qwen3-Omni.

Test Plan

vllm bench serve \
    --omni \
  --dataset-name random \
  --port 28889 \
  --max-concurrency 10 \
  --model /home/models/Qwen/Qwen3-Omni-30B-A3B-Instruct \
  --endpoint /v1/chat/completions \
  --backend openai-chat-omni \
  --num-prompts 100 \
  --random-input-len 100 \
  --ignore-eos \
  --percentile-metrics ttft,tpot,itl,e2el,audio_ttfp,audio_rtf \
  --random-output-len 100 \
  --extra_body '{"modalities": ["text", "audio"]}'

Test Result

Before this PR:

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             10
Benchmark duration (s):                  786.00
Request throughput (req/s):              0.13
Peak concurrent requests:                15.00
----------------End-to-end Latency----------------
Mean E2EL (ms):                          73081.40
Median E2EL (ms):                        74923.60
P99 E2EL (ms):                           141293.25
================== Text Result ===================
Total input tokens:                      11400
Total generated tokens:                  9753
Output token throughput (tok/s):         12.41
Peak output token throughput (tok/s):    260.00
Peak concurrent requests:                15.00
Total Token throughput (tok/s):          26.91
---------------Time to First Token----------------
Mean TTFT (ms):                          300.15
Median TTFT (ms):                        79.13
P99 TTFT (ms):                           1702.24
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          20.03
Median TPOT (ms):                        14.26
P99 TPOT (ms):                           44.94
---------------Inter-token Latency----------------
Mean ITL (ms):                           19.21
Median ITL (ms):                         12.50
P99 ITL (ms):                            141.90
================== Audio Result ==================
Total audio duration generated(s):       3340.60
Total audio frames generated:            80174100
Audio throughput(audio duration/s):      4.25
---------------Time to First Packet---------------
Mean AUDIO_TTFP (ms):                    55828.35
Median AUDIO_TTFP (ms):                  57789.24
P99 AUDIO_TTFP (ms):                     127832.68
-----------------Real Time Factor-----------------
Mean AUDIO_RTF:                          2.24
Median AUDIO_RTF:                        2.13
P99 AUDIO_RTF:                           4.27
==================================================

After this PR:

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             10
Benchmark duration (s):                  216.69
Request throughput (req/s):              0.46
Peak concurrent requests:                14.00
----------------End-to-end Latency----------------
Mean E2EL (ms):                          20828.36
Median E2EL (ms):                        21234.42
P99 E2EL (ms):                           25964.29
================== Text Result ===================
Total input tokens:                      11400
Total generated tokens:                  9715
Output token throughput (tok/s):         44.83
Peak output token throughput (tok/s):    265.00
Peak concurrent requests:                14.00
Total Token throughput (tok/s):          97.44
---------------Time to First Token----------------
Mean TTFT (ms):                          1700.18
Median TTFT (ms):                        1060.38
P99 TTFT (ms):                           6937.31
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          51.93
Median TPOT (ms):                        51.86
P99 TPOT (ms):                           79.57
---------------Inter-token Latency----------------
Mean ITL (ms):                           49.47
Median ITL (ms):                         0.01
P99 ITL (ms):                            1067.63
================== Audio Result ==================
Total audio duration generated(s):       3295.06
Total audio frames generated:            79081155
Audio throughput(audio duration/s):      15.21
---------------Time to First Packet---------------
Mean AUDIO_TTFP (ms):                    3208.85
Median AUDIO_TTFP (ms):                  2684.91
P99 AUDIO_TTFP (ms):                     8015.75
-----------------Real Time Factor-----------------
Mean AUDIO_RTF:                          0.64
Median AUDIO_RTF:                        0.63
P99 AUDIO_RTF:                           0.88
==================================================

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@amy-why-3459
Copy link
Copy Markdown
Contributor Author

Please take a look and help add a nightly-test label. @gcanlin

@gcanlin gcanlin added the nightly-test label to trigger buildkite nightly test CI label Apr 13, 2026
vllm-project#2551)"

This reverts commit 5d58abb.

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
@gcanlin gcanlin added ready label to trigger buildkite CI and removed nightly-test label to trigger buildkite nightly test CI labels Apr 13, 2026
Copy link
Copy Markdown
Collaborator

@gcanlin gcanlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing!

@gcanlin gcanlin merged commit 2b70e89 into vllm-project:main Apr 13, 2026
7 of 8 checks passed
daixinning pushed a commit to daixinning/vllm-omni that referenced this pull request Apr 13, 2026
…h AsyncLL… (vllm-project#2716)

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
lengrongfu pushed a commit to lengrongfu/vllm-omni that referenced this pull request May 1, 2026
…h AsyncLL… (vllm-project#2716)

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
…h AsyncLL… (vllm-project#2716)

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants