[Bugfix] clean per-stage audio RTF and align audio metrics

[ZacheryAU]

Purpose

Solve #4681 and some per-stage audio metrics in benchmark

Summary

• Remove per-stage audio_rtf from calculation in StagePool to printing in vllm bench serve --omni --print-stage
• As type of multimodal outputs changed from dict to MultimodalPayload which implements Mapping ( [Refactor] Output Processor Phase 2: separate multimodal output channel (#1601) #2744), type checking should be updated as well to fix failure of collecting data in stage benchmark
• Fix duplicated audio frames and durations calculation

Test Plan

vLLM Version:
0.23.0
vLLM-Omni Commit:
cd35572

• Run vllm bench serve --omni --print-stage with audio-output model to see:
  • if audio_rtf no longer in stage benchmark
  • if audio frames and durations stage benchmark identical to serving benchmark

Test Result

Qwen/Qwen3-Omni-30B-A3B-Instruct

Show more

============ Serving Benchmark Result ============
Successful requests:                     3         
Failed requests:                         0         
Maximum request concurrency:             1         
Benchmark duration (s):                  31.34     
Request throughput (req/s):              0.10      
Peak concurrent requests:                2.00      
----------------End-to-end Latency----------------
Mean E2EL (ms):                          10446.52  
Median E2EL (ms):                        10097.34  
P99 E2EL (ms):                           11125.40  
================== Text Result ===================
Total input tokens:                      7524      
Total generated tokens:                  2700      
Output token throughput (tok/s):         86.14     
Peak output token throughput (tok/s):    103.00    
Peak concurrent requests:                2.00      
Total Token throughput (tok/s):          326.20    
---------------Time to First Token----------------
Mean TTFT (ms):                          670.54    
Median TTFT (ms):                        671.56    
P99 TTFT (ms):                           680.59    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          10.48     
Median TPOT (ms):                        10.47     
P99 TPOT (ms):                           10.50     
---------------Inter-token Latency----------------
Mean ITL (ms):                           10.48     
Median ITL (ms):                         10.35     
P99 ITL (ms):                            14.77     
================== Audio Result ==================
Total audio duration generated(s):       175.47    
Total audio frames generated:            4211340   
Audio throughput(audio duration/s):      5.60      
Streaming continuity OK rate:            100.00%   
---------------Time to First Packet---------------
Mean AUDIO_TTFP (ms):                    914.64    
Median AUDIO_TTFP (ms):                  917.63    
P99 AUDIO_TTFP (ms):                     918.27    
-----------------Real Time Factor-----------------
Mean AUDIO_RTF:                          0.16      
Median AUDIO_RTF:                        0.17      
P99 AUDIO_RTF:                           0.17      

============= Stage Benchmark Result =============
=============== Stage 0 (thinker) ================
-------------------Stage Timing-------------------
Mean stage_gen_time (ms):                10050.68  
Median stage_gen_time (ms):              10052.00  
P99 stage_gen_time (ms):                 10061.17  
================== Text Result ===================
Stage generated tokens:                  2700      
-----------Serving Time to First Token------------
Mean Serving TTFT (ms):                  667.44    
Median Serving TTFT (ms):                668.09    
P99 Serving TTFT (ms):                   677.11    
---------------Time to First Token----------------
Mean TTFT (ms):                          657.37    
Median TTFT (ms):                        657.96    
P99 TTFT (ms):                           659.32    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          10.45     
Median TPOT (ms):                        10.46     
P99 TPOT (ms):                           10.46     
---------------Inter-token Latency----------------
Mean ITL (ms):                           10.45     
Median ITL (ms):                         10.43     
P99 ITL (ms):                            11.22     
================ Stage 1 (talker) ================
-------------------Stage Timing-------------------
Mean stage_gen_time (ms):                9578.89   
Median stage_gen_time (ms):              8859.88   
P99 stage_gen_time (ms):                 11061.96  
============= Internal Stream Result =============
-----------Serving Time to First Chunk------------
Mean Serving TTFC (ms):                  853.83    
Median Serving TTFC (ms):                857.45    
P99 Serving TTFC (ms):                   857.79    
-----Time per Output Chunk (excl. 1st chunk)------
Mean TPOP (ms):                          11.82     
Median TPOP (ms):                        11.82     
P99 TPOP (ms):                           11.83     
---------------Inter-chunk Latency----------------
Mean ICL (ms):                           11.82     
Median ICL (ms):                         11.12     
P99 ICL (ms):                            20.26     
=============== Stage 2 (code2wav) ===============
-------------------Stage Timing-------------------
Mean stage_gen_time (ms):                9585.15   
Median stage_gen_time (ms):              8863.95   
P99 stage_gen_time (ms):                 11070.19  
================== Audio Result ==================
Stage audio duration generated(s):       175.47    
Stage audio frames generated:            4211340   
-----------Serving Time to First Packet-----------
Mean Serving AUDIO_TTFP (ms):            911.32    
Median Serving AUDIO_TTFP (ms):          914.18    
P99 Serving AUDIO_TTFP (ms):             914.87    
==================================================

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

cc @R2-Y

[R2-Y]

LGTM, thanks.

[hsliuustc0106]

fix https://buildkite.com/vllm/vllm-omni/builds/11923/canvas

[Gaohan123]

LGTM. Thanks