[Perf] Add tuned triton moe config for Qwen3.5 H200, 9.9% E2E throughput improvement

[yewentao256]

Purpose

1. updating tuning script, adding support for Qwen3.5
2. add a tuned config for it

Note: Computing resource is limited so just running for bs 128, which is the hot path in practice.

Test

export MODEL="Qwen/Qwen3.5-35B-A3B-FP8"
vllm serve $MODEL --port 9256 --enable-expert-parallel --enable-expert-parallel --profiler-config.profiler=torch --profiler-config.torch_profiler_dir=/home/yewentao256/profile_vllm

Acc

lm_eval --model local-completions --model_args "base_url=http://127.0.0.1:9256/v1/completions,model=$MODEL,num_concurrent=1024" --tasks gsm8k

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.7801|±  |0.0114|
|     |       |strict-match    |     5|exact_match|↑  |0.7650|±  |0.0117|

Perf

vllm bench serve --model $MODEL --dataset-name random --host 127.0.0.1 --port 9256 --random-input-len 2 --random-output-len 512 --request-rate inf --num-prompts 128 --num-warmups 16

# now
============ Serving Benchmark Result ============
Successful requests:                     128       
Failed requests:                         0         
Benchmark duration (s):                  9.44      
Total input tokens:                      256       
Total generated tokens:                  65536     
Request throughput (req/s):              13.56     
Output token throughput (tok/s):         6940.23   
Peak output token throughput (tok/s):    7168.00   
Peak concurrent requests:                128.00    
Total token throughput (tok/s):          6967.34   
---------------Time to First Token----------------
Mean TTFT (ms):                          169.15    
Median TTFT (ms):                        167.24    
P99 TTFT (ms):                           201.44    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          18.11     
Median TPOT (ms):                        18.12     
P99 TPOT (ms):                           18.12     
---------------Inter-token Latency----------------
Mean ITL (ms):                           18.11     
Median ITL (ms):                         18.06     
P99 ITL (ms):                            19.42     
==================================================

# main
============ Serving Benchmark Result ============
Successful requests:                     128       
Failed requests:                         0         
Benchmark duration (s):                  10.38     
Total input tokens:                      256       
Total generated tokens:                  65536     
Request throughput (req/s):              12.33     
Output token throughput (tok/s):         6313.91   
Peak output token throughput (tok/s):    6528.00   
Peak concurrent requests:                128.00    
Total token throughput (tok/s):          6338.57   
---------------Time to First Token----------------
Mean TTFT (ms):                          306.44    
Median TTFT (ms):                        305.06    
P99 TTFT (ms):                           317.39    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          19.69     
Median TPOT (ms):                        19.69     
P99 TPOT (ms):                           19.69     
---------------Inter-token Latency----------------
Mean ITL (ms):                           19.69     
Median ITL (ms):                         19.71     
P99 ITL (ms):                            21.24     
==================================================

[gemini-code-assist]

Code Review

This pull request introduces a tuned Triton MoE configuration for H200, which, according to the benchmarks, yields a significant E2E throughput improvement. The changes also update the tuning script to support the Qwen3.5 model and improve robustness by refactoring how model architecture and data types are determined. The code modifications are sound and directly support the PR's objectives. I found no issues with the implementation.