[AutoRound] Support Qwen Omni W4A16 quantization model

[lvliang-intel]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Enable loading offline AutoRound W4A16 quantized checkpoints for Qwen Omni multi-stage models (Qwen2.5-Omni and Qwen3-Omni). Extends the AutoRound/INC support added in #1777 (FLUX) to the Omni pipeline.

Related: #1325, #1777

Test Plan

• Unit tests in tests/diffusion/quantization/test_component_routing.py covering
• Manual end-to-end loading verified with checkpoint Qwen3-Omni-30B-A3B W4A16 and Qwen2.5-Omni-7B W4A16, the original models also are covered.

Test Result

Unit tests: all 31 tests pass.
CMD and Model:

CUDA_VISIBLE_DEVICES=0,1 python examples/offline_inference/qwen2_5_omni/end2end.py --model Intel/Qwen2.5-Omni-7B-int4-AutoRound --output-wav output_audio --query-type use_audio
CUDA_VISIBLE_DEVICES=0,1 python examples/offline_inference/qwen3_omni/end2end.py --model Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound --output-wav output_audio --query-type use_audio

Model	Metric	BF16 Baseline	W4A16 (AutoRound)
Qwen3-Omni-30B-A3B	Checkpoint Size (GiB)	66	25
	Size Reduction	--	62%
Qwen2.5-Omni-7B	Checkpoint Size (GiB)	21	12
	Size Reduction	--	43%

Accuracy Test:

CUDA_VISIBLE_DEVICES=0,1 vllm serve Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound --omni  --port 8801  --max-model-len 32768   --served-model-name Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound
CUDA_VISIBLE_DEVICES=0,1 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni  --port 8801  --max-model-len 32768   --served-model-name Qwen3-Omni-30B-A3B-Instruct

Using evalscope to test dataset omni-bench:

 python - <<'PY'
from evalscope import TaskConfig, run_task
task_cfg = TaskConfig(
    model='Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound',
    api_url='http://127.0.0.1:8801/v1',
    api_key='EMPTY',
    eval_type='openai_api',
    datasets=['omni_bench'],
    dataset_args={
        'omni_bench': {
            'extra_params': {
                'use_image': True,
                'use_audio': True,
            }
        }
    },
    eval_batch_size=1,
    generation_config={
        'max_tokens': 10000,
        'temperature': 0.0,
    },
    limit=100,
    ignore_errors=True,
)
run_task(task_cfg=task_cfg)
PY

Model	Dataset	Metric	Subset	Num	Score	Delta vs BF16
Qwen3-Omni-30B-A3B-Instruct	omni_bench	mean_acc	default	100	0.46	0.00
Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound	omni_bench	mean_acc	default	100	0.44	-0.02

Performance Benchmark Results: FP8 (W8A8) vs INT4 (W4A16 AutoRound)

fp8_bench.json
int4_bench.json

Model: Qwen3-Omni-30B-A3B-Instruct
Hardware: 8x NVIDIA RTX 5090 D (32GB)
Configuration: Thinker TP=4 (GPU 0-3), Talker TP=2 (GPU 4-5), Code2Wav TP=2 (GPU 6-7)
Benchmark: 200 ShareGPT prompts, request_rate=10, max_concurrency=32, temperature=0, custom_output_len=256

Serving Performance

Metric	FP8 (W8A8)	INT4 (W4A16)	Diff
Successful requests	200	200	-
Failed requests	0	0	-
Benchmark duration (s)	248.31	239.42	INT4 -3.6%
Request throughput (req/s)	0.81	0.84	INT4 +3.7%

Text Generation

Metric	FP8 (W8A8)	INT4 (W4A16)	Diff
Total generated tokens	40,015	39,902	-
Output token throughput (tok/s)	161.15	166.66	INT4 +3.4%
Peak output token throughput (tok/s)	768.00	704.00	FP8 +9.1%

Audio Generation

Metric	FP8 (W8A8)	INT4 (W4A16)	Diff
Total audio duration generated (s)	14,134.53	12,523.22	-
Audio throughput (audio dur/s)	56.92	52.31	FP8 +8.8%

Latency

Metric	FP8 (W8A8)	INT4 (W4A16)	Diff
Mean TTFT (ms)	90.14	107.01	FP8 -18.7%
Median TTFT (ms)	46.35	58.55	FP8 -20.9%
P99 TTFT (ms)	526.80	420.74	INT4 -20.1%
Mean TPOT (ms)	17.70	21.23	FP8 -16.6%
Median TPOT (ms)	12.29	15.73	FP8 -21.9%
P99 TPOT (ms)	47.42	50.58	FP8 -6.3%
Mean ITL (ms)	17.61	21.11	FP8 -16.6%
Median ITL (ms)	11.75	14.90	FP8 -21.1%
P99 ITL (ms)	128.49	131.67	FP8 -2.4%
Mean E2EL (ms)	36,718.00	35,465.59	INT4 -3.4%
Median E2EL (ms)	35,711.51	35,053.92	INT4 -1.8%
P99 E2EL (ms)	52,743.62	46,585.58	INT4 -11.7%

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
• The test results. Please paste the results comparison before and after, or the e2e results.
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
• (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[yiliu30]

Overall LGTM — it would be great to include a command for running the end-to-end example in PR desc.

[hsliuustc0106]

Well-structured PR. 31 unit tests, comprehensive benchmarks, docs updated. The accuracy delta (-0.02 on omni_bench) is acceptable for W4A16.

[lvliang-intel]

@hsliuustc0106 @lishunyang12,
please help to approve this PR if no further comments.

[lvliang-intel]

Rerun accuray test with latest code:
| Model | Dataset | Metric | Subset | Num | Score | Cat.0 |
| Qwen3-Omni-30B-A3B-Instruct | omni_bench | mean_acc | default | 100 | 0.46 | default |

| Model | Dataset | Metric | Subset | Num | Score | Cat.0 |
| Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound | omni_bench | mean_acc | default | 100 | 0.47 | default |

[lishunyang12]

Fix CI

[lvliang-intel]

@lishunyang12,
CI finally passed, please help to approve and merge this PR, thanks.

[lishunyang12]

let's wait for ci to complete

[hsliuustc0106]

@amy-why-3459 transformers will be upgraded to 5.x next week, please check the logic here

[lvliang-intel]

@hsliuustc0106 @lishunyang12,
could you please approve the PR?