[Bugfix]: resolve torch.compile cache conflict between mm_encoder_tp_modes

[HirokenOvo]

Purpose

PR #23207 introduced torch compile support for the ViT part of Qwen2.5-VL. This PR addresses an issue where enabling torch.compile for the vision encoder (--compilation-config '{"compile_mm_encoder": true}') caused crashes when switching between --mm-encoder-tp-mode "weights" and --mm-encoder-tp-mode "data".

The Problem:

vLLM uses VllmConfig.compute_hash() to identify unique configurations for caching compiled graphs. However, mm_encoder_tp_mode was missing from this hash calculation. As a result, running the model with weights mode generated a cache that data mode would try to reuse (or vice versa). Since these modes result in different tensor shapes/strides for the ViT, this caused an AssertionError in the generated Inductor kernels.

The Solution:

1. Updated vllm/config/multimodal.py: Added mm_encoder_tp_mode to the factors used in MultiModalConfig.compute_hash().
2. Updated vllm/config/vllm.py: Modified VllmConfig.compute_hash() to explicitly include the multimodal_config hash if and only if compile_mm_encoder is enabled. This ensures correct cache isolation without affecting the hash for non-compiled runs.

Related Error Log

vllm serve /data/models/qwen2_5vl-3B/ --compilation-config '{"compile_mm_encoder": true}' --tensor-parallel-size 2 --mm-encoder-tp-mode "data" --max-model-len 8192 --gpu-memory-utilization 0.5
vllm serve /data/models/qwen2_5vl-3B/ --compilation-config '{"compile_mm_encoder": true}' --tensor-parallel-size 2 --mm-encoder-tp-mode "weights" --max-model-len 8192 --gpu-memory-utilization 0.5

(Worker_TP0 pid=2909827) ERROR 01-22 16:56:04 [multiproc_executor.py:839]   File "/tmp/torchinductor_root/he/chehoqaxuhke4t2nqjka646fx2in2rwogdaqfv5djiz374ppmcpq.py", line 550, in call
(Worker_TP0 pid=2909827) ERROR 01-22 16:56:04 [multiproc_executor.py:839]     assert_size_stride(arg4_1, (3456, 1152), (1152, 1))
(Worker_TP0 pid=2909827) ERROR 01-22 16:56:04 [multiproc_executor.py:839] AssertionError: expected size 1728==3456, stride 1152==1152 at dim=0

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please

[DarkLight1337]

Thanks for fixing

[DarkLight1337]

cc @ywang96 @Isotr0py