[BugFix][Multi Modal] Fix TensorSchema shape mismatch in Molmo

[wwl2755]

Number of crops would be dynamic in different images, so need to make it dynamic.

Fix: #24544

Related: #22022

cc: @bbeckca @DarkLight1337

Test

python -m vllm.entrypoints.openai.api_server --model allenai/Molmo-7B-D-0924 --trust-remote-code

vllm bench serve   --backend openai-chat   --endpoint-type openai-chat   --model allenai/Molmo-7B-D-0924   --endpoint /v1/chat/completions   --dataset-name hf   --dataset-path lmarena-ai/VisionArena-Chat   --hf-split train   --hf-output-len 1   --num-prompts 10   --seed 40

Result

============ Serving Benchmark Result ============
Successful requests:                     10        
Benchmark duration (s):                  1.15      
Total input tokens:                      816       
Total generated tokens:                  10        
Request throughput (req/s):              8.70      
Output token throughput (tok/s):         8.70      
Total Token throughput (tok/s):          719.00    
---------------Time to First Token----------------
Mean TTFT (ms):                          695.62    
Median TTFT (ms):                        549.66    
P99 TTFT (ms):                           1132.05   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          0.00      
Median TPOT (ms):                        0.00      
P99 TPOT (ms):                           0.00      
---------------Inter-token Latency----------------
Mean ITL (ms):                           0.02      
Median ITL (ms):                         0.02      
P99 ITL (ms):                            0.02      
==================================================

[gemini-code-assist]

Code Review

This pull request addresses a shape mismatch issue in the Molmo model by correctly marking the 'number of crops' (nc) dimension as dynamic. This is a crucial fix for handling batches of images that produce a variable number of crops. My review identifies a related issue where the 'token sequence positions' (tp) dimension, which can also vary between images, was not marked as dynamic. I've suggested a fix to prevent potential validation errors.

[gemini-code-assist]

The number of token sequence positions (tp) can also be dynamic for different images, similar to the number of crops (nc). The number of tokens depends on the image's height and width. To prevent potential ValueError during tensor shape validation when processing a batch of images with varying sizes, tp should also be marked as a dynamic dimension.

Suggested change

	feat_is_patch: Annotated[
	Union[torch.Tensor, list[torch.Tensor]],
	TensorShape("bn", "nc", "tp", dynamic_dims={"nc"})]
	feat_is_patch: Annotated[
	Union[torch.Tensor, list[torch.Tensor]],
	TensorShape("bn", "nc", "tp", dynamic_dims={"nc", "tp"})]

[Isotr0py]

Thanks for fixing! This model wasn't covered by CI because of its requirement conflicts 😅

[DarkLight1337]

Thanks, cc @Isotr0py we should make the test more robust to catch this case

[bbeckca]

Thanks @wwl2755 for the fix!