[Transformers v5] fix missing pixtral/voxtral multimodal dispatch

[allgather]

Purpose

fix #38382

Transformers decides which processor components to call by looking at the processor constructor and mistral processors only show tokenizer.

This made it so the pixtral image processor and voxtral feature extractor stopped running, but vllm still got text tokens, just no mm kwargs. This is why the issue showed:

FAILED models/multimodal/processing/test_tensor_schema.py::test_model_tensor_schema[mistralai/Pixtral-12B-2409] - RuntimeError: Expected there to be 3 image items in keyword > arguments corresponding to 3 image data items, but only found 0!

(EngineCore pid=1621824)   File "/home/harry/vllm/vllm/multimodal/processing/processor.py", line 1374, in _merge_mm_kwargs
(EngineCore pid=1621824)     missing_kwargs_item = missing_kwargs[missing_next_idx]
(EngineCore pid=1621824)                           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
(EngineCore pid=1621824) IndexError: list index out of range

Test Result

tests from the issue desc passed. ran on 1xa100

tests/entrypoints/openai/speech_to_text/test_transcription_validation.py::test_basic_audio[mistralai/Voxtral-Mini-3B-2507]

pytest -s -vv 'tests/entrypoints/openai/speech_to_text/test_transcription_validation.py::test_basic_audio[mistralai/Voxtral-Mini-3B-2507]'

(EngineCore pid=4587) DEBUG 03-28 01:00:40 [v1/worker/gpu_model_runner.py:3894] ubatch_slices: None, ubatch_slices_padded: None
(APIServer pid=4445) INFO:     127.0.0.1:33102 - "POST /v1/audio/transcriptions HTTP/1.1" 200 OK
[RemoteOpenAIServer] Server 4445 terminated gracefully
[RemoteOpenAIServer] GPU memory released to 0.54 GB (target: 2.69 GB) in 0.0s
PASSED

================== 1 passed, 17 warnings in 97.66s (0:01:37) ===================

tests/entrypoints/openai/realtime/test_realtime_validation.py::test_multi_chunk_streaming[mistralai/Voxtral-Mini-4B-Realtime-2602]

pytest -s -vv 'tests/entrypoints/openai/realtime/test_realtime_validation.py::test_multi_chunk_streaming[mistralai/Voxtral-Mini-4B-Realtime-2602]'

(EngineCore pid=5250) DEBUG 03-28 01:04:40 [v1/worker/gpu_model_runner.py:3873] Running batch with cudagraph_mode: NONE, batch_descriptor: BatchDescriptor(num_tokens=1, num_reqs=None, uniform=False, has_lora=False, num_active_loras=0), should_ubatch: False, num_tokens_across_dp: None
(APIServer pid=5108) DEBUG 03-28 01:04:40 [entrypoints/.../realtime/connection.py:287] Connection cleanup complete: ws-0a9950a4-a31b-4473-bd66-2aff29096eb2
(APIServer pid=5108) INFO:     Finished server process [5108]
[RemoteOpenAIServer] Server 5108 terminated gracefully
[RemoteOpenAIServer] GPU memory released to 0.54 GB (target: 2.69 GB) in 0.0s
PASSED

================== 1 passed, 21 warnings in 94.21s (0:01:34) ===================

tests/entrypoints/openai/realtime/test_realtime_validation.py follow-up failures

pytest -s -vv -rA \
  'tests/entrypoints/openai/realtime/test_realtime_validation.py::test_empty_commit_does_not_crash_engine[mistralai/Voxtral-Mini-4B-Realtime-2602]' \
  'tests/entrypoints/openai/realtime/test_realtime_validation.py::test_session_update_invalid_model_returns_error[mistralai/Voxtral-Mini-4B-Realtime-2602]' \
  'tests/entrypoints/openai/realtime/test_realtime_validation.py::test_commit_without_session_update_returns_error[mistralai/Voxtral-Mini-4B-Realtime-2602]'

==================================== PASSES ====================================
PASSED tests/entrypoints/openai/realtime/test_realtime_validation.py::test_empty_commit_does_not_crash_engine[mistralai/Voxtral-Mini-4B-Realtime-2602]
PASSED tests/entrypoints/openai/realtime/test_realtime_validation.py::test_session_update_invalid_model_returns_error[mistralai/Voxtral-Mini-4B-Realtime-2602]
PASSED tests/entrypoints/openai/realtime/test_realtime_validation.py::test_commit_without_session_update_returns_error[mistralai/Voxtral-Mini-4B-Realtime-2602]

================== 3 passed, 21 warnings in 130.18s (0:02:10) ==================

cc @hmellor

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[gemini-code-assist]

Code Review

This pull request updates the MistralCommonPixtralProcessor and MistralCommonVoxtralProcessor classes to allow passing an optional image_processor or feature_extractor during initialization. If these components are not provided, they are instantiated using the tokenizer's internal encoders as before. I have no feedback to provide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[hmellor]

If it is of any additional help, the refactor in huggingface/transformers#43514 is that caused the incompatibility with the refactor mentioned in the issue that this PR fixes.