[cjackal]

Overview

Fixes engine startup for Pixtral (+ other models with the same problem) when a modality is embedding-only (--limit-mm-per-prompt '{"<modality>": 0}' with --enable-mm-embeds). This is the root cause of the failure observed in #65648.

Claims

• Every model registered as of now obeys the contract that Add embedding input functionality for disabled modalities [remake] #32493 introduced.

Validation

vllm serve mistralai/Pixtral-12B-2409 --tokenizer-mode mistral \
    --limit-mm-per-prompt '{"image": 0}' --enable-mm-embeds

Details

#32493 added embedding input for disabled modality. It introduced the following split:

• User inputs are checked against --limit-mm-per-prompt and --enable-mm-embeds once, in the renderer (BaseRenderer calling info.parse_mm_data).
• Data the processor builds or re-parses is parsed with validate=False. Add embedding input functionality for disabled modalities [remake] #32493 switched both call sites:
  • the dummy inputs used for profiling (get_dummy_inputs)
  • the cache-miss items in _get_cache_missing_items

The split is needed because MultiModalBudget profiles each embedding-only modality with mm_counts={modality: 1} while its limit is 0. A dummy input that gets validated can never pass that check.

4 models including Pixtral override get_dummy_inputs and call self.info.parse_mm_data(dummy_mm_data) with the default validate=True. This PR fixes the processor-side contract violation. Without it, any deployment of these models that relies on embedding-only modalities still crashes at startup, including KV/EC consumers that should receive embeddings.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[NickLucche]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99985 for commit 27a12a3cfb5e.

ℹ️ This PR is 18 commits behind upstream main. Merged upstream main into this branch as 27a12a3cfb5e before starting CI. Pull the branch before pushing more changes.