[Bugfix] Implement multimodal_cpu_fields in model runner

[zhangxinyuehfad]

What this PR does / why we need it?

Related to #4084 Implement multimodal_cpu_fields in model runner

Does this PR introduce any user-facing change?

How was this patch tested?

• vLLM version: v0.12.0
• vLLM main: vllm-project/vllm@ad32e3e

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to ensure specific multimodal data fields are on the CPU before being used by the model, which is a good refactoring of model-specific logic into a more general component. My main feedback is to make the list of CPU-bound fields configurable rather than hardcoded to improve maintainability and support for future models.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[shen-shanshan]

Please delete the whole file patch_qwen3_vl.py.

[shen-shanshan]

According to vllm-project/vllm#28168, I suppose this multimodal_cpu_fields is different for every model, we should not just use one multimodal_cpu_fields for all MM models.

Besides, we should avoid letting users to manually configure this by passing an additional_config, which could lead to an awful user experience.

[shen-shanshan]

Currently, this multimodal_cpu_fields is only used by Qwen series VL models, and the multimodal_cpu_fields of these models are all the same. Thus, your resolution could work temporarily, but I suppose this is not a good choice to implement the function like this for long term.

[shen-shanshan]

LGTM if all CI passed.

Note that we need to remove this logic after this field is deprecated in vllm.

[zhangxinyuehfad]

Please delete the whole file patch_qwen3_vl.py.

done

[zhangxinyuehfad]

According to vllm-project/vllm#28168, I suppose this multimodal_cpu_fields is different for every model, we should not just use one multimodal_cpu_fields for all MM models.

Besides, we should avoid letting users to manually configure this by passing an additional_config, which could lead to an awful user experience.

Yes, I deleted multimodal_cpu_fields from additional_config.

[shen-shanshan]

CC @wangxiyuan All CI passed.