[VLM] Remove input processor from clip and siglip#13165

simon-mo merged 3 commits intovllm-project:mainvllm-project/vllm:mainfrom

Isotr0py:cleanup-vitIsotr0py/vllm:cleanup-vitCopy head branch name to clipboard