[5/N] Initialize MM components in context managers (Q-Z)#32695
[5/N] Initialize MM components in context managers (Q-Z)#32695DarkLight1337 merged 2 commits intovllm-project:mainfrom
Conversation
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
There was a problem hiding this comment.
Code Review
This pull request refactors the initialization of multimodal components across several models (Qwen2-Audio, Qwen3-Omni-MoE-Thinker, Siglip, SkyworkR1V, Tarsier, Ultravox, Voxtral) by introducing _mark_tower_model and _mark_language_model context managers. This change enhances modularity and allows for more flexible management of model sub-components, particularly for scenarios like multimodal encoder-only modes where certain parts of the model might be skipped. The removal of explicit get_language_model methods and some assert statements indicates a shift towards a more streamlined and potentially more robust initialization pattern, where the presence of components is either guaranteed by design or handled gracefully within the new context-managed blocks. The additions of getattr checks for deepstack_input_embeds in qwen3_omni_moe_thinker.py are good for ensuring robustness when components might be conditionally initialized.
…t#32695) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…t#32695) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
…t#32695) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: mohammad najafi <mohammad.najafi@amd.com>
…t#32695) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…t#32695) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Purpose
Part of #32631
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.