[Renderer] Move MM Hash parsing into Renderer#34711
[Renderer] Move MM Hash parsing into Renderer#34711vllm-bot merged 5 commits intovllm-project:mainfrom
Conversation
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
There was a problem hiding this comment.
Code Review
This pull request refactors the parsing of MultiModalUUIDDict to MultiModalUUIDItems from the multimodal processor to the renderer, which is a good architectural improvement. The changes are propagated through many files, and the refactoring itself seems solid. However, I've identified a critical issue where tokenization_kwargs are no longer included in the multimodal cache hash computation. This could lead to cache corruption and incorrect model outputs. I've provided a detailed comment and a suggested fix for this issue.
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
Hi @DarkLight1337, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Purpose
Follow-up to #34560
MultiModalUUIDDict -> MultiModalUUIDItemsin Renderer instead of MM processor.tokenization_kwargsfrom MM hash calculation as the MM data isn't affected by that.MultiModalProcessor.applyargument list to put UUIDs before miscellaneous kwargs._is_emptycheck with modality selection inget_dummy_mm_inputs.Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.