Skip to content

[Multimodal] Generate mm_hash based on request metadata when caching is turned off#23690

Merged
ywang96 merged 16 commits intovllm-project:mainfrom
ywang96:gen-uuid-when-caching-off
Aug 27, 2025
Merged

[Multimodal] Generate mm_hash based on request metadata when caching is turned off#23690
ywang96 merged 16 commits intovllm-project:mainfrom
ywang96:gen-uuid-when-caching-off

Conversation

@ywang96
Copy link
Member

@ywang96 ywang96 commented Aug 26, 2025

Purpose

Since #23308 and #22711, mm_hash is a required information required by the engine core. This introduced a regression when users explicitly specify enable_prefix_caching=False and disable_mm_preprocessor_cache=True (now as mm_processor_cache_gb=0) since previously mm_data will not be hashed under this scenario but now they will.

This PR add mm_hash_overrides generated based on request_id, modality string and index of the item in the modality list as the identifier of the item so that hashing is no longer required since request_id is considered unique inside vLLM.

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Roger Wang <hey@rogerw.io>
@mergify mergify bot added multi-modality Related to multi-modality (#4194) v1 labels Aug 26, 2025
Signed-off-by: Roger Wang <hey@rogerw.io>
@ywang96 ywang96 removed the request for review from patrickvonplaten August 27, 2025 03:23
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if tests pass

ywang96 added 10 commits August 27, 2025 04:11
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
@ywang96 ywang96 enabled auto-merge (squash) August 27, 2025 17:25
@ywang96 ywang96 merged commit 8bf6266 into vllm-project:main Aug 27, 2025
41 checks passed
@ywang96 ywang96 deleted the gen-uuid-when-caching-off branch August 27, 2025 20:56
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
…is turned off (vllm-project#23690)

Signed-off-by: Roger Wang <hey@rogerw.io>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Sep 3, 2025
…is turned off (vllm-project#23690)

Signed-off-by: Roger Wang <hey@rogerw.io>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
…is turned off (vllm-project#23690)

Signed-off-by: Roger Wang <hey@rogerw.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek Related to DeepSeek models llama Related to Llama models multi-modality Related to multi-modality (#4194) ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants