[Perf] Avoid repeated multimodal prompt update scans - #51774
Merged
DarkLight1337 merged 3 commits intoAug 11, 2026
Merged
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai>
gty111
requested review from
DarkLight1337,
NickLucche,
shen-shanshan,
tjtanaa and
ywang96
as code owners
August 11, 2026 06:57
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai>
DarkLight1337
enabled auto-merge (squash)
August 11, 2026 07:32
Member
|
/ci run |
|
✅ Triggered Buildkite CI #83315 for commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Avoid quadratic prompt-update planning when many multimodal items share the
same target.
For non-empty replacements, the previous implementation applies one item per
round. Each round still walks all unresolved items and rebuilds the match list,
resulting in roughly O(N²) item processing.
This change compiles updates with identical ordered
(mode, target)choicesinto FIFO queues. Matching only examines the next item in each queue and
consumes items in their original priority order. The common shared-target path
becomes O(prompt length + output length + item count).
Alternative targets, insertion and replacement modes,
PromptIndex,overlapping matches, and modality priority retain their existing behavior.
This is not a duplicate: no open PR addresses repeated shared-target update
rounds; #50716 optimizes individual scans but does not remove the per-item
rounds.
AI assistance was used.
Performance
Synthetic
apply_token_matchesbenchmark:This is approximately a 995x speedup. The expanded tokens and match results
are identical.
Test Plan
.venv/bin/python -m pytest tests/multimodal/test_processing.py -q .venv/bin/python -m pytest tests/models/multimodal/processing/test_common.py \ -k 'Qwen2-VL or llava-1.5 or SmolVLM2-2.2B' -q pre-commit run ruff-check --files \ vllm/multimodal/processing/processor.py tests/multimodal/test_processing.py pre-commit run ruff-format --files \ vllm/multimodal/processing/processor.py tests/multimodal/test_processing.py pre-commit run mypy-3.12 --files \ vllm/multimodal/processing/processor.py tests/multimodal/test_processing.py \ --hook-stage manualTest Result