[Bugfix][Rust Frontend][Multimodal] Align DeepSeek V4.1 and Kimi K3 media with rendered placeholders - #56366
Merged
BugenZhao merged 4 commits intoSep 14, 2026
Conversation
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
approved these changes
Sep 14, 2026
BugenZhao
left a comment
Member
There was a problem hiding this comment.
Thanks @reidliu41 this is a good catch! I've added a commit to put media_order into RenderedPrompt and use None for default ordering.
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Member
|
/ci run |
BugenZhao
enabled auto-merge (squash)
September 14, 2026 09:55
|
✅ @reidliu41, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #88787 for commit |
Shreya-gaur
pushed a commit
to Shreya-gaur/vllm_private
that referenced
this pull request
Sep 14, 2026
…edia with rendered placeholders (vllm-project#56366) Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Codex <noreply@openai.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
ItsRoy69
pushed a commit
to ItsRoy69/vllm
that referenced
this pull request
Sep 15, 2026
…edia with rendered placeholders (vllm-project#56366) Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Codex <noreply@openai.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Rust chat rendering and multimodal extraction previously determined media
order independently.
DeepSeek V4.1 and Kimi K3 can reorder tool results to match the preceding
assistant tool-call order. The rendered image placeholders therefore followed
tool-call order, while media extraction still followed raw request order. When
tool results arrived out of order, images could be silently associated with
the wrong tool results.
DeepSeek V4.1 can also omit historical developer messages. Images from those
messages were still extracted even though no corresponding placeholder was
rendered, causing multimodal placeholder expansion to fail.
Make renderers report media source locations while emitting placeholders, and
use that order during multimodal extraction. DeepSeek V4.1 and Kimi K3 provide
their exact rendered media order. Renderers that preserve request order retain
the existing behavior through the default implementation.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.