Skip to content

fix(vllm): expand Kimi K3 media pads at the worker - #12394

Merged
furionw merged 1 commit into
mainfrom
qiwa/k3-vllm-media-pad
Jul 30, 2026
Merged

fix(vllm): expand Kimi K3 media pads at the worker#12394
furionw merged 1 commit into
mainfrom
qiwa/k3-vllm-media-pad

Conversation

@furionw

@furionw furionw commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • convert each native frontend media pad into the checkpoint Kimi image-placeholder token sequence at the vLLM adapter boundary
  • count image or unified vision-chunk payloads for scalar and list media
  • reject pad/media mismatches and invalid K3 metadata instead of silently misaligning inputs
  • cache only successful K3 resolution and definite non-K3 results
  • retain the sparse splice path for very long prompts

Stack

Base: PR1, qiwa/k3-native-frontend

Validation

  • focused pre-commit hooks
  • pre-commit run --all-files
  • Python bytecode compilation
  • unit coverage for single and multiple images, vision_chunk, mismatch, rollout compatibility, failures, caching, and long prompts

Runtime qualification follows this branch using vllm/vllm-openai:kimi-k3.

Summary by CodeRabbit

  • New Features

    • Added support for Kimi K3 multimodal prompts by converting structural media placeholders into the format expected by the model.
    • Supports prompts containing multiple images and vision chunking.
  • Bug Fixes

    • Added validation for mismatched image placeholders and provided images.
    • Preserves prompts that are already correctly formatted.
    • Safely avoids rewriting requests when required media or model metadata is unavailable.

@datadog-official

This comment has been minimized.

@furionw
furionw marked this pull request as ready for review July 29, 2026 22:18
@furionw
furionw requested review from a team as code owners July 29, 2026 22:18

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread components/src/dynamo/vllm/multimodal_utils/request_processor.py
Base automatically changed from qiwa/k3-native-frontend to main July 30, 2026 04:21
@furionw
furionw requested review from a team as code owners July 30, 2026 04:21
@rmccorm4
rmccorm4 enabled auto-merge (squash) July 30, 2026 04:42
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Kimi K3 prompt expansion

Layer / File(s) Summary
Pad resolution and token splicing
components/src/dynamo/vllm/multimodal_utils/request_processor.py
Adds cached Kimi K3 metadata resolution, structural-pad validation, and replacement of media pads with native image placeholder token ids.
Prompt builder integration
components/src/dynamo/vllm/multimodal_utils/request_processor.py
Routes token ids through Kimi K3 expansion during prompt construction.
Expansion behavior validation
components/src/dynamo/vllm/tests/multimodal_utils/test_vllm_request_processor.py
Adds coverage for expansion, mismatches, metadata failures, caching, no-op cases, unified vision chunks, and long prompts.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers the change and validation, but it misses required template sections like Overview, Details, reviewer start, and Related Issues. Reformat the PR body to the required template and add the missing sections, especially reviewer guidance and either a linked issue or the no-issue checkbox.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change: expanding Kimi K3 media pads in the vLLM worker.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

@rmccorm4
rmccorm4 disabled auto-merge July 30, 2026 04:59
@furionw
furionw force-pushed the qiwa/k3-vllm-media-pad branch from ab434f7 to 3d50424 Compare July 30, 2026 17:30

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread components/src/dynamo/vllm/multimodal_utils/request_processor.py
@furionw
furionw removed request for a team July 30, 2026 18:09
@furionw
furionw enabled auto-merge (squash) July 30, 2026 18:10
@furionw
furionw merged commit 0d5034a into main Jul 30, 2026
106 checks passed
@furionw
furionw deleted the qiwa/k3-vllm-media-pad branch July 30, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants