Skip to content

[Model] Use explicit types in get_generation_prompt - #33551

Merged
DarkLight1337 merged 2 commits into
vllm-project:mainfrom
DarkLight1337:models-generation-prompt
Feb 2, 2026
Merged

[Model] Use explicit types in get_generation_prompt#33551
DarkLight1337 merged 2 commits into
vllm-project:mainfrom
DarkLight1337:models-generation-prompt

Conversation

@DarkLight1337

@DarkLight1337 DarkLight1337 commented Feb 2, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Member

Purpose

Improve code readability and make it easier to figure out what needs to be updated in case we change the prompt schema during Renderer refactor.

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@DarkLight1337
[Model] Use explicit types in get_generation_prompt
1f717ac 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 2, 2026
@mergify mergify Bot added the qwen Related to Qwen models label Feb 2, 2026
gemini-code-assist[bot]
gemini-code-assist Bot reviewed Feb 2, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the get_generation_prompt method across several multimodal models to use explicit prompt types like TextPrompt, TokensPrompt, and ExplicitEncoderDecoderPrompt instead of generic dictionaries with cast. This is a good change that improves code readability and type safety. The implementation is consistent and well-executed across all modified files. I have no further comments.

@DarkLight1337
DarkLight1337 enabled auto-merge (squash) February 2, 2026 09:43
@DarkLight1337
Handle TextPrompt
da6e5c9 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@mergify mergify Bot added the frontend label Feb 2, 2026
@DarkLight1337
DarkLight1337 merged commit b10d05b into vllm-project:main Feb 2, 2026
52 checks passed
@DarkLight1337
DarkLight1337 deleted the models-generation-prompt branch February 2, 2026 12:38
PiratePai pushed a commit to PiratePai/epd_shm that referenced this pull request Feb 3, 2026
@DarkLight1337 @PiratePai
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
6c9b7b1 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Pai <416932041@qq.com>
PiratePai pushed a commit to PiratePai/epd_shm that referenced this pull request Feb 3, 2026
@DarkLight1337 @PiratePai
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
4eb17ca 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Pai <416932041@qq.com>
gameofdimension pushed a commit to gameofdimension/vllm that referenced this pull request Feb 5, 2026
@DarkLight1337 felix01.yu
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
4b2d503 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: felix01.yu <felix01.yu@vipshop.com>
mystous pushed a commit to mystous/vllm_hybrid that referenced this pull request May 10, 2026
@DarkLight1337
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
66bc99a 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
@DarkLight1337
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
8fd66c1 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
0826joyce pushed a commit to 0826joyce/vllm-serving-optimization that referenced this pull request May 19, 2026
@DarkLight1337
[Model] Use explicit types in get_generation_prompt (vllm-project#3…
e6c4f2a 
…3551)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@ExtReMLapin

ExtReMLapin commented Jun 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@DarkLight1337

I'm reviewing the whisper implementation and git blame points there.

It seems that https://github.com/vllm-project/vllm/blame/main/vllm/model_executor/models/whisper.py#L848 <|prev|> is NOT inside whisper vocab, <|startofprev|> is tho

REF #44662

@DarkLight1337

DarkLight1337 commented Jun 5, 2026

Copy link
Copy Markdown
Member Author

This PR didn't introduce this change, you can check the diffs in this PR

@ExtReMLapin

ExtReMLapin commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Oopsie

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py approved these changes
@NickLucche NickLucche Awaiting requested review from NickLucche
@patrickvonplaten patrickvonplaten Awaiting requested review from patrickvonplaten patrickvonplaten is a code owner
@sighingnow sighingnow Awaiting requested review from sighingnow sighingnow is a code owner
@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner
@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang chaunceyjiang is a code owner
+1 more reviewer
@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

frontend qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DarkLight1337 @ExtReMLapin @Isotr0py