[Bugfix] Fix Plamo3 rope handling#29092
Conversation
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
There was a problem hiding this comment.
Code Review
This pull request aims to fix a bug in the Plamo3 model's RoPE handling. While the change fixes a TypeError from an updated function signature, it introduces a critical correctness bug by removing layer-specific logic for rope_theta. My review includes a comment with a code suggestion to fix this new bug by correctly preserving the dynamic rope_theta selection.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@Alnusjaponica can you check if this works? I copied the RoPE initialization code from Gemma3 model. |
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
Is Plamo2's config implemented in vLLM? If yes it can be updated to create a nested |
|
No, it's only in vLLM's modeling file for type annotation purposes |
hmellor
left a comment
There was a problem hiding this comment.
Overall LGTM, just one comment on something I missed in Gemma 3 which someone has a fix for.
Side notes about v5 withthis model:
- It look like it's a custom model, which means that unless the model authors update their repo, it will never support v5 style RoPE.
- However, including the v5 block could be a good way to convince them to update it, because there would be no downtime where their model was unsupported.
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Purpose
Fix a failure in model initialization tests caused by an incompatibility between #28834 and #28542 .
https://buildkite.com/vllm/ci/builds/39885#019aa0ed-01f7-4df6-a93b-2d95f144776c
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.