Fix RoPE related failures in Transformers nightly tests#29333
Fix RoPE related failures in Transformers nightly tests#29333hmellor merged 5 commits intovllm-project:mainfrom
Conversation
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request aims to fix an issue with RoPE parameter handling for nightly builds of the transformers library. The change correctly identifies the need to move rope_theta into the rope_parameters dictionary for newer transformers versions. However, the current implementation has a flaw in how it handles interleaved models, which could lead to runtime errors. I've provided a critical review comment with a code suggestion to address this potential bug and ensure the fix is robust for different model architectures.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
hmellor
changed the title
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
…#29333) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
…#29333) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
…#29333) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
2 participants
Should fix the RoPE related failures in the Transformers nightly tests by:
rope_thetawhen loading v4 custom models in v5original_max_position_embeddingsfor v4 (all models) and v5 (v4 custom models)config.rope_parametersif it was present in the configconfig.rope_parametersis always available in model codeTest plan: