Fix Mistral yarn warning in Transformers v5#37292
Fix Mistral yarn warning in Transformers v5#37292hmellor wants to merge 3 commits intovllm-project:mainfrom
Conversation
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
@juliendenize could you please confirm that this fix works for you? |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a warning from newer versions of the transformers library for Mistral models using YaRN scaling. It does so by adding ignore_keys_at_rope_validation to the model configuration. My review suggests adding a version guard to this change to ensure compatibility across different transformers versions.
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
juliendenize
left a comment
juliendenize
left a comment
There was a problem hiding this comment.
Hey sorry took a bit of time to answer was off for two days 😄
I had to change to make it >5.2.0 for it to work, hence my suggestion. This is due to 5.3.0.dev being < 5.3.0.
Thanks for finding a clean, Transformers native, solution !
| config["rope_parameters"][new_name] = cast(yarn_config.pop(old_name)) | ||
|
|
||
| # Ignore apply_yarn_scaling in Transformers > v5 RoPE validation to remove warnings | ||
| if Version(TRANSFORMERS_VERSION) >= Version("5.3.0"): |
There was a problem hiding this comment.
| if Version(TRANSFORMERS_VERSION) >= Version("5.3.0"): | |
| if Version(TRANSFORMERS_VERSION) >= Version("5.3.0.dev0"): |
There was a problem hiding this comment.
Unfortunately if we use 5.2 then there will be JSON serialisation errors because ignore_keys_at_rope_validation is a set.
Does it work for you if you use 5.3.0.dev?
There was a problem hiding this comment.
Yep i updated the suggestion 😄
2 participants
As of huggingface/transformers#41250 the
ignore_keysargument tovalidate_ropewas removed in favour ofClassVars attached to the config classes themselves.This PR sets this new
ClassVarfor Mistral models which useyarnso that the warning from Transformers RoPE validation is suppressed.cc @juliendenize