fix: use get_rope_config() to support models without rope_parameters#21135
fix: use get_rope_config() to support models without rope_parameters#21135Fridge003 merged 10 commits intosgl-project:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a compatibility issue that arose after a recent Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request correctly addresses an AttributeError for trust-remote-code models by introducing a helper function, get_rope_config, to robustly access RoPE parameters. The changes improve compatibility with different Hugging Face transformers versions. I have one suggestion to make the logic for retrieving partial_rotary_factor more robust against falsy values like 0.
alphabetc1
force-pushed
the
fix/missing_repo_parameters
branch
from
ae0f0cf to
b584dd1
Compare
alphabetc1
changed the title
alphabetc1
force-pushed
the
fix/missing_repo_parameters
branch
from
b584dd1 to
0a1d7a2
Compare
|
/tag-and-rerun-ci |
|
/rerun-failed-ci |
1 similar comment
|
/rerun-failed-ci |
|
/rerun-failed-ci |
The `get_rope_config()` helper crashes with `AttributeError: 'Grok1Config' object has no attribute 'rope_theta'` because the Grok-1 HuggingFace config does not define `rope_theta` (it relies on the standard default of 10000). This was introduced in #21135 which migrated grok.py from a safe `getattr(config, "rope_theta", 10000)` to `get_rope_config(config)`, but the v4 fallback path in `get_rope_config()` accesses `config.rope_theta` directly without `getattr`. Fix: - Use `getattr(config, "rope_theta", None)` in `get_rope_config()` so it never crashes on configs missing the attribute - Restore the Grok-1 default of 10000 in `grok.py` when rope_theta is None Fixes AMD nightly failures: nightly-8-gpu-grok1-int4 and nightly-8-gpu-mi35x-grok1-int4 (both exit code 255).
The Grok-1 HuggingFace config does not define `rope_theta` (it relies on the standard default of 10000). After #21135 migrated grok.py from `getattr(config, "rope_theta", 10000)` to `get_rope_config(config)`, loading Grok-1 crashes with: AttributeError: 'Grok1Config' object has no attribute 'rope_theta' Fix by replacing the `get_rope_config()` call in grok.py with local rope_theta extraction that safely falls back to 10000, matching the original behavior before #21135. Fixes AMD nightly failures: nightly-8-gpu-grok1-int4 and nightly-8-gpu-mi35x-grok1-int4 (both exit code 255).
3 participants
Motivation
PR #17784 ("Upgrade transformers==5.3.0") batch-replaced
getattr(config, "rope_theta", ...) / getattr(config, "rope_scaling", ...)with directconfig.rope_parameters["rope_theta"].This works for built-in HuggingFace config classes (e.g. LlamaConfig) where
rope_parametersis defined in__init__, but breaks fortrust-remote-codemodels like GLM-4-MoE.Affected trust-remote-code models:
glm4, glm4_moe, ernie4, exaone, hunyuan, minicpm, minicpm3,xverse, xverse_moe, baichuan, orion, deepseek, qwen, grok, solar, iquest_loopcoder, llada2, step3_vl
Modifications
Accuracy Tests
Benchmarking and Profiling
Checklist
Review Process
/tag-run-ci-label,/rerun-failed-ci,/tag-and-rerun-ci