[Test] Verify grok.py rope_theta fix on NVIDIA (draft — do not merge)#21725
Closed
michaelzhang-ai wants to merge 4 commits intosgl-project:mainfrom
Closed
[Test] Verify grok.py rope_theta fix on NVIDIA (draft — do not merge)#21725michaelzhang-ai wants to merge 4 commits intosgl-project:mainfrom
michaelzhang-ai wants to merge 4 commits intosgl-project:mainfrom
Conversation
The Grok-1 HuggingFace config does not define `rope_theta` (it relies on the standard default of 10000). After sgl-project#21135 migrated grok.py from `getattr(config, "rope_theta", 10000)` to `get_rope_config(config)`, loading Grok-1 crashes with: AttributeError: 'Grok1Config' object has no attribute 'rope_theta' Fix by replacing the `get_rope_config()` call in grok.py with local rope_theta extraction that safely falls back to 10000, matching the original behavior before sgl-project#21135. Fixes AMD nightly failures: nightly-8-gpu-grok1-int4 and nightly-8-gpu-mi35x-grok1-int4 (both exit code 255).
…a fix Temporarily re-enable the disabled dummy Grok-1 test on NVIDIA CI (stage-b-test-2-gpu-large) to verify sgl-project#21518's grok.py change doesn't break NVIDIA. This PR is draft-only for CI verification.
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
Author
|
Closing this draft verification PR. Since it comes from a fork branch, we can't reliably run/validate the intended internal NVIDIA stage here. We'll keep verification on the main fix PR path instead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Draft PR to run NVIDIA CI and verify that PR #21518's
grok.pychange (saferope_thetafallback for Grok-1 INT4) does not break NVIDIA.Do not merge — this only re-enables the disabled dummy Grok-1 test for CI verification.
Changes
grok.py— from [AMD] Fix Handle missing rope_theta in get_rope_config for Grok-1 #21518: replacesget_rope_config(config)with safegetattr+ fallback forGrok1Configwhich lacksrope_thetatest_dummy_grok_models.py— removesdisabled="Temporarily disabled"to run dummy Grok-1 onstage-b-test-2-gpu-largeExpected result
stage-b-test-2-gpu-large→test_dummy_grok_1passes (usesmodel_type: "mixtral"→MixtralConfigwithrope_thetadefault).