fix(glm5): support FP8 context-parallel launches - #17
Closed
JackRao123 wants to merge 7 commits into
Closed
Conversation
pstefa1707
reviewed
Jul 10, 2026
Comment on lines
+120
to
+121
| logger.warning( | ||
| "Skipping the GLM-5 qk head-dim workaround (raw config.json " |
Collaborator
There was a problem hiding this comment.
Lets just error in this case
The per-block Python loop made GLM-5.2-FP8's 800B load CPU-bound (8 workers pegged ~50 min). Two repeat_interleaves + one multiply, verified bit-exact against the loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The qk_rope_head_dim workaround (transformers GlmMoeDsaConfig collapses the qk head-dim split, corrupting kv_a_proj shapes: 704 != 576) only read <base_model>/config.json as a filesystem path, so it silently no-opped when base_model is a hub repo id and the 800B weight load failed. Resolve the raw config through hf_hub_download when the local read misses — transformers has already cached config.json by the time the bridge runs, so this works offline (HF_HUB_OFFLINE) too. Warn loudly when neither path resolves. Prod launches GLM-5.2-FP8 by HF id, so this unblocks the registry row in trainers#592. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JackRao123
force-pushed
the
jackrao/glm-cp-bridge-20260907
branch
from
July 12, 2026 00:45
fa04a14 to
8e6e368
Compare
Keep the Bridge submodule aligned with the current LM#14 head before the dependent Bridge and Trainers PRs merge. Signed-off-by: Jack Rao <jack.rao@baseten.co>
The raw config preserves GLM qk dimensions, so conversion must terminate instead of continuing with invalid MLA shapes. Remove redundant comments. Signed-off-by: Jack Rao <jack.rao@baseten.co> Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the Bridge submodule aligned with the current LM#14 head before the dependent PRs merge. Signed-off-by: Jack Rao <jack.rao@baseten.co>
JackRao123
changed the title
State the Transformer configuration defect and why the raw config is required. Signed-off-by: Jack Rao <jack.rao@baseten.co> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prepare the GLM-5.2 FP8 Bridge path for the 131k context-parallel trainer:
base_modelis a hub ID, and fail before conversion if it is unavailable. The raw config preserves the qk head-dimension split required for valid weight shapes.3rdparty/Megatron-LMto Megatron-LM#14 (a1fab1bb), which fixes packed-CP DSA indexer causal-mask alignment.Validation
Integration target and merge order
The deleted
trainers-main-20260907target has been replaced bytrainers-mainat the identical base commit (8e2d2db), so this branch needs no rebase.This PR and Megatron-LM#14 can be reviewed in parallel, but merge LM#14 first. If LM#14 is squash- or rebase-merged, this branch will repin
3rdparty/Megatron-LMto the landed LMtrainers-mainSHA before it merges. Trainers will then advance its Bridge gitlink.Dependency
Merge Megatron-LM#14 before this PR; the pinned commit is its current head.