[model] fix: use local GPT spec for CPU-only initialization - #5317
Closed
cuichenx wants to merge 1 commit into
Closed
[model] fix: use local GPT spec for CPU-only initialization#5317cuichenx wants to merge 1 commit into
cuichenx wants to merge 1 commit into
Conversation
Avoid constructing CUDA-only Transformer Engine modules when GPT models are initialized on a host without visible CUDA. Keep the TE path unchanged when CUDA is available, and propagate the same backend choice to MTP specs. This revives the narrow spec-selection portion of #3853 now that full-model CPU import and GPU/TE checkpoint-load compatibility are explicit validation gates. Co-authored-by: Yu Yao <54727607+yaoyu-33@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
cuichenx
added
area:ckpt
This was referenced Aug 4, 2026
Contributor
Author
|
Closing this standalone approach. Selecting local GPT/MTP specs does not by itself solve driverless Transformer Engine import, and local-to-TE checkpoint schema compatibility is not guaranteed across model families. Any future CPU-only conversion fix should be validated as an end-to-end design rather than landing this isolated selector. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What
use_cpu_initialization=Trueand CUDA is unavailable, avoiding CUDA-only Transformer Engine module
construction on CPU-only hosts.
This revisits the narrow spec-selection portion of #3853 in response to a real
full
openai/gpt-oss-20bCPU-import failure. It does not revive that closed PR'sbroader provider monkeypatches.
Compatibility gate
The local-spec versus later GPU/TE checkpoint compatibility concern that closed
#3853 is treated as an acceptance gate, not assumed away. Current MCore provides
canonical sharded-state mappings and direct local-GPT-spec ↔ TE-GPT-spec
save/load coverage, including SequentialMLP ↔ TEGroupedMLP interchange.
Before this PR is called verified, a full GPT-OSS checkpoint produced by the
CPU-only local-spec path must load into the GPU/TE model from exact clean pushed
commits. If it does not, this approach is not acceptable.
Validation
tests/unit_tests/models/test_gpt_provider.py: 29 passed in adriverless 26.06 container.
uv run --no-sync pre-commit run --all-files.openai/gpt-oss-20bCPU import from a clean public integrationcommit.
No tests will be quarantined without Chen's explicit approval. No local source
patch or unpublished overlay may be counted as verification evidence.