-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mixtral to NeMo conversion script. #8155
Mixtral to NeMo conversion script. #8155
Conversation
nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py
Outdated
Show resolved
Hide resolved
85a4249
to
eccdece
Compare
Could you provide the convert NeMo to huggingface script? |
@cdj0311 will try to have the NeMo to HF by next week |
eccdece
to
e32c3f2
Compare
10804f8
to
e22b72e
Compare
65c3935
to
3f985b6
Compare
jenkins |
4c45018
to
a5ba328
Compare
jenkins |
Signed-off-by: Alexandros Koumparoulis <[email protected]>
d70c027
to
d926b1a
Compare
jenkins |
1 similar comment
jenkins |
…cify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]>
aa09f9e
to
171f37f
Compare
Signed-off-by: Alexandros Koumparoulis <[email protected]>
for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]>
b3af500
to
02a2a2f
Compare
jenkins |
Signed-off-by: Alexandros Koumparoulis <[email protected]>
jenkins |
33a71fd
to
2ee101b
Compare
Signed-off-by: Alexandros Koumparoulis <[email protected]>
f3c8be2
to
efdd42e
Compare
jenkins |
jenkins |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks!
* HF-Mixtral to NeMo conversion script. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Pass MoE options from NeMo config to TransformerConfig. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Add version check for get_gpt_layer_with_transformer_engine_spec Signed-off-by: Alexandros Koumparoulis <[email protected]> * Determine MoE support by attempting to import MoETokenDispatcher. Using importlib.metadata.version would be an alternative, however, a) it requires having mcore installed via pip (not always the case) and b) one might override megatron's location (e.g. via PYTHONPATH) and as a result would get inaccurate version from importlib.metadata. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-NeMo to Mixtral-HF converter. Signed-off-by: Alexandros Koumparoulis <[email protected]> * fixup: Update mcore_supports_moe due to file rename in upcoming MoE Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-converters: use `set_expert_model_parallel_world_size` to specify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Fix import Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]> * Jenkins: install lightning. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Match latest MoE parameter names. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]>
* HF-Mixtral to NeMo conversion script. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Pass MoE options from NeMo config to TransformerConfig. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Add version check for get_gpt_layer_with_transformer_engine_spec Signed-off-by: Alexandros Koumparoulis <[email protected]> * Determine MoE support by attempting to import MoETokenDispatcher. Using importlib.metadata.version would be an alternative, however, a) it requires having mcore installed via pip (not always the case) and b) one might override megatron's location (e.g. via PYTHONPATH) and as a result would get inaccurate version from importlib.metadata. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-NeMo to Mixtral-HF converter. Signed-off-by: Alexandros Koumparoulis <[email protected]> * fixup: Update mcore_supports_moe due to file rename in upcoming MoE Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-converters: use `set_expert_model_parallel_world_size` to specify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Fix import Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]> * Jenkins: install lightning. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Match latest MoE parameter names. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Signed-off-by: stevehuang52 <[email protected]>
Hi @ericharper. I browsed CodeQL suggestions in this pull request. This is a good example when Flake8 is better than CodeQL: CodeQL fails to detect undefined variables (a simple issue), see |
Another example are the unused imports |
* HF-Mixtral to NeMo conversion script. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Pass MoE options from NeMo config to TransformerConfig. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Add version check for get_gpt_layer_with_transformer_engine_spec Signed-off-by: Alexandros Koumparoulis <[email protected]> * Determine MoE support by attempting to import MoETokenDispatcher. Using importlib.metadata.version would be an alternative, however, a) it requires having mcore installed via pip (not always the case) and b) one might override megatron's location (e.g. via PYTHONPATH) and as a result would get inaccurate version from importlib.metadata. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-NeMo to Mixtral-HF converter. Signed-off-by: Alexandros Koumparoulis <[email protected]> * fixup: Update mcore_supports_moe due to file rename in upcoming MoE Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-converters: use `set_expert_model_parallel_world_size` to specify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Fix import Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]> * Jenkins: install lightning. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Match latest MoE parameter names. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Signed-off-by: Sasha Meister <[email protected]>
* HF-Mixtral to NeMo conversion script. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Pass MoE options from NeMo config to TransformerConfig. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Add version check for get_gpt_layer_with_transformer_engine_spec Signed-off-by: Alexandros Koumparoulis <[email protected]> * Determine MoE support by attempting to import MoETokenDispatcher. Using importlib.metadata.version would be an alternative, however, a) it requires having mcore installed via pip (not always the case) and b) one might override megatron's location (e.g. via PYTHONPATH) and as a result would get inaccurate version from importlib.metadata. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-NeMo to Mixtral-HF converter. Signed-off-by: Alexandros Koumparoulis <[email protected]> * fixup: Update mcore_supports_moe due to file rename in upcoming MoE Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-converters: use `set_expert_model_parallel_world_size` to specify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Fix import Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]> * Jenkins: install lightning. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Match latest MoE parameter names. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Signed-off-by: Pablo Garay <[email protected]>
* HF-Mixtral to NeMo conversion script. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Pass MoE options from NeMo config to TransformerConfig. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Add version check for get_gpt_layer_with_transformer_engine_spec Signed-off-by: Alexandros Koumparoulis <[email protected]> * Determine MoE support by attempting to import MoETokenDispatcher. Using importlib.metadata.version would be an alternative, however, a) it requires having mcore installed via pip (not always the case) and b) one might override megatron's location (e.g. via PYTHONPATH) and as a result would get inaccurate version from importlib.metadata. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-NeMo to Mixtral-HF converter. Signed-off-by: Alexandros Koumparoulis <[email protected]> * fixup: Update mcore_supports_moe due to file rename in upcoming MoE Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mixtral-converters: use `set_expert_model_parallel_world_size` to specify MoE world size. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Fix import Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Alexandros Koumparoulis <[email protected]> * Jenkins: install lightning. Signed-off-by: Alexandros Koumparoulis <[email protected]> * Match latest MoE parameter names. Signed-off-by: Alexandros Koumparoulis <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]>
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use this
Jenkins CI
To run Jenkins, a NeMo User with write access must comment
jenkins
on the PR.Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information