Skip to content

[Model Support] Qwen3-Next - #1068

Merged
yaoyu-33 merged 18 commits into
mainfrom
bridge/qwen3next
Oct 28, 2025
Merged

[Model Support] Qwen3-Next#1068
yaoyu-33 merged 18 commits into
mainfrom
bridge/qwen3next

Conversation

@yaoyu-33

@yaoyu-33 yaoyu-33 commented Oct 23, 2025 •
edited by ananthsub
Loading

Copy link
Copy Markdown
Contributor

part of #786

@copy-pr-bot

copy-pr-bot Bot commented Oct 23, 2025

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 mentioned this pull request Oct 23, 2025
Closed
@yaoyu-33
Merge branch 'main' into qwen3next
e1df237
@yaoyu-33
formatting
dd77d85 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

yaoyu-33 commented Oct 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

torchrun --nproc_per_node=8 examples/conversion/hf_to_megatron_generate_text.py --hf_model_path="Qwen/Qwen3-Next-80B-A3B-Instruct" --prompt="Hello, how are you?" --ep 8
were you able to do any verifications?
Prompt: Hello, how are you?
Generated: Hello, how are you? liberty rig rig rig定arellaijn翻 van-INF定itecture翻定itecture翻翻 rig定itecture

Output non-sense right now

yuzhongw-nvidia and others added 5 commits October 24, 2025 08:02
@yuzhongw-nvidia
fix: (1) fix zero-centered layernorm; (2) remove mtp from functional …
240cf70 
…test
@yaoyu-33
Merge remote-tracking branch 'yuzhongw/qwen3next' into qwen3next
ff65131 
# Conflicts:
#	tests/functional_tests/models/test_qwen3_next_conversion.py
@yaoyu-33
revert perform_initialization since it's fixed in mcore now
f52344e 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33
fix comments
67e6727 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33
build: Adding dependencies
898cba1
@yaoyu-33
yaoyu-33 requested a review from a team as a code owner October 24, 2025 17:54
@yaoyu-33
lint
0362f36 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

yaoyu-33 commented Oct 24, 2025

Copy link
Copy Markdown
Contributor Author

/ok to test 0362f36

@yaoyu-33
fix checkpointing issue
147ed73 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

yaoyu-33 commented Oct 24, 2025

Copy link
Copy Markdown
Contributor Author

/ok to test 147ed73

@yaoyu-33
lint
054120f 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

yaoyu-33 commented Oct 27, 2025

Copy link
Copy Markdown
Contributor Author

/ok to test 054120f

@yaoyu-33

yaoyu-33 commented Oct 27, 2025

Copy link
Copy Markdown
Contributor Author

/ok to test cbfb04b

@yaoyu-33
fix unit tests
1e75af2 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

yaoyu-33 commented Oct 27, 2025

Copy link
Copy Markdown
Contributor Author

/ok to test 1e75af2

@yaoyu-33
yaoyu-33 requested a review from ananthsub October 28, 2025 20:59
@ananthsub ananthsub added the r0.2.0 Cherry-pick label for r0.2.0 release branch label Oct 28, 2025
@ananthsub ananthsub mentioned this pull request Oct 28, 2025
Closed
ko3n1g
ko3n1g requested changes Oct 28, 2025
View reviewed changes

@ko3n1g ko3n1g left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One discussion regarding dependencies is required from my side

Comment thread pyproject.toml
"Topic :: Utilities",
]
dependencies = [
"transformers==4.57.1",

@ko3n1g ko3n1g Oct 28, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just a looser pin than this? If we know that the lower boundary is 4.57.1, I would pledge for >=. Even better would be to have no pin, and to gracefully raise exception on the affected codepaths when transformers is <4.57.1. That way we would be more flexible

ko3n1g
ko3n1g approved these changes Oct 28, 2025
View reviewed changes

@ko3n1g ko3n1g left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline that a follow up PR will follow

@yaoyu-33
yaoyu-33 merged commit 5949b42 into main Oct 28, 2025
34 of 36 checks passed
@yaoyu-33
yaoyu-33 deleted the bridge/qwen3next branch October 28, 2025 22:30
chtruong814 pushed a commit that referenced this pull request Oct 28, 2025
@yaoyu-33 @yuzhongw-nvidia NeMo Bot
[Model Support] Qwen3-Next (#1068)
7eab1fa 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>
Signed-off-by: NeMo Bot <nemo-bot@nvidia.com>
paul-gibbons pushed a commit to paul-gibbons/Megatron-Bridge that referenced this pull request Oct 29, 2025
@yaoyu-33 @yuzhongw-nvidia @paul-gibbons
[Model Support] Qwen3-Next (NVIDIA-NeMo#1068)
5651cd4 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>
Signed-off-by: Paul Gibbons <pgibbons@nvidia.com>
nv-mollys pushed a commit that referenced this pull request Oct 31, 2025
@yaoyu-33 @yuzhongw-nvidia mollys
[Model Support] Qwen3-Next (#1068)
426432f 
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>
Signed-off-by: mollys <mollys@mollys.nvidia.com>
@pruprakash pruprakash mentioned this pull request Apr 23, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ananthsub ananthsub ananthsub approved these changes
@ko3n1g ko3n1g ko3n1g approved these changes

Assignees

No one assigned

Labels

r0.2.0 Cherry-pick label for r0.2.0 release branch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yaoyu-33 @ananthsub @ko3n1g @yuzhongw-nvidia