Skip to content

feat: Support qwen3-next, mcore path #1510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 30 commits into from
Closed

feat: Support qwen3-next, mcore path #1510

wants to merge 30 commits into from

Conversation

@ahmadki
Copy link
Member

@ahmadki ahmadki commented Nov 12, 2025 •
edited
Loading

What does this PR do ?

This is a draft PR to support qwen3-next with mcore path. Although the code works, this PR is still a WIP and will be replaced by another PR once work is finalized. The reamining TODO items are:

  • merge some necessary fixes in autmodel and megatron
  • fix the sliding_window assertion
  • tune configs and hparams, collect "release reference" runs and update the test metrics

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

yaoyu-33 and others added 29 commits November 5, 2025 17:44
@yaoyu-33 @ahmadki
update to use mixed_precision_wrapper api
ee2c41c
@yaoyu-33 @ahmadki
Update refit_verifier.py
9971111 
Signed-off-by: Yu Yao <[email protected]>
@yaoyu-33 @ahmadki
update setup.py
b552ee6
@guyueh1 @coderabbitai @ahmadki
fix: Fix policy worker placement when using unified placement group (#…
936208f 
…1341)

Signed-off-by: Guyue Huang <[email protected]>
Signed-off-by: Guyue Huang <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@yaoyu-33 @ahmadki
update setup.py
e79dc87
@yaoyu-33 @ahmadki
revert setup.py change for megatron-bridge
188234e
@yaoyu-33 @ahmadki
update run_mcore_engine import path
55fa4a6
@terrykong @ahmadki
chore: major version bump (torch 2.8, vllm 0.11, ray 2.49) & SP fixes (…
66ca65d 
…#1334)

Signed-off-by: Terry Kong <[email protected]>
@terrykong @ahmadki
fix dependency issues
6b0f4a9 
Signed-off-by: Terry Kong <[email protected]>

fix more stuff

Signed-off-by: Terry Kong <[email protected]>

fix

Signed-off-by: Terry Kong <[email protected]>
@ahmadki
grpo qwen3-next initial config
1cc6c0c 
Signed-off-by: Ahmad Kiswani <[email protected]>
@ahmadki
added local env variables files to gitignore
3a15db4 
Signed-off-by: Ahmad Kiswani <[email protected]>
@ahmadki
updated transformers version (needed for qwen_next) support
51d42e7
@ahmadki
sft config
b9c6f6b
@ahmadki
disabled sliding_window assertion for now
0b3a5ab
@ahmadki
mbridge with qwen3-next adapter
9dc285c
@ahmadki
use mbridge main branch
576cad4
@ahmadki
megatron dependencies
6efc4e3
@ahmadki
automodel version updates
d315135
@ahmadki
update git modules
430c071
@ahmadki
own repo of Automodel
28d8324
@ahmadki
updated configs
16f11a3
@ahmadki
fixed to mbridge imports
7094a7a
@ahmadki
config update
a0b9c98
@ahmadki
config fixes
4991047
@ahmadki
sft: enabled expert_model_parallel
1fd30cd
@ahmadki
tests + minimize configs
a522c9a
@ahmadki
temp fixes to dockerfile
c10604b
@ahmadki
revert Docker changes
274b535
@ahmadki
enabled checkpointing by default
ad0233e
@ahmadki ahmadki mentioned this pull request Nov 12, 2025
Open
@ahmadki
Copy link
Member Author

ahmadki commented Nov 17, 2025

Closing in favor of: #1530

@ahmadki ahmadki closed this Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ahmadki @yaoyu-33 @guyueh1 @terrykong