Skip to content

Qwen3 Dense#2044

Merged
Borda merged 8 commits intoLightning-AI:mainfrom
ysjprojects:qwen3
May 27, 2025
Merged

Qwen3 Dense#2044
Borda merged 8 commits intoLightning-AI:mainfrom
ysjprojects:qwen3

Conversation

@ysjprojects
Copy link
Collaborator

@ysjprojects ysjprojects commented May 15, 2025

Links:
https://arxiv.org/abs/2505.09388
https://qwenlm.github.io/blog/qwen3/
#2031

Added Qwen3 dense models (Currently Qwen3-8B).

image

NOTE: transformers>=4.51.0 is required (Awaiting #2029)

@Borda
Copy link
Collaborator

Borda commented May 16, 2025

seems you may need to bump the dependency for transformers.models.qwen3

Copy link
Contributor

@KaelanDt KaelanDt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Could you also add other versions of the dense model (not just 8B)? Should just be adding the configs in config.py and modifying the readme

@ysjprojects ysjprojects changed the title Qwen 3 Qwen3 Dense May 17, 2025
@ysjprojects
Copy link
Collaborator Author

Very nice! Could you also add other versions of the dense model (not just 8B)? Should just be adding the configs in config.py and modifying the readme

yes! will do

@Borda Borda requested a review from KaelanDt May 22, 2025 12:11
@ysjprojects
Copy link
Collaborator Author

@Borda @KaelanDt All Qwen3 dense models added, ready for review

@Borda Borda enabled auto-merge (squash) May 24, 2025 06:33
@Borda Borda disabled auto-merge May 27, 2025 10:49
@Borda Borda merged commit 1602e96 into Lightning-AI:main May 27, 2025
24 checks passed
@ysjprojects ysjprojects deleted the qwen3 branch May 29, 2025 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants