Skip to content

Commit b6ccf22

Browse files
authored
Fix num of layers for deepseek-v3 (#1845)
Fix the number of layer issue introduced by #1804
1 parent 44e9218 commit b6ccf22

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

torchtitan/models/deepseek_v3/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@
134134
dim=7168,
135135
inter_dim=18432,
136136
moe_inter_dim=2048,
137-
n_layers=4,
137+
n_layers=61,
138138
n_dense_layers=3,
139139
n_heads=128,
140140
moe_args=MoEArgs(

0 commit comments

Comments
 (0)