Skip to content

Commit 08d2bd7

Browse files
authored
[BUGFIX] deepseek-v2-lite failed due to fused_qkv_a_proj name update (#21414)
Signed-off-by: Chendi.Xue <[email protected]>
1 parent 4f76a05 commit 08d2bd7

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

vllm/model_executor/models/deepseek_v2.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -885,13 +885,16 @@ def load_weights(self, weights: Iterable[tuple[str,
885885
# for mlp.experts[0].gate_gate_up_proj, which breaks load.
886886
if (("mlp.experts." in name) and name not in params_dict):
887887
continue
888-
name = name.replace(weight_name, param_name)
888+
name_mapped = name.replace(weight_name, param_name)
889889

890890
# QKV fusion is optional, fall back to normal
891891
# weight loading if it's not enabled
892+
# if go with fusion option, then update name
892893
if ((param_name == "fused_qkv_a_proj")
893-
and name not in params_dict):
894+
and name_mapped not in params_dict):
894895
continue
896+
else:
897+
name = name_mapped
895898
# Skip loading extra bias for GPTQ models.
896899
if name.endswith(".bias") and name not in params_dict:
897900
continue

0 commit comments

Comments
 (0)