Update deepseek recipes to use bridge, add recipe tests - #895
Conversation
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test 5f31518 |
ananthsub
left a comment
ananthsub
left a comment
There was a problem hiding this comment.
it doesn't look like the configs map 1:1 with what was previously there
|
/ok to test b96243e |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test 98ed0fa |
|
/ok to test c1e76ee |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test c5e065f |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/ok to test d718b2b |
|
/ok to test 85779ee |
@yaoyu-33, there was an error processing your request: See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/ |
|
/ok to test 85779ee |
| from megatron.bridge.training.deepep import validate_deepep as _validate_deepep | ||
|
|
||
| _apply_deepep(model_cfg) | ||
| _validate_deepep(model_cfg) |
There was a problem hiding this comment.
don't need to validate this explicitly - it's called here:
Megatron-Bridge/src/megatron/bridge/training/config.py
Lines 1219 to 1220 in 6476ea0
| # ( | ||
| # deepseek_v3_config, | ||
| # "deepseek_v3", | ||
| # {"tensor_parallelism": 2, "pipeline_parallelism": 1, "expert_parallelism": 1}, | ||
| # { | ||
| # "num_layers": 2, | ||
| # "num_moe_experts": 8, | ||
| # "moe_router_topk": 1, | ||
| # "moe_layer_freq": [0, 1], | ||
| # "pipeline_model_parallel_layout": [["embedding"] + ["decoder"] * 2 + ["mtp", "loss"]], | ||
| # }, | ||
| # ), |
There was a problem hiding this comment.
why are these commented out?
There was a problem hiding this comment.
takes longer to finish in ci, need to revisit this
2 participants
No description provided.