Skip to content

Commit

Permalink
remove duplicate 340b params
Browse files Browse the repository at this point in the history
Signed-off-by: Malay Nagda <[email protected]>
  • Loading branch information
malay-nagda committed Jan 6, 2025
1 parent 5730fac commit 10120db
Showing 1 changed file with 0 additions and 2 deletions.
2 changes: 0 additions & 2 deletions launcher_scripts/conf/training/nemotron/nemotron_340b.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,6 @@ model:
defer_embedding_wgrad_compute: True
wgrad_deferral_limit: 22
cross_entropy_loss_fusion: True
enable_vboost: True
ub_tp_comm_overlap: True
apply_rope_fusion: True
deteministic_mode: False
Expand All @@ -161,7 +160,6 @@ model:
fp8_amax_history_len: 1024 # Number of steps for which amax history is recorded per tensor
fp8_amax_compute_algo: max # 'most_recent' or 'max'. Algorithm for computing amax from history
fp8_wgrad: True
ub_tp_comm_overlap: False

optim:
name: mcore_distributed_optim
Expand Down

0 comments on commit 10120db

Please sign in to comment.