Skip to content

Commit f4b386e

Browse files
2015aroras2015aroras
authored
Merge pull request #704 from allenai/shanea/fix-olmo-1.7-batch-size
Change OLMo April/July global batch size to 1024
2 parents 1e71ce3 + 809d2f1 commit f4b386e

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

‎configs/official/OLMo-7B-0424.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ save_num_unsharded_checkpoints_to_keep: -1
7676
load_path: null
7777

7878
max_duration: 2ep
79-
global_train_batch_size: 512
79+
global_train_batch_size: 1024
8080
device_train_microbatch_size: 2
8181

8282
precision: amp_bf16

‎configs/official/OLMo-7B-0724.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ save_num_unsharded_checkpoints_to_keep: -1
7676
load_path: null
7777

7878
max_duration: 2ep
79-
global_train_batch_size: 512
79+
global_train_batch_size: 1024
8080
device_train_microbatch_size: 2
8181

8282
precision: amp_bf16

0 commit comments

Comments
 (0)