Update README for FSDP#832
Conversation
This corrects the command for FSDP
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
regisss
left a comment
regisss
left a comment
There was a problem hiding this comment.
LGTM!
Is this compatible with Synapse 1.14 (so merge directly into main) or from 1.15 (so merge into synapse_1.15)?
libinta
left a comment
libinta
left a comment
There was a problem hiding this comment.
***** train metrics *****
epoch = 2.0
max_memory_allocated (GB) = 82.02
memory_allocated (GB) = 25.98
total_memory_available (GB) = 94.62
train_loss = 0.8538
train_runtime = 1:21:26.40
train_samples_per_second = 1.273
train_steps_per_second = 0.016
03/25/2024 19:01:48 - INFO - main - *** Evaluate ***
[INFO|trainer.py:1688] 2024-03-25 19:01:48,298 >> ***** Running Evaluation *****
[INFO|trainer.py:1690] 2024-03-25 19:01:48,298 >> Num examples = 125
[INFO|trainer.py:1693] 2024-03-25 19:01:48,298 >> Batch size = 1
100%|██████████| 16/16 [00:35<00:00, 2.37s/it]***** eval metrics *****
100%|██████████| 16/16 [00:37<00:00, 2.35s/it]
epoch = 2.0
eval_accuracy = 0.7927
eval_loss = 0.7583
eval_runtime = 0:00:39.94
eval_samples = 125
eval_samples_per_second = 3.129
eval_steps_per_second = 0.401
max_memory_allocated (GB) = 82.02
memory_allocated (GB) = 25.98
perplexity = 2.1347
total_memory_available (GB) = 94.62
4 participants
This corrects the command for FSDP