diff --git a/examples/language-modeling/README.md b/examples/language-modeling/README.md index ca947f30ef..909593427d 100644 --- a/examples/language-modeling/README.md +++ b/examples/language-modeling/README.md @@ -549,7 +549,8 @@ python3 ../gaudi_spawn.py --use_deepspeed --world_size 8 run_lora_clm.py \ --throughput_warmup_steps 3 \ --lora_rank 4 \ --lora_target_modules "q_proj" "v_proj" "k_proj" "o_proj" \ - --validation_split_percentage 4 + --validation_split_percentage 4 \ + --use_flash_attention True ``` - Multi-card finetuning of Falcon-180B: