Skip to content

readme: replace tabs with spaces #1485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
regisss merged 1 commit into from
Nov 15, 2024
Merged

readme: replace tabs with spaces #1485

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 32 additions & 32 deletions examples/language-modeling/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,38 +475,38 @@ python ../gaudi_spawn.py \
- Multi-card finetuning of Llama2-7B with FP8:
```bash
PT_HPU_AUTOCAST_LOWER_PRECISION_OPS_LIST=ops_bf16.txt python ../gaudi_spawn.py \
--world_size 8 --use_mpi run_lora_clm.py \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset_name tatsu-lab/alpaca \
--bf16 True \
--output_dir ./model_lora_llama \
--num_train_epochs 3 \
--per_device_train_batch_size 16 \
--gradient_accumulation_steps 1 \
--eval_strategy "no" \
--save_strategy "no" \
--learning_rate 3e-4 \
--warmup_ratio 0.03 \
--lr_scheduler_type "constant" \
--max_grad_norm 0.3 \
--logging_steps 20 \
--do_train \
--do_eval \
--use_habana \
--use_lazy_mode \
--throughput_warmup_steps 18 \
--lora_rank=8 \
--lora_alpha=16 \
--lora_dropout=0.05 \
--lora_target_modules "q_proj" "v_proj" \
--dataset_concatenation \
--max_seq_length 512 \
--ddp_bucket_cap_mb 50 \
--adam_epsilon 1e-08 \
--validation_split_percentage 10 \
--low_cpu_mem_usage True \
--pipelining_fwd_bwd \
--fp8 True
--world_size 8 --use_mpi run_lora_clm.py \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset_name tatsu-lab/alpaca \
--bf16 True \
--output_dir ./model_lora_llama \
--num_train_epochs 3 \
--per_device_train_batch_size 16 \
--gradient_accumulation_steps 1 \
--eval_strategy "no" \
--save_strategy "no" \
--learning_rate 3e-4 \
--warmup_ratio 0.03 \
--lr_scheduler_type "constant" \
--max_grad_norm 0.3 \
--logging_steps 20 \
--do_train \
--do_eval \
--use_habana \
--use_lazy_mode \
--throughput_warmup_steps 18 \
--lora_rank=8 \
--lora_alpha=16 \
--lora_dropout=0.05 \
--lora_target_modules "q_proj" "v_proj" \
--dataset_concatenation \
--max_seq_length 512 \
--ddp_bucket_cap_mb 50 \
--adam_epsilon 1e-08 \
--validation_split_percentage 10 \
--low_cpu_mem_usage True \
--pipelining_fwd_bwd \
--fp8 True
```

- Multi-card finetuning of codegen-16B-mono:
Expand Down