[1/9] padding-free

[garrett361]

This PR adds the ability to train with padding-free collation, thereby removing all padding from per_device_train_batch_size>1 training examples. The HF model must have proper support for padding-free training, otherwise the model outputs will be silently wrong. LLama and bamba (as of huggingface/transformers#35861) are examples of models which have padding-free training support.

To use padding-free, just specify --padding-free True

	PR	Title
1	>15	padding-free
2	#16	clean_checkpoints_at_end
3	#17	final_lr_ratio
4	#18	add_seed_and_date_to_run_name
5	#19	additional_model_arguments
6	#20	sync_each_batch=True grad acc
7	#21	no grad acc averaging for sum losses
8	#22	extra reporting
9	#23	local_main_process_first when building dataset

[fabianlim]

should be open_instruct.padding_free

[garrett361]

that makes it more robust w/r/t the cwd the script is launched from, I guess?

[garrett361]

done

[dangxuanhong]

Thanks and LGTM