-
Notifications
You must be signed in to change notification settings - Fork 362
Closed
Labels
Description
Removing the all
section of the result file has broken the create metadata card mechanism.
To reproduce:
python run_evals_accelerate.py --model_args "pretrained=HuggingFaceH4/zephyr-7b-beta" --tasks "lighteval|triviaqa|0|0" --override_batch_size 1 --output_dir="tmp/" --max_samples 1 --save_details --push_details_to_hub --results_org open-llm-leaderboard --push_results_to_hub
Also, having datasets
version below 2.18.0 yields an error when fetching some files for the details dataset.