Skip to content

Push details to hub does not work #99

@NathanHB

Description

@NathanHB

Removing the all section of the result file has broken the create metadata card mechanism.

To reproduce:

 python  run_evals_accelerate.py --model_args "pretrained=HuggingFaceH4/zephyr-7b-beta" --tasks "lighteval|triviaqa|0|0"  --override_batch_size 1  --output_dir="tmp/"  --max_samples 1 --save_details --push_details_to_hub --results_org open-llm-leaderboard --push_results_to_hub

Also, having datasets version below 2.18.0 yields an error when fetching some files for the details dataset.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions