diff --git a/docs/source/adding-a-custom-task.mdx b/docs/source/adding-a-custom-task.mdx index e5160024d..3c34c5bfa 100644 --- a/docs/source/adding-a-custom-task.mdx +++ b/docs/source/adding-a-custom-task.mdx @@ -171,7 +171,7 @@ Once your file is created you can then run the evaluation with the following com ```bash lighteval accelerate \ - "pretrained=HuggingFaceH4/zephyr-7b-beta" \ + "model_name=HuggingFaceH4/zephyr-7b-beta" \ "community|{custom_task}|{fewshots}|{truncate_few_shot}" \ --custom-tasks {path_to_your_custom_task_file} ``` diff --git a/docs/source/quicktour.mdx b/docs/source/quicktour.mdx index 798883f12..de5791b46 100644 --- a/docs/source/quicktour.mdx +++ b/docs/source/quicktour.mdx @@ -27,7 +27,7 @@ To evaluate `GPT-2` on the Truthful QA benchmark with [🤗 ```bash lighteval accelerate \ - "pretrained=gpt2" \ + "model_name=openai-community/gpt2" \ "leaderboard|truthfulqa:mc|0|0" ``` @@ -59,7 +59,7 @@ When specifying a path to file, it should start with `./`. ```bash lighteval accelerate \ - "pretrained=gpt2" \ + "model_name=openai-community/gpt2" \ ./path/to/lighteval/examples/tasks/recommended_set.txt # or, e.g., "leaderboard|truthfulqa:mc|0|0|,leaderboard|gsm8k|3|1" ``` @@ -79,7 +79,7 @@ You can then evaluate a model using data parallelism on 8 GPUs like follows: ```bash accelerate launch --multi_gpu --num_processes=8 -m \ lighteval accelerate \ - "pretrained=gpt2" \ + "model_name=openai-community/gpt2" \ "leaderboard|truthfulqa:mc|0|0" ``` @@ -92,7 +92,7 @@ To evaluate a model using pipeline parallelism on 2 or more GPUs, run: ```bash lighteval accelerate \ - "pretrained=gpt2,model_parallel=True" \ + "model_name=openai-community/gpt2,model_parallel=True" \ "leaderboard|truthfulqa:mc|0|0" ```