You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'd like to use vicuna_bench. After python gen_api_answer.py --bench-name vicuna_bench --model gpt-3.5-turbo succeeds, python gen_judgment.py --bench-name vicuna_bench --model-list gpt-3.5-turbo failed with
88%|█████████████████████████████████████████████████████████████████████████████████████████▎ | 70/80 [17:27<02:29, 14.97s/it]
Traceback (most recent call last):
File ".../FastChat/fastchat/llm_judge/gen_judgment.py", line 309, in <module>
play_a_match_func(match, output_file=output_file)
File ".../FastChat/fastchat/llm_judge/common.py", line 203, in play_a_match_single
score, user_prompt, judgment = run_judge_single(
File ".../FastChat/fastchat/llm_judge/common.py", line 141, in run_judge_single
kwargs["ref_answer_2"] = ref_answer["choices"][0]["turns"][1]
IndexError: list index out of range
The text was updated successfully, but these errors were encountered:
I'd like to use
vicuna_bench
. Afterpython gen_api_answer.py --bench-name vicuna_bench --model gpt-3.5-turbo
succeeds,python gen_judgment.py --bench-name vicuna_bench --model-list gpt-3.5-turbo
failed withThe text was updated successfully, but these errors were encountered: