Skip to content

Commit 83f0467

Browse files
authored
Add Nanbeige2-16B-Chat to AlpacaEval (#345)
1 parent 18a4e96 commit 83f0467

File tree

6 files changed

+69316
-0
lines changed

6 files changed

+69316
-0
lines changed

Diff for: results/Nanbeige2-16B-Chat/model_outputs.json

+4,832
Large diffs are not rendered by default.

Diff for: results/Nanbeige2-16B-Chat/weighted_alpaca_eval_gpt4_turbo/annotations.json

+64,464
Large diffs are not rendered by default.

Diff for: src/alpaca_eval/leaderboards/data_AlpacaEval_2/weighted_alpaca_eval_gpt4_turbo_leaderboard.csv

+1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Nanbeige-Plus-Chat-v0.1,56.70300973017392,1.482841874951873,456,347,2,805,56.770
1515
Qwen1.5-110B-Chat,33.77709527565118,1.3776163153661627,255,545,5,805,31.98757763975156,community,1631,43.90555221078692,
1616
gpt4_1106_preview_concise,22.92019444047205,1.232517714329424,172,622,11,805,22.049689440993788,dev,1136,41.896601591245386,
1717
aligner-2b_claude-3-opus-20240229,34.46337362321739,1.314666526302454,225,475,105,805,34.47204968944099,community,1669,41.823071715247664,
18+
Nanbeige2-16B-Chat,37.03608605005168,1.4340261272580377,288,514,3,805,35.962732919254655,community,1867,40.591286349562864,0.8504106275373426
1819
claude-3-opus-20240229,29.10526953334248,1.3941539442369442,223,579,3,805,27.888198757763977,minimal,1388,40.5095080124761,
1920
gpt4,23.576789314782605,1.275704201206918,179,618,8,805,22.732919254658384,verified,1365,38.12808974440021,
2021
aligner-2b_qwen1.5-72b-chat,31.773037737123104,1.2392772646245978,180,473,152,805,31.801242236024844,community,1812,36.725868878524274,

Diff for: src/alpaca_eval/metrics/weights/weighted_alpaca_eval_gpt4_turbo/length_controlled_v1/baseline_gpt4_1106_preview.csv

+1
Original file line numberDiff line numberDiff line change
@@ -165,3 +165,4 @@ Llama-3-Instruct-8B-SimPO-ExPO,-1.1153280231371028,0.8741611299275304,-0.1029222
165165
merlinite-7B-AOT,-0.9472382718509442,0.8407838130728476,-0.8954727783980261
166166
Together-MoA,-1.0555583531357304,0.8453234405641900,0.9217351025640278
167167
Together-MoA-Lite,-1.0572386816426196,0.7849833974539681,0.5628671529713698
168+
Nanbeige2-16B-Chat,-1.4383673979411902,0.8415127360873783,-0.3850159994606512
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
Nanbeige2-16B-Chat:
2+
prompt_template: "Nanbeige2-16B-Chat/prompt.txt"
3+
fn_completions: "openai_completions"
4+
completions_kwargs:
5+
model_name: "Nanbeige2-16B-Chat"
6+
max_tokens: 4096
7+
temperature: 0.3
8+
top_p: 0.9
9+
requires_chatml: true
10+
is_chat: true
11+
client_kwargs:
12+
base_url: "https://nanbeige.zhipin.com/api/gpt/open/chat/openai/v1"
13+
pretty_name: "Nanbeige2 16B Chat"
14+
link: "https://huggingface.co/Nanbeige/Nanbeige2-16B-Chat"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
<|im_start|>system
2+
你是一个名为\"南北阁\"的人工智能助手,正在与人类用户进行交谈。你的目标是以最有帮助和最逻辑的方式回答问题,同时确保内容的安全性。你的回答中不应包含任何有害、政治化、宗教化、不道德、种族主义、非法的内容。请确保你的回答不带有社会偏见,符合社会主义价值观。如果遇到的问题无意义或事实上不连贯,请不要回答错误的内容,而是解释问题为何无效或不连贯。如果你不知道问题的答案,也请勿提供错误的信息。<|im_end|>
3+
<|im_start|>user
4+
{instruction}<|im_end|>

0 commit comments

Comments
 (0)