Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Higgs Llama3-70B V2 Results #367

Merged
merged 2 commits into from
Jul 17, 2024
+69,389 −0
Merged

Add Higgs Llama3-70B V2 Results #367

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c906a24
Add higgs-llama-3-70b-v2 to AlpacaEval
sxjscience Jul 16, 2024
a9b00da
Add higgs-llama-3-70b-v2 to AlpacaEval
sxjscience Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Loading
Loading status checks…
Add higgs-llama-3-70b-v2 to AlpacaEval
@sxjscience
sxjscience committed Jul 16, 2024
commit a9b00da2bd3fdcf046a72364f8f4c8157119e69a
1 change: 1 addition & 0 deletions ...lpaca_eval/leaderboards/data_AlpacaEval_2/weighted_alpaca_eval_gpt4_turbo_leaderboard.csv
View file
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@ Storm-7B-best-of-64,63.04099075186919,1.4253258915161846,519,286,0,805,64.472049
Together-MoA-Lite,56.593045622273294,1.4464848562244548,456,347,2,805,56.77018633540373,community,1968,59.1415240989275,0.7580510219326322
aligner-2b_gpt-4-turbo-2024-04-09,46.77089325668323,1.3378060774476594,371,417,17,805,40.18633540372671,community,1370,58.33130206276722,
gpt-4o-2024-05-13,51.32757578249279,1.4700094589795554,429,369,7,805,53.72670807453416,minimal,1873,57.45682883335095,
higgs-llama-3-70b-v2,68.63519246435168,1.3151765652301792,563,240,2,805,70.06211180124224,community,2657,56.76317433000503,0.8571649314205525
gpt-4-turbo-2024-04-09,46.11526538763708,1.474073957743638,370,426,9,805,46.52173913043478,minimal,1802,55.01530093647852,
SPPO-Gemma-2-9B-It-PairRM,48.23404468746583,1.4568887170812033,386,418,1,805,48.01242236024844,community,1803,53.96983730150777,0.6516507968188552
claude-3-5-sonnet-20240620,40.56021409682828,1.4679655403720542,312,493,0,805,38.75776397515528,community,1488,52.36675427146999,
1 change: 1 addition & 0 deletions ...ights/weighted_alpaca_eval_gpt4_turbo/length_controlled_v1/baseline_gpt4_1106_preview.csv
View file
Original file line number Diff line number Diff line change
@@ -170,3 +170,4 @@ Nanbeige2-16B-Chat,-1.4383673979411902,0.8415127360873783,-0.3850159994606512
openpipe-moa-gpt-4-turbo-v1,-1.0482540803063984,0.8922946327161730,1.1183646496339554
SPPO-Llama-3-Instruct-8B-PairRM,-1.0191251760902622,0.8783306469909790,-0.4989987701412274
SPPO-Gemma-2-9B-It-PairRM,-1.1139152907711427,0.5972758612054220,0.2481716650556719
higgs-llama-3-70b-v2,-1.3408055191105048,0.9224458425462844,0.4939211483441316
12 changes: 12 additions & 0 deletions src/alpaca_eval/models_configs/higgs-llama-3-70b-v2/configs.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
higgs-llama-3-70b-v2:
prompt_template: "Mixtral-8x7B-Instruct-v0.1/togetherai_prompt.txt"
fn_completions: "openai_completions"
completions_kwargs:
model_name: "higgs-llama-3-70b-240629-v2"
max_tokens: 4096
num_procs: 8
requires_chatml: True
client_kwargs:
base_url: 'https://api.boson.ai/v1'
pretty_name: "Higgs-Llama-3-70B V2"
link: "https://boson.ai/"