feat: add additional goosebench evals#1571

Merged

ahau-square merged 35 commits intomainfrom

Mar 10, 2025

Commits on Mar 10, 2025

ollama model + structured output shim

alicehau
authored and
Alice Hau
committed
tool shim generic

alicehau
authored and
Alice Hau
committed
refactoring toolshim

alicehau
authored and
Alice Hau
committed
refactor

alicehau
authored and
Alice Hau
committed
ollama toolshim ready

alicehau
authored and
Alice Hau
committed
clean up

alicehau
authored and
Alice Hau
committed
lint

alicehau
authored and
Alice Hau
committed
format

alicehau
authored and
Alice Hau
committed
add toolshim to openai provider

alicehau
authored and
Alice Hau
committed
update toolshim prompt:
Alice Hau
committed
prompt update

alicehau
authored and
Alice Hau
committed
refactor
Alice Hau
committed
undo change to reference agent
Alice Hau
committed
undo ref agent changes
Alice Hau
committed
refactor config
Alice Hau
committed
refactor
Alice Hau
committed
check env vars
Alice Hau
committed
fix tests
Alice Hau
committed
clean up
Alice Hau
committed
add small model evals
Alice Hau
committed
add initial evals
Alice Hau
committed
updated suite assignments
Alice Hau
committed
rebase
Alice Hau
committed
updatae external extensions str
Alice Hau
committed
add time and tools metrics and write to file utils

alicehau
authored and
Alice Hau
committed
add copying of session file into workdir

alicehau
authored and
Alice Hau
committed
add metric to check tool use

alicehau
authored and
Alice Hau
committed
rename suite

alicehau
authored and
Alice Hau
committed
update name, account for non bool metrics

alicehau
authored and
Alice Hau
committed
format

alicehau
authored and
Alice Hau
committed
update benchmark script to allow toolshim

alicehau
authored and
Alice Hau
committed
remove changes from toolshim branch
Alice Hau
committed
add back token usage mods
Alice Hau
committed
instruct flappy bird to use pyenv
Alice Hau
committed
remove toolshiim from this PR
Alice Hau
committed