feat: frequency, min_new_tokens, presence, and repetition penalties - #973

Merged

Ying1123 merged 7 commits into

sgl-project:mainfrom

tmfi-us:ryan/srt/schedule_batch/implement-penalties

Aug 8, 2024

Commits on Aug 7, 2024

feat: frequency, min_new_tokens, presence, and repetition penalties
vhain
committed
edaab22

Commits on Aug 8, 2024

Merge branch 'main' into ryan/srt/schedule_batch/implement-penalties
vhain
authored
6dc819c
prefix request-body with extra- on bench_serving
vhain
committed
8e2ef00
avoid using relative imports
vhain
committed
8fc6364
avoid using localhost in test_srt_endpoint_with_penalizers
vhain
committed
7a01b34
mention value constraints for penalty sampling params
vhain
committed
560176b
write a comment about penalizer init
vhain
committed
d95dd54