Prompt formatter API and canary transcribe tensor input support#9206

Merged

titu1994 merged 22 commits intomainfrom

prompt-formatter-and-canary-tensor-dataset

Jun 1, 2024

Commits on May 21, 2024

Commits on May 22, 2024

Improved test coverage + proper preamble support
pzelasko
committed
Fix usage of PromptFormatter for MT-AED class + fix tokenization/formatting issues
pzelasko
committed
Move some canary hacks to canary prompt formatter, improve validation, add tests for aggtok
pzelasko
committed

Commits on May 24, 2024

code review: partial
pzelasko
committed

Commits on May 29, 2024

Accept multi-turn, single-turn, and legacy prompt format in transcribe() and transcribe_speech.py
pzelasko
committed