Skip to content

Add pre-tokenizer regexes for BLOOM and gpt3-finnish#8850

Merged
ggerganov merged 1 commit intoggml-org:masterfrom
Exploder98:bloom-pre-tokenizer
Aug 15, 2024
Merged

Add pre-tokenizer regexes for BLOOM and gpt3-finnish#8850
ggerganov merged 1 commit intoggml-org:masterfrom
Exploder98:bloom-pre-tokenizer

Commits

Commits on Aug 4, 2024