🚨 🚨 🚨 Fix Issue 15003: SentencePiece Tokenizers Not Adding Special Tokens in convert_tokens_to_string#15775
Merged
sgugger merged 29 commits intohuggingface:mainfrom Nov 2, 2022
beneyal:15003-albert-tokenizer-special-tokens
Merged
🚨 🚨 🚨 Fix Issue 15003: SentencePiece Tokenizers Not Adding Special Tokens in convert_tokens_to_string#15775sgugger merged 29 commits intohuggingface:mainfrom beneyal:15003-albert-tokenizer-special-tokens
convert_tokens_to_string#15775sgugger merged 29 commits intohuggingface:mainfrom
beneyal:15003-albert-tokenizer-special-tokens
Commits
Commits on Feb 22, 2022
- committed
- committed
- committed
- committed
- committed
- committed
- committed
Commits on Feb 24, 2022
Commits on Feb 25, 2022
- committed
- committed
Ben Eyal - committedBen Eyal
- committedBen Eyal
Commits on Mar 4, 2022
- committed
- committed
- committed
- committed
- committed
- committed
Commits on Apr 28, 2022
Commits on Apr 29, 2022
Commits on Nov 2, 2022
- committed
- committed
- committed
- committed