-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
P1Priority of the issue for triage purpose: Needs to be fixed soon.Priority of the issue for triage purpose: Needs to be fixed soon.enhancementNew feature or requestNew feature or request
Milestone
Description
Porting BERTTokenizers enables several text embedding generation models. Requires #6988.
https://github.com/huggingface/text-embeddings-inference?tab=readme-ov-file#text-embeddings.
https://github.com/huggingface/transformers/blob/v4.37.0/src/transformers/models/bert/tokenization_bert.py#L137
We already have some BERT implementation which may be sufficient.
luisquintanilla and abbottdev
Metadata
Metadata
Assignees
Labels
P1Priority of the issue for triage purpose: Needs to be fixed soon.Priority of the issue for triage purpose: Needs to be fixed soon.enhancementNew feature or requestNew feature or request