[Question]: tarsclassifier prediction is very slow for more then 100 classes

[sainiudit]

Question

I am using tarsclassifier and fine tuned it for 100+ classes, prediction is taking forever to run any work around for this

[alanakbik]

Hello @sainiudit, unfortunately TARS is slow in scenarios with many classes. The reason is that TARS makes an independent forward pass through the model for each class. So if you have 100 classes, each sentence requires 100 forward passes through the model in order to predict.

We worked together with a team at Bayer to scale TARS up to 80,000 classes. This is not integrated into Flair, but there is code and a paper available:

• Paper: https://arxiv.org/abs/2206.02662
• Code: https://github.com/Bayer-Group/xtars-naacl2022