TensorFlow Implementation of Vision Transformer

My Code: https://github.com/abrahammathews2000/ViT-TensorFlow-Implementation/blob/main/ViT_TensorFlow_Implementation_v1.ipynb

Link to original paper

https://iclr.cc/virtual/2021/poster/3013

Original Paper Title: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Authors: Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

Other online resources I used to learn to implement ViT

Codes:

1. Keras Implementation: https://keras.io/examples/vision/image_classification_with_vision_transformer/
2. PyTorch Implementation: https://www.learnpytorch.io/08_pytorch_paper_replicating/
3. TensorFlow Implementation: https://github.com/taki0112/vit-tensorflow/blob/main/vit_tensorflow/vit.py

Blogs:

1. https://khvmaths.medium.com/vision-transformer-understanding-the-underlying-concept-83d699d71180
2. https://medium.com/analytics-vidhya/vision-transformers-bye-bye-convolutions-e929d022e4ab
3. https://deepganteam.medium.com/vision-transformers-for-computer-vision-9f70418fe41a
4. https://jalammar.github.io/illustrated-transformer/
5. https://machinelearningmastery.com/the-attention-mechanism-from-scratch/