PyTorch implementaion of Data-eifficient Image Transformers. I have written code for student network and teacher network sperately and then combined them together. Like to the paper: https://arxiv.org/pdf/2012.12877.pdf The student network is a vision transformer which learns itself as well as takes valueable lessons from the teacher network. Teacher network is a convolutional network which is pre-trained. For time being I have used Teacher network which is custom built. One can use different variants of ResNet, VGG, etc. according to the use case. Following is the overall block diagram shown in the paper.
