As part of our first community hangout, we’re excited to be hosting a few sprints. This is one of the same:
The plan with ML Sprints is to run week-long activities where our community will contribute to projects.
This is one of three wikis that we’re inviting you to contribute to! This Wiki is meant to serve as a collection of best resources to learn about Transformer models and their applications.
This is a wiki! This means all of you can edit it, please do so!
Attention Is All You Need (2017)
End-to-End Object Detection with Transformers (2020)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2021)
The Annotated Transformer
Transformer Deep Dive
The illustrated Transformer
Attention is All you need by Yannic Kilcher
GPT-2 by Yannic Kilcher
BERT by Yannic Kilcher
RoBERTa by Yannic Kilcher
Utilizing Transformer Representations Efficiently
On Stability of Few-Sample Transformer Fine-Tuning
Speeding up Transformer w/ Optimization Strategies