deep-learning
The Transformer architecture
taught
Taught in
-
The position-wise feed-forward network
Deep Learning / Neural Networks
-
Residual connections and layer normalization
Deep Learning / Neural Networks
-
The Transformer encoder
Deep Learning / Neural Networks
-
The Transformer decoder
Deep Learning / Neural Networks
-
Training a small Transformer
Deep Learning / Neural Networks
-
Efficient attention and a look at state-space models
Deep Learning / Neural Networks
-
The Transformer block and decoder-only models
Wiring Language Models
Requires
- self-attention required
- positional-encoding required
- residual-connections required
Required by
Local knowledge map
Local knowledge map
Click to expand
Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.