deep-learning
Self-attention
taught
Taught in
-
Multi-head attention
Deep Learning / Neural Networks
-
Self-attention versus CNNs and RNNs
Deep Learning / Neural Networks
-
The Transformer block and decoder-only models
Wiring Language Models
Requires
- multi-head-attention required
Required by
- Contrastive image-text embedding (the CLIP-style mechanism) recommended
- Positional encoding required
- The Transformer architecture required
Local knowledge map
Local knowledge map
Click to expand
Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.