language-models
The decoder-only, causally-masked shape of an LLM
taught
Why a modern LLM is a stack of Transformer blocks with causal (left-to-right) masking and no separate encoder -- reading a token sequence in and a next-token distribution out, one position at a time.
Taught in
-
The Transformer block and decoder-only models
Wiring Language Models
Requires
-
transformer-architecture
recommended
deep-learning ch.21 builds the full encoder/decoder Transformer; this course only needs the decoder-only causal-LM specialization that every model in this course actually uses.
Required by
- Vision-language models recommended
Local knowledge map
Local knowledge map
Click to expand
Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.