Concept clusters
Related ideas that live chapters apart, pulled onto one page — every factoring technique, every way to solve a quadratic, and more.
Every activation function, compared
Sigmoid, tanh, ReLU, leaky ReLU, ELU, GELU, and SiLU -- one family, one decision rule for which to reach for.
9 concepts →
Every normalization scheme, and when to use it
Batch, layer, and RMSNorm -- three ways to keep a deep network's activations in a stable range, sorted by what each one actually normalizes over.
11 concepts →
The optimizer family, one decision chain
Plain gradient descent, SGD, momentum, and Adam -- the same update rule, extended one idea at a time.
6 concepts →
Attention mechanisms, one story
Queries, keys, and values, scoring functions, the scaling factor, and multi-head attention -- one mechanism, built up section by section.
11 concepts →
The architecture-family decision chain
CNNs, RNNs, and Transformers -- three ways to build in an assumption about structure, and which assumption fits which data.
10 concepts →
Diagnosing a network that won't train
A loss that won't move, a loss that explodes, and an accuracy that's stuck -- the fixed, ordered checklist for telling a real bug from an unlucky hyperparameter.
9 concepts →