The architecture-family decision chain

Each architecture family bakes in a different assumption about the input, and picking one is really picking an assumption. A CNN assumes locality and translation invariance -- a pattern that matters in one corner of an image matters the same way elsewhere -- and pays for that assumption with far fewer parameters than a fully connected layer would need. An RNN assumes order matters and processes a sequence one step at a time, carrying a hidden state forward, which suits a stream that arrives incrementally but struggles with very long-range dependencies and can't be parallelized across time steps. A Transformer assumes neither locality nor strict sequential order -- self-attention lets every position look at every other position directly, in parallel, at the cost of quadratic compute in sequence length. In practice: reach for a CNN on grid-structured data (images) where locality is a genuine prior, reach for an RNN when a sequence must be processed causally and one step at a time with bounded memory, and reach for a Transformer when long-range dependencies matter more than compute cost -- which is why it has displaced RNNs for most large-scale sequence modeling today.