Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Transformer

An architecture built entirely from attention and feed-forward layers, with no recurrence or convolution.

Examples

  • Stacking six encoder blocks, each with multi-head attention and a feed-forward sublayer, builds one Transformer encoder.

See also

self-attention multi-head attention positional encoding encoder-decoder architecture
Read the full section →
Graph settings

Calculator