mechanism

A specific piece of machinery inside a model that does one job, like attention or a feed-forward layer.

A mechanism is a distinct computational component inside a model's architecture, responsible for one specific operation -- attention (mixing information across positions) and the feed-forward layer (processing each position independently) are the two mechanisms inside every Transformer block.

Examples

Attention is the mechanism that lets a model weigh earlier words when predicting the next one.