language model

A system that assigns a probability to sequences of text, most often used by predicting the next token given the ones before it.

A language model is a probability distribution over sequences of tokens, almost always factored by the chain rule into a product of next-token conditionals: $P(w_1, \ldots, w_n) = \prod_{i=1}^n P(w_i \mid w_1, \ldots, w_{i-1})$ . Training a language model means fitting this conditional distribution to a large corpus; using one means sampling from it, token by token.

Examples

Autocomplete on a phone keyboard is a small language model predicting your next word.