logit

A model's raw, unnormalized output score for one vocabulary entry, before softmax turns the whole set into probabilities.

A logit is one entry of a model's raw output vector, one per vocabulary token, before softmax normalization. Logits can be any real number and are not themselves probabilities; softmax exponentiates and normalizes them into a valid probability distribution over the vocabulary.

Examples

Before softmax, a model might output a logit of 8.2 for "cat" and 1.1 for "dog" at some position -- neither number is a probability yet.