Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

vocabulary

The fixed set of tokens a tokenizer can produce and a model can read or predict.

A tokenizer's vocabulary is the fixed, finite set of tokens it can emit, each with a unique integer ID. A model's output layer has exactly one logit per vocabulary entry, so vocabulary size directly sets the size of that final layer.

Examples

  • GPT-2's vocabulary has 50,257 entries, so its final output layer produces exactly 50,257 logits at every generation step.

See also

token tokenization
Read the full section →
Graph settings

Calculator