Glossary

The share of predictions a classifier got exactly right.

The simplest model you compare every fancier model against.

A plot of observed frequency against predicted probability, checking whether a model's confidence is honest.

Predicting a category, not a number -- spam or not-spam, not a price.

The number a model multiplies a feature by -- how much that feature moves the prediction.

A number measuring whether two quantities tend to rise and fall together.

The full collection of examples a model learns from or is tested on.

The line (or surface) separating one predicted class from another.

A tree diagram recording every merge in hierarchical clustering, in order.

The whole dataset as one matrix -- one row per sample, one column per feature.

The full pattern of which values are common and which are rare for some quantity.

A dense vector representation of text, placing similar meanings close together.

Uncertainty from limited knowledge (reducible) vs. genuine randomness (irreducible).

A number computed from data that stands in for a true, usually unknown, value.

The proportion of a dataset's total variance one principal component captures.

One data sample's measurements, written as a single vector.

A model's ability to perform well on data it never saw during training.

A classifier that models each class's own feature distribution, then applies Bayes' theorem.

A measure of how mixed a tree leaf's classes are, used to choose splits.

The direction and steepness of a function's fastest increase -- used to know which way to step to reduce error.

A setting chosen before training, not learned from the data.

Replacing a missing value with a principled substitute.

A similarity measure between two points, substituted for the inner product to fit a nonlinear boundary.

A past value used as a predictor for the current time step.

A final, unsplit region of a decision tree, holding its own prediction.

A plot of training and validation error against training set size.

Squashes any number into a value between 0 and 1, so it can be read as a probability.

High correlation between two or more features themselves, not between a feature and the target.

Replacing a categorical feature with one binary column per category.

A free validation estimate from bagging, using each tree's own left-out bootstrap samples.

The internal numbers a model adjusts during training.

A fixed sequence of data-processing and modeling steps, chained so they run the same way every time.

Of everything predicted positive, how much actually was.

The model's guess at the answer for one input.

Cleaning and reshaping raw data before a model ever sees it.

A number between 0 and 1 measuring how likely something is.

Of everything actually positive, how much the model caught.

Predicting a number, not a category -- a house price or a temperature, not a yes/no label.

Adding a penalty term to a model's loss to discourage overly large or complex fits.

How far a prediction was from the true value: y minus y-hat.

The posterior probability, under the current model, that a point belongs to a given mixture component.

Synthetic Minority Oversampling Technique -- generates new minority-class points by interpolation.

Rescaling a feature to have mean 0 and standard deviation 1.

A training point sitting on, or violating, the margin -- the only points that determine the fitted boundary.

A feature that is a consequence of the target, not a genuine predictor available beforehand.

The known correct outputs, one per data sample.

Data touched exactly once, at the end, to report a model's final performance.

A text feature weight combining how often a word appears in one document with how rare it is overall.

The subset of samples a model actually learns from.

Held-back data used repeatedly during development to tune hyperparameters and compare models.