Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Recipe

Run PCA in practice

Use when: you have a design matrix with many features and want to reduce it to a smaller number of useful components

Goal: a reduced design matrix, with a clear reason for how many components you kept

Taught in Running PCA in practice

Quick recipe

  1. Center the data
  2. Scale the features
  3. Compute the covariance matrix (or use SVD directly)
  4. Pick the number of components
  5. Project the data onto the components you picked
  1. 1

    Subtract each feature's mean, so every feature has mean zero.

    Why: PCA's derivation from Section 5 assumes the data is already centered at the origin.

  2. 2

    Divide each feature by its standard deviation, using Chapter 4's scaling.

    Why: without this step, a feature with larger raw units would swamp the variance calculation, regardless of how useful it really is.

  3. 3

    Compute the covariance matrix of the scaled, centered data, or apply SVD directly to that same design matrix.

    Why: the principal components are exactly this matrix's eigenvectors.

  4. 4

    Use a scree plot, or a running total of explained variance, often set at 90-95 percent. If the scree plot shows no clear elbow, fall back to the explained-variance total instead.

    Why: too few components throws away real information. Too many keeps noise, which defeats the whole point of shrinking the data.

  5. 5

    Transform the original scaled, centered data into the smaller space spanned by the components you picked.

    Why: this is the actual, reduced design matrix. Use it for training a model or for plotting, from here on.

Common mistakes

  • Running PCA before scaling and centering the features, letting features with larger raw units swamp the rest.
  • Keeping components based on training-set variance alone, without checking whether they hold up on new data.
  • Fitting PCA on the full dataset before splitting into train and test, which leaks test-set information into the components.