Recipe
Run PCA via the SVD directly
Use when: you need principal components or explained variance for a centered data matrix X, without ever forming the covariance matrix
Goal: the principal directions and the variance each one explains, computed entirely from X's own singular values
Taught in PCA is an SVD
Quick recipe
-
1
Subtract each variable's mean so X's columns average to zero, then compute X=U\Sigma V^{\mathsf T} directly; never form X^{\mathsf T}X.
Example
A centered data matrix X with n=5 observations has singular values \sigma_1=8,\sigma_2=4.
Why: the covariance-matrix route and the direct SVD route only agree once X is genuinely centered, and skipping straight to the SVD avoids ever squaring X's own condition number.
-
2
The columns of V, the right singular vectors, are the principal directions, in the same decreasing order as the singular values, with no separate eigendecomposition to run.
Example
In the point cloud behind this section's covariance-lab widget, V's first column points along the direction of greatest spread, exactly the widget's own first principal axis.
Why: C=\frac1{n-1}V\Sigma^2V^{\mathsf T} is already C's own eigendecomposition, and V is already available from Step 1.
-
3
Divide the squared singular value by n-1: \lambda_i=\sigma_i^2/(n-1), giving the variance the corresponding principal direction explains.
Example
With \sigma_1=8,\sigma_2=4,n=5: \lambda_1=64/4=16, \lambda_2=16/4=4.
Why: this is exactly the eigenvalue the covariance matrix would have produced, reached without ever computing that matrix.
-
4
Divide one \lambda_i by the sum of every \lambda_i to get that component's own share of the total variance.
Example
16/(16+4)=0.8, so the first component explains 80\% of the total variance.
Why: a raw variance number is hard to interpret alone; the proportion shows whether one or two components already capture most of the data's structure.
-
5
If X's own condition number \kappa(X) is known, compare it against \kappa(X^{\mathsf T}X)=\kappa(X)^2 to see how much worse the covariance route would have been.
Example
\kappa(X)=1000 gives \kappa(X^{\mathsf T}X)=1{,}000{,}000, a million-fold worse conditioning avoided entirely by Step 1's direct route.
Why: this is the concrete numerical payoff for skipping the covariance matrix, not just a theoretical preference.
Common mistakes
- Running the SVD on an un-centered data matrix; the link between \sigma_i and explained variance only holds once X is centered.
- Dividing by n instead of n-1 when converting a singular value into a sample variance.
- Forming X^{\mathsf T}X first out of habit, then squaring an already poor condition number for no benefit.
- Reading U's columns as the principal directions instead of V's; the two play different roles in this identity.