Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Recipe

Compute a truncated SVD's approximation error

Use when: you truncate A's SVD to rank k and need the error \lVert A-A_k\rVert, or want to know if a better rank-k matrix could exist

Goal: the operator-norm and Frobenius-norm error of the rank-k truncation, both read directly off the discarded singular values

Taught in The Eckart-Young theorem

Quick recipe

  1. List singular values in decreasing order and pick k
  2. Form A_k by dropping every singular value past k
  3. Compute the operator-norm error as the first discarded singular value
  4. Compute the Frobenius-norm error if that norm is what's being asked for
  5. State the optimality guarantee with no further checking
  1. 1

    Order \sigma_1\ge\sigma_2\ge\cdots\ge\sigma_r, then decide how many of the largest ones to keep, k.

    Example

    A=\text{diag}(5,3,1), so r=3; truncate to k=2.

    Why: the truncation A_k is defined entirely by which singular values survive, so this ordering has to be settled first.

  2. 2

    Keep \sigma_1,\dots,\sigma_k paired with their own \mathbf u_i,\mathbf v_i; discard the rest.

    Example

    A_2 keeps \sigma_1=5,\sigma_2=3 and discards \sigma_3=1, so A_2=\text{diag}(5,3,0).

    Why: this is literally the object the theorem is about, so every remaining step assumes A_k is built exactly this way.

  3. 3

    \lVert A-A_k\rVert=\sigma_{k+1} directly, with no matrix subtraction needed at all.

    Example

    \sigma_{k+1}=\sigma_3=1, so \lVert A-A_2\rVert=1.

    Why: the theorem's proof shows this equality directly, so the largest discarded singular value already is the error.

  4. 4

    \lVert A-A_k\rVert_F=\sqrt{\sigma_{k+1}^2+\cdots+\sigma_r^2}, the square root of every discarded singular value squared and summed.

    Example

    Only \sigma_3=1 is discarded here, so \lVert A-A_2\rVert_F=\sqrt{1^2}=1; with more than one discarded value the two norms would typically differ.

    • only the single worst discarded direction matters — use the operator norm from Step 3 instead.
    • the total error across every discarded direction matters — use the Frobenius norm here.

    Why: the two norms answer different questions, one about the single worst discarded direction and one about every discarded direction combined.

  5. 5

    Because Eckart-Young proves this bound applies to every possible rank-k matrix at once, nothing built any other way can beat the number from Step 3 or 4.

    Example

    For k=1 on A=\text{diag}(4,3,1), no rank-1 matrix at all can beat error 3=\sigma_2; the truncation already achieves the theoretical minimum.

    Why: this is what lets the recipe end the moment Step 3 or 4 is done, with no separate search left to run.

Common mistakes

  • Computing \lVert A-A_k\rVert as the largest KEPT singular value instead of the largest DISCARDED one.
  • Mixing up the operator-norm formula (one term) with the Frobenius-norm formula (a sum of squares) and using the wrong one.
  • Believing a specific alternative rank-k matrix needs to be checked; the theorem already covers every alternative at once.
  • Forgetting that k counts KEPT singular values, so the first discarded one is \sigma_{k+1}, not \sigma_k.