Chapter 16's image-retrieval systems and Chapter 10's classical feature-based classifiers both lean on the same fact. A whole dataset of images can often be described by far fewer numbers than its raw pixel count suggests. This section recalls the machinery behind that fact. That machinery is the covariance matrix of an image dataset, its eigen-decomposition, and what "keeping only the top few directions" buys you. It covers exactly the depth Chapters 10 and 16 need.
Every image in a dataset is one point. It sits in an enormous space.
There's one coordinate per pixel. The covariance matrix is the
pixel-by-pixel version of an ordinary recipe. That recipe answers one
question: "which directions does this pile of points actually spread
out along?" linear-algebra built that same recipe for an ordinary 2D
scatterplot. Here, it just uses a million coordinates instead of two.
Diagonalizing that matrix answers the real question. These images could differ from one another in countless ways. Which handful of directions actually captures most of that real variation? The biggest eigenvalue's own eigenvector is the single most important such direction. Reshape it back into a picture, and that's what makes it an eigenimage.
Eigenimages genuinely look strange to a human eye — ghostly, faintly patterned. That's because they aren't a photo of anything. They're a direction of change. This whole recipe has a name: PCA, short for principal component analysis. Each of those directions is a principal component. "Eigenimage" is just what a principal component is called. That's its name once it's reshaped back into picture form.
Keeping only the top few eigenimages is the entire payoff. Instead of needing every pixel, a handful of numbers can describe how one image differs from another. That handful does the job. Think: how much of eigenimage 1, how much of eigenimage 2, and so on. It can capture almost the whole story. Sometimes, as the worked example below shows, it captures all of it, with a single direction.
Treat each of N flattened images as a point \mathbf x_i \in \mathbb R^n, where n is the pixel count. Their mean is \bar{\mathbf x}. Their covariance matrix is C = \frac1N\sum_i(\mathbf x_i-\bar{\mathbf x})(\mathbf x_i-\bar{\mathbf x})^{\mathsf T}. C measures exactly how the images in the dataset spread out, and along which directions. C is symmetric, so it diagonalizes as C = V\Lambda V^{\mathsf T}. This gives real eigenvalues \lambda_1\ge\lambda_2\ge\cdots\ge0 and orthogonal eigenvectors \mathbf v_k. Reshape any one eigenvector back into an image, and the result is an eigenimage. Eigenvalue \lambda_k measures how much of the dataset's total variance that one direction alone accounts for. The total variance is \operatorname{tr}(C)=\sum_k\lambda_k. Ranking by decreasing \lambda_k orders the eigenimages. At one end: "captures the most of how these images differ from each other." At the other: "barely matters." Dimensionality reduction keeps only the top K\ll n eigenimages. That describes every image by just K numbers — its coefficients along \mathbf v_1,\dots,\mathbf v_K — instead of its full n-pixel vector. This is the practical payoff whenever a dataset's real variation lives in far fewer directions than its raw pixel count. This whole procedure has a name: principal component analysis (PCA) — mean, covariance, eigen-decomposition, then keep the top K directions. An eigenimage is exactly what a principal component looks like, once it's reshaped back into an image instead of left as a plain vector.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Digital Image Processing
This chapter's own "eigenimage" is best described as...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The brightest single pixel value has nothing to do with eigenvectors or the covariance matrix at all.
A training photo is one of the raw images fed IN to this machinery -- an eigenimage is computed OUT of the covariance matrix, not selected from the dataset.
That describes the MEAN image, a separate quantity this chapter also uses -- an eigenimage specifically comes from the covariance matrix's own eigendecomposition.
Why
An eigenimage is exactly an eigenvector of the dataset's covariance matrix (linear-algebra ch.14 §7's own recalled machinery), reshaped via Section 2's vec inverse back into image form.
From Digital Image Processing
The low-rank idea recalled from linear-algebra ch.14 §8 carries over to a dataset of images as...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Bit-depth reduction (Chapter 1) is a completely different, per-pixel operation -- unrelated to keeping a subset of eigenimages.
Cropping changes an image's own resolution before it's even flattened -- the low-rank idea instead keeps the full resolution but represents it with fewer eigenimage coefficients.
Discarding whole images by brightness has nothing to do with ranking DIRECTIONS of variation by eigenvalue.
Why
Low-rank structure, recalled directly from linear-algebra ch.14 §8, means keeping only the top K eigenvectors (here, eigenimages) ranked by eigenvalue, rather than needing every one of the n dimensions to describe the dataset well.
Formula
Eigenvalues of a Symmetric 2×2 Matrix
also known as: the characteristic equation, real spectral case
Recalled from Advanced Multivariable Calculus · Matrices, determinants, and linear maps — first taught there.
For a symmetric matrix A = \begin{pmatrix} a & b \\ b & d \end{pmatrix}, the eigenvalues are the roots of the characteristic equation \det(A - \lambda I) = \lambda^2 - (a+d)\lambda + (ad - b^2) = 0, namely \lambda = \frac{(a+d) \pm \sqrt{(a-d)^2 + 4b^2}}{2}. Because the discriminant (a-d)^2 + 4b^2 is a sum of squares, it is never negative — a symmetric matrix always has two real eigenvalues (counted with multiplicity), never a complex pair.
\det(A - \lambda I) = (a-\lambda)(d-\lambda) - b^2 = \lambda^2 - (a+d)\lambda + (ad - b^2), a quadratic in \lambda. Apply the quadratic formula with A=1, B=-(a+d), C=ad-b^2: \lambda = \frac{(a+d) \pm \sqrt{(a+d)^2 - 4(ad-b^2)}}{2}. Expanding (a+d)^2 - 4(ad-b^2) = a^2 + 2ad + d^2 - 4ad + 4b^2 = (a-d)^2 + 4b^2 gives the stated discriminant. It is a sum of two squares, so it is \geq 0, and the two roots are always real.
A dataset whose variation lives along one direction
Step through past the covariance matrix to the two eigenimages -- compare their eigenvalues. One direction captures every bit of how these four images differ; the other captures none of it.
Loading interactive…
Worked example
Finding this dataset's eigenvalues
-
1
This dataset's covariance matrix works out to C=\begin{bmatrix}1000&-1000\\-1000&1000\end{bmatrix}. Using the symmetric 2\times2 eigenvalue formula above, with a=d=1000 and b=-1000, find both eigenvalues. The discriminant term is (a-d)^2+4b^2; here a-d=0, so it reduces to just 4b^2.
-
2\lambda = \frac{(a+d)\pm\sqrt{(a-d)^2+4b^2}}{2} = \frac{2000\pm\sqrt{0+4{,}000{,}000}}{2} = \frac{2000\pm2000}{2}\boxed{\lambda_1=2000,\ \ \lambda_2=0}
Check: \lambda_1+\lambda_2=2000=\operatorname{tr}(C)=1000+1000. The eigenvalues must always sum to the trace.
Loading interactive…
Worked example
A special case: one direction explains everything
-
1
From above, \lambda_1=2000 and \lambda_2=0, so the total variance is \operatorname{tr}(C)=2000. What percentage of this dataset's variance does the top eigenimage alone explain? Divide the top eigenvalue by the total.
-
2\frac{2000}{2000} = 100\%
The second eigenimage's own eigenvalue is exactly 0. This dataset's four images differ from their mean along only one direction. That's a special case: every image here is \bar{\mathbf x} plus some multiple of a single pattern. It's the cleanest possible illustration of Section 1's own low-rank idea. Keeping just K=1 of the 2 eigenimages here loses none of the dataset's real structure, not just most of it.
Loading interactive…
Where this goes next
Image retrieval and classical features
Chapter 16 and Chapter 10 both represent an image by a short vector of numbers. That's instead of its full pixel grid. Chapter 10's own PCA-for-vision feature step is exactly this section's machinery, unchanged. The classic eigenfaces technique is the historical example here. It made this approach famous for vision specifically. It uses a face-photo dataset's own top eigenimages. This was never just an abstract trick for shrinking data.
Check your understanding (counts toward your progress)
A symmetric 2\times2 covariance matrix has a=d=1000 and b=-1000 (using C=\begin{bmatrix}a&b\\b&d\end{bmatrix}). Using \lambda=\frac{(a+d)\pm\sqrt{(a-d)^2+4b^2}}{2}, what is the larger eigenvalue?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
(a-d)^2=0 and 4b^2=4{,}000{,}000, so \lambda = \frac{2000\pm2000}{2}, giving 2000 and 0 -- the larger root is 2000.
Correct answer
Also accepted:
Why
An eigenvalue \lambda_k of an image dataset's covariance matrix measures...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Average brightness is captured by the dataset's MEAN vector, not by any single eigenvalue of the covariance matrix.
The number of images, N, is a fixed count set before any eigendecomposition -- it isn't something an eigenvalue reports.
Resolution is a property of the original images' own shape, unrelated to how variance splits across directions.
Why
Each eigenvalue measures how much of the dataset's own spread lies along that eigenvector's particular direction, out of the total variance \operatorname{tr}(C)=\sum_k\lambda_k.
Explain what it means for a dataset's covariance matrix to have a zero eigenvalue, in terms of how the images in that dataset actually vary.
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Why
A zero eigenvalue means the dataset has NO variation at all along that eigenvector's direction -- every image in the dataset agrees perfectly along that particular direction (differing only along the directions with nonzero eigenvalues). This is the cleanest possible case of low-rank structure -- the dataset's real variety lives entirely in fewer directions than its raw pixel count would suggest.
A covariance matrix has eigenvalues \lambda_1=360 and \lambda_2=40. What percentage of the dataset's total variance does the top eigenimage explain?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Total variance is 360+40=400; the top eigenvalue's share is 360/400=0.9, or 90\%.
Correct answer
Also accepted:
Why
To treat a dataset of images with the covariance-and-eigendecomposition machinery, each image is first...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Reducing a whole image to one number would throw away essentially all of its content -- flattening keeps every pixel, just reorganized into a vector.
Real datasets are not restricted to 2-pixel images -- that limit belongs only to one specific teaching widget, not the underlying mathematics.
The histogram alone discards spatial and color information the covariance machinery is meant to use.
Why
Flattening turns an H\times W (or H\times W\times C) image into one long vector in \mathbb R^n, exactly the kind of point the covariance-and-eigendecomposition recipe expects.
Why do eigenimages typically look strange -- ghostly and faintly patterned -- to a human eye?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The eigendecomposition is an exact mathematical computation -- it doesn't corrupt or lose pixel data.
Resolution has nothing to do with why an eigenimage looks strange -- even a full-resolution eigenimage looks ghostly.
An eigenimage is not a photograph of a real scene at all, dim lighting or otherwise -- it's a mathematical direction, reshaped.
Why
An eigenimage represents a direction the dataset varies along, not any single real photo -- which is exactly why it doesn't resemble an ordinary picture of anything.
Practice: Covariance, eigenimages, and how much they explain (4 problems · doesn’t affect your progress)
Why does flattening every image into a vector before building the covariance matrix lose no real information?
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Why
Flattening only reorders the pixel values into a single long list -- every pixel's own value is still present, just addressed by one index instead of two (or three), the same reversible reshaping DIP's own tensor-to-vector convention uses.
A covariance matrix has eigenvalues \lambda_1=270 and \lambda_2=30. What percentage of total variance does the SECOND eigenimage explain?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Total variance is 270+30=300; the second eigenvalue's share is 30/300=0.1, or 10\%.
Correct answer
Also accepted:
Why
Sorting eigenimages by decreasing eigenvalue orders them from...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
The eigenvalue itself is exactly the amount of dataset variance that eigenvector's direction explains, so sorting by it ranks directions from most to least important.
most of the dataset's own variation captured, to least
A symmetric 2\times2 covariance matrix has a=d=500, b=-500. Using \lambda=\frac{(a+d)\pm\sqrt{(a-d)^2+4b^2}}{2}, what is the SMALLER eigenvalue?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
(a-d)^2=0, 4b^2=1{,}000{,}000, so \lambda=\frac{1000\pm1000}{2}, giving 1000 and 0 -- the smaller root is 0.
Correct answer
Also accepted:
Why