A Fourier coefficient is a projection wearing a different notation. Once functions are treated as vectors in their own right, one question gets asked: "how much of f points along \phi_n?" That question is answered by exactly the same shadow-onto-a-direction computation. It's the same computation that already answers "how much of \mathbf u points along \mathbf v?" for ordinary vectors. The inner product just becomes an integral, instead of a sum of products.
Projection is a shadow. Shine a light straight down onto the line through v. Then \operatorname{proj}_vu is where u's shadow lands. What's left over is u minus its shadow. That leftover points straight up from that line, at a right angle to it. This holds no matter which u or v you started with. The inner product is just the tool that measures how long that shadow is. That right-angle fact is actually where the shadow-length formula comes from. It's not a bonus fact checked afterward. Demand that the leftover piece meet v squarely. Solving that one demand for the shadow's length is exactly what produces the formula above.
An inner product \langle\, \cdot\,,\cdot\,\rangle generalizes the dot product to any vector space. It keeps the properties that matter: symmetry, linearity in each argument, and \langle v,v\rangle\geq0, with equality only at v=0. Two vectors are orthogonal when \langle u,v\rangle=0. The orthogonal projection of u onto a one-dimensional subspace spanned by v is \operatorname{proj}_v u = \frac{\langle u,v\rangle}{\langle v,v\rangle}\,v, and the residual u-\operatorname{proj}_vu is always orthogonal to v. The projection is the closest point to u that lies along v's direction. That formula is not asserted. It falls out of one requirement. Write u=cv+r, for some scalar c and leftover r. Demand \langle r,v\rangle=0. That's exactly the closest-point condition. Substituting r=u-cv gives \langle u-cv,v\rangle=0. So \langle u,v\rangle=c\langle v,v\rangle. Solving for c recovers c=\langle u,v\rangle/\langle v,v\rangle.
The shadow and the leftover, live
Drag u or v. The green segment is u's shadow along v; the dashed amber segment is what's left over. Watch the right-angle mark confirm the leftover always meets v squarely.
Loading interactive…
Worked example
Projecting one vector onto another
-
1
Project \mathbf u=(4,3) onto \mathbf v=(5,1).
-
2
\langle u,v\rangle = 4(5)+3(1)=23, and \langle v,v\rangle=25+1=26.
-
3\boxed{\operatorname{proj}_v u = \frac{23}{26}(5,1) = \left(\frac{115}{26},\frac{23}{26}\right)}
Loading interactive…
Worked example
Confirming orthogonality of the residual
-
1
Confirm that \mathbf u-\operatorname{proj}_v\mathbf u is orthogonal to \mathbf v=(5,1), using the projection above.
-
2
u-\operatorname{proj}_vu = (4,3)-\left(\tfrac{115}{26},\tfrac{23}{26}\right) = \left(\tfrac{104-115}{26},\tfrac{78-23}{26}\right) = \left(-\tfrac{11}{26},\tfrac{55}{26}\right).
-
3\left\langle\left(-\tfrac{11}{26},\tfrac{55}{26}\right),(5,1)\right\rangle = -\tfrac{55}{26}+\tfrac{55}{26} = 0.\qquad\boxed{\text{orthogonal, as required}}
Loading interactive…
Where this goes next
Fourier coefficients
A Fourier series coefficient is nothing but this projection formula, in disguise. The dot product gets replaced by \langle f,g\rangle=\int f(x)g(x)\,dx. And \mathbf v gets replaced by one basis function \phi_n: c_n = \langle f,\phi_n\rangle/\langle\phi_n,\phi_n\rangle. There's a reason an orthogonal set of basis functions is worth building. It's the same reason orthogonality mattered here. It's what lets each coefficient be computed on its own, with no interference from the others. Ch.03, "Fourier series and eigenfunction expansions," is where this projection formula turns into that coefficient formula for real.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
For the weighted dot product \mathbf u \cdot \mathbf v = 2u_1v_1 + 5u_2v_2 + 3u_3v_3, find \mathbf u \cdot \mathbf v for \mathbf u = (-1, 2, 4) and \mathbf v = (3, -1, 0).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
2(-1)(3) + 5(2)(-1) + 3(4)(0) = -6 - 10 + 0 = -16. Note the result is negative even though this is a legitimate inner product — positive definiteness only guarantees \mathbf u\cdot\mathbf u \ge 0 for a vector with itself, not that \mathbf u\cdot\mathbf v is nonnegative for two different vectors.
Correct answer
Also accepted:
Why
Multiply each weight by the matching pair of coordinates from \mathbf u and \mathbf v, then add the three terms.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Using the weighted dot product \mathbf u \cdot \mathbf v = 5u_1v_1 + 3u_2v_2 on \mathbb R^2, write the expression (in terms of u_1, u_2) for \mathbf u \cdot \mathbf u.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Any equivalent form counts — 2x+3, 3 + 2·x, … Use the keypad below the field for fractions, roots, and other symbols.
Why
Substituting v_1=u_1 and v_2=u_2 into the formula gives 5u_1u_1 + 3u_2u_2 = 5u_1^2 + 3u_2^2 — a sum of positive weights times squares, which is why positive definiteness always holds for weights that are all positive.
Correct answer
Also accepted:
Why
Substitute v_1=u_1 and v_2=u_2 directly into the weighted formula and simplify.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Using the weighted dot product \mathbf u \cdot \mathbf v = 2u_1v_1 + u_2v_2 on \mathbb R^2, compute \mathbf u \cdot \mathbf v for \mathbf u = (3, -2) and \mathbf v = (1, 5).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
2(3)(1) + (-2)(5) = 6 - 10 = -4. Each coordinate's product is scaled by its own weight (2 and 1) before the two terms are added.
Correct answer
Also accepted:
Why
Quick review
Practice from earlier coursework, pulled in for this topic.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
In the plane widget above, if the spanning vectors \mathbf u and \mathbf v happen to become parallel, what happens?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The widget doesn't silently fall back to using one vector — a degenerate span means there's no well-defined plane at all for the formula to project onto.
The residual only vanishes when \mathbf p already lies in W — parallel spanning vectors say nothing about where \mathbf p is, so the residual doesn't automatically become zero.
Two vectors span a plane only when they point in genuinely different directions; parallel (or one being \mathbf 0) vectors span at most a line.
Why
\mathbf u \times \mathbf v = \mathbf 0 exactly when \mathbf u and \mathbf v are parallel (or one is \mathbf 0), which is the widget's degenerate case — \operatorname{span}\{\mathbf u, \mathbf v\} shrinks to a line, and 'projection onto the plane' stops being well-defined.
Think about what happens to \operatorname{span}\{\mathbf u,\mathbf v\} once \mathbf u and \mathbf v point along the same line.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
According to the Best Approximation Theorem, \operatorname{proj}_W(\mathbf v) is the unique point of W that...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That's backwards — the theorem says the projection is the closest point of W to \mathbf v, not the farthest.
The projection generally is not orthogonal to \mathbf v itself — it's the residual \mathbf v - \operatorname{proj}_W(\mathbf v) that's orthogonal to W.
Matching norms isn't the theorem's claim at all — the theorem is entirely about distance to \mathbf v, and equal norm isn't guaranteed or required.
Why
The Best Approximation Theorem says \operatorname{proj}_W(\mathbf v) is the one point of W that makes \lVert \mathbf v - \mathbf w \rVert as small as possible — every other point of W is strictly farther from \mathbf v.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Using the same \mathbf w_1 = (1,1,0), \mathbf w_2 = (0,0,1), and \mathbf v = (3,5,7) from before, find the coefficient \frac{\mathbf v \cdot \mathbf w_2}{\mathbf w_2 \cdot \mathbf w_2}.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\mathbf v \cdot \mathbf w_2 = 3(0)+5(0)+7(1) = 7 and \mathbf w_2 \cdot \mathbf w_2 = 1, so the coefficient is 7/1 = 7.
Correct answer
Also accepted:
Why