Solving A\mathbf{x}=\mathbf{b} by hand, using row operations on the matrix [A\,|\,\mathbf{b}], is already familiar. This course asks a follow-up question. A first linear algebra course doesn't need to ask it. What changes once the arithmetic is floating point, not exact fractions?
Row reduction is the same "remove one variable at a time" process from a first algebra course. It's just set up so it works for any number of equations. On paper, divide by a tiny nonzero number, or divide by a large one — it makes no difference. The fraction is just uglier.
Gaussian elimination reduces [A\,|\,\mathbf{b}] to row-echelon form. It uses three elementary row operations: swap two rows, scale a row by a nonzero constant, or add a multiple of one row to another. Then it recovers \mathbf{x} by back-substitution. Every step that eliminates an entry below a pivot divides by that pivot's value. Exact arithmetic never cares how small a nonzero pivot is — \frac{1}{0.0001} is just as exact as \frac{1}{1}.
Row reduction, one operation at a time
Step through eliminating the entries below each pivot and watch the augmented matrix reach row-echelon form.
Loading interactive…
Worked example
Back-substitution after elimination
-
1
Given row-echelon form \begin{bmatrix}1&2&|&5\\0&1&|&3\end{bmatrix}, find x_1,x_2.
-
2
From the bottom row: x_2=3.
-
3
Substitute into the top row: x_1+2(3)=5 \Rightarrow x_1=-1.
-
4\boxed{x_1=-1,\ x_2=3}
Loading interactive…
Worked example
A pivot that's technically nonzero but dangerously small
-
1
Suppose elimination produces a pivot of 0.0001, instead of a larger entry you could swap in. In exact arithmetic, is the algorithm still valid?
-
2
Yes. Exact arithmetic never cares how small a nonzero pivot is.
-
3
In floating-point arithmetic, though, divide by 0.0001 and you amplify whatever rounding error is already in that row — by roughly 10{,}000\times.
-
4\boxed{\text{mathematically fine, numerically dangerous}}
This is exactly the gap this course's own linear-systems chapter opens with.
Loading interactive…
Where this goes next
What changes with floating-point arithmetic
This course's own chapter on direct methods for linear systems picks up exactly here. It asks what goes wrong when that tiny pivot isn't exact. In floating-point arithmetic, divide by a number close to zero — but not exactly zero — and rounding error can blow up fast. That's why partial pivoting matters: choose the largest available pivot, not just any nonzero one. It's a numerical-stability requirement, not a style choice.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
A system's row echelon form has last two rows (0,2,-1,7) and (0,0,3,9), for unknowns x,y,z. Using z solved from the last row, what is y?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
The last row gives 3z=9 \Rightarrow z=3. Substituting into the row above, 2y-z=7 \Rightarrow 2y-3=7 \Rightarrow 2y=10 \Rightarrow y=5.
Correct answer
Also accepted:
Why
Solve the last row for z first, then substitute that value into the row above to solve for y.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Gaussian elimination reaches row echelon form. What additional step is needed to actually read off the values of the unknowns?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Row echelon form doesn't hand over every value directly — only the last row (with one unknown left) can be solved by inspection; the rest still need substitution.
Elimination only needs to run once, forward; running it again wastes the work already done and produces no new information.
The determinant tells you whether a unique solution exists at all — it doesn't produce the actual values of the unknowns.
Why
Row echelon form leaves a staircase where the bottom row has only one unknown, solvable directly. Back-substitution then works upward, plugging each newly-found value into the row above to peel off the next unknown.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Row 1 of an augmented matrix is (1, -3, 2, 8) and row 2 is (4, 1, -5, 3). What multiple of row 1 should be subtracted from row 2 to eliminate its first entry?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Row 2's first entry is 4 and row 1's leading entry is 1, so subtracting 4 times row 1 from row 2 sends the first entry to 4-4(1)=0.
Correct answer
Also accepted:
Why