Section 1's sign test on D and f_{xx} only works in two variables, and even there it's really shorthand for a fact about eigenvalues. The Hessian is always a symmetric matrix, and a symmetric matrix's eigenvalues are the one number that decides bowl, dome, or saddle in any number of variables at once. This section puts eigenvalues and symmetric matrices back on the table and connects them to a new object, the quadratic form, that turns a matrix's eigenvalue signs into a picture.
Theorem
The Characteristic Equation
Recalled from Linear Algebra — Vectors, Matrices, and the Geometry of Data · Eigenvalues and the characteristic equation — first taught there.
For an n \times n matrix A, the scalar \lambda is an eigenvalue of A if and only if \det(A - \lambda I) = 0.
By definition, \lambda is an eigenvalue of A exactly when there exists a nonzero vector \mathbf{x} with A\mathbf{x} = \lambda\mathbf{x}. Move everything to one side. Since \lambda\mathbf{x} = \lambda I\mathbf{x}, this rearranges to A\mathbf{x} - \lambda I\mathbf{x} = \mathbf{0}, \qquad\text{i.e.}\qquad (A - \lambda I)\mathbf{x} = \mathbf{0}. So \lambda is an eigenvalue of A precisely when the homogeneous system (A - \lambda I)\mathbf{x} = \mathbf{0} has a solution \mathbf{x} other than the trivial one — equivalently, precisely when \text{Nul}(A - \lambda I) contains more than just \mathbf{0}.
A square matrix has a nontrivial null space if and only if it fails to be invertible: if A - \lambda I were invertible, multiplying both sides of (A - \lambda I)\mathbf{x} = \mathbf{0} by (A - \lambda I)^{-1} would force \mathbf{x} = \mathbf{0}, ruling out a nonzero solution. So a nonzero \mathbf{x} can exist only when A - \lambda I is singular.
Finally, a square matrix is singular exactly when its determinant is zero. Chaining the equivalences together: \lambda \text{ is an eigenvalue of } A \iff (A-\lambda I)\mathbf{x} = \mathbf{0} \text{ has a nontrivial solution} \iff \det(A - \lambda I) = 0.
Sanity check: take A = I, the identity matrix. Then A - \lambda I = (1-\lambda)I, and \det\left((1-\lambda)I\right) = (1-\lambda)^n, which vanishes only at \lambda = 1. That matches the identity matrix's only eigenvalue: every nonzero vector satisfies I\mathbf{x} = 1\cdot\mathbf{x}, so \lambda = 1 is the sole eigenvalue, exactly as the characteristic equation predicts.
Theorem
Symmetric Matrices Are Orthogonally Diagonalizable
Recalled from Linear Algebra — Vectors, Matrices, and the Geometry of Data · Symmetric matrices and orthogonal diagonalization — first taught there.
Let A be an n \times n real matrix. Then A is symmetric — A = A^{\mathsf T} — if and only if there is an orthogonal matrix Q and a diagonal matrix D with A = QDQ^{\mathsf T}. The columns of Q form an orthonormal basis of \mathbb{R}^n made of eigenvectors of A, and the diagonal entries of D are the matching eigenvalues.
(\Leftarrow) Orthogonal diagonalizability forces symmetry. Suppose A = QDQ^{\mathsf T} with Q orthogonal and D diagonal. Diagonal matrices equal their own transpose, and transposing a product reverses the order of its factors, so A^{\mathsf T} = (QDQ^{\mathsf T})^{\mathsf T} = (Q^{\mathsf T})^{\mathsf T}D^{\mathsf T}Q^{\mathsf T} = QDQ^{\mathsf T} = A. So A is symmetric. The rest of the proof handles the harder direction.
Symmetric matrices have only real eigenvalues. Let \lambda be an eigenvalue of A, allowing for the moment that it might be complex, with eigenvector v (possibly with complex entries), so Av = \lambda v. Let \bar v denote the entrywise complex conjugate of v, and consider the scalar \bar v^{\mathsf T} A v. Computing it one way, \bar v^{\mathsf T} A v = \bar v^{\mathsf T}(\lambda v) = \lambda (\bar v^{\mathsf T} v) = \lambda \lVert v \rVert^2, since \bar v^{\mathsf T} v = \sum_i \overline{v_i} v_i = \sum_i |v_i|^2 = \lVert v \rVert^2 is real and positive (v \neq 0, being an eigenvector). Computing the same scalar the other way, using A^{\mathsf T} = A and that \bar v^{\mathsf T} A v is a 1\times 1 matrix and so equal to its own transpose, \bar v^{\mathsf T} A v = \left(\bar v^{\mathsf T} A v\right)^{\mathsf T} = v^{\mathsf T} A^{\mathsf T} \bar v = v^{\mathsf T} A \bar v. Taking the complex conjugate of Av = \lambda v gives A\bar v = \bar\lambda \bar v (since A has real entries, conjugation passes through it untouched), so v^{\mathsf T} A \bar v = v^{\mathsf T}(\bar\lambda \bar v) = \bar\lambda (v^{\mathsf T}\bar v) = \bar\lambda \lVert v\rVert^2, using v^{\mathsf T}\bar v = \overline{\bar v^{\mathsf T} v} = \lVert v\rVert^2 again. The two computations of \bar v^{\mathsf T} A v must agree: \lambda \lVert v \rVert^2 = \bar\lambda \lVert v \rVert^2. Since \lVert v \rVert^2 \neq 0, this forces \lambda = \bar\lambda, i.e. \lambda is real. So every eigenvalue of A is real, and consequently every eigenvalue has a real eigenvector (a real matrix minus a real multiple of I_n has a real null space).
Eigenvectors for distinct eigenvalues are orthogonal. Let Av_1 = \lambda_1 v_1 and Av_2 = \lambda_2 v_2 with \lambda_1 \neq \lambda_2. Using symmetry, \lambda_1 (v_1 \cdot v_2) = (Av_1) \cdot v_2 = v_1^{\mathsf T}A^{\mathsf T} v_2 = v_1^{\mathsf T}Av_2 = v_1 \cdot (Av_2) = \lambda_2(v_1 \cdot v_2). So (\lambda_1 - \lambda_2)(v_1 \cdot v_2) = 0, and since \lambda_1 \neq \lambda_2, this forces v_1 \cdot v_2 = 0.
Building Q by induction on n. The base case n=1 is trivial: any 1\times 1 matrix is already diagonal, and Q = [1] is orthogonal. For the inductive step, assume every symmetric (n-1)\times(n-1) matrix is orthogonally diagonalizable, and let A be a symmetric n \times n matrix. By the real-eigenvalue fact above, A has a real eigenvalue \lambda_1 with a real, nonzero eigenvector; rescale it to a unit vector q_1, so Aq_1 = \lambda_1 q_1 and \lVert q_1 \rVert = 1. Extend \{q_1\} to an orthonormal basis \{q_1, q_2, \dots, q_n\} of \mathbb{R}^n (extend to any basis and apply Gram–Schmidt, which leaves q_1 fixed since it is already a unit vector orthogonal to nothing else yet chosen), and let P be the orthogonal matrix with these vectors as columns.
The subspace W = \operatorname{span}(q_2,\dots,q_n) = q_1^\perp is invariant under A: for w \in W, (Aw)\cdot q_1 = w \cdot (Aq_1) = w \cdot (\lambda_1 q_1) = \lambda_1(w\cdot q_1) = 0, using symmetry in the first step exactly as above, so Aw is again orthogonal to q_1, i.e. Aw \in W. Consequently P^{\mathsf T}AP has the block form P^{\mathsf T}AP = \begin{bmatrix} \lambda_1 & 0 \\ 0 & A_1 \end{bmatrix} for some symmetric (n-1)\times(n-1) matrix A_1: the first column of P^{\mathsf T}AP is P^{\mathsf T}(Aq_1) = P^{\mathsf T}(\lambda_1 q_1) = \lambda_1 e_1 (since P^{\mathsf T}q_1 = e_1, q_1 being the first column of the orthogonal matrix P), giving the zero block below \lambda_1; and P^{\mathsf T}AP is symmetric because A is, which forces the zero block to the right of \lambda_1 as well. By the inductive hypothesis, A_1 = Q_1D_1Q_1^{\mathsf T} for some orthogonal (n-1)\times(n-1) matrix Q_1 and diagonal D_1. Setting Q = P\begin{bmatrix} 1 & 0 \\ 0 & Q_1 \end{bmatrix}, \qquad D = \begin{bmatrix} \lambda_1 & 0 \\ 0 & D_1 \end{bmatrix}, the block-diagonal factor is orthogonal (its columns are orthonormal, being e_1 together with the orthonormal columns of Q_1), so Q is a product of orthogonal matrices and hence orthogonal itself. Unwinding the block product, Q^{\mathsf T}AQ = \begin{bmatrix} 1 & 0 \\ 0 & Q_1^{\mathsf T} \end{bmatrix}P^{\mathsf T}AP\begin{bmatrix} 1 & 0 \\ 0 & Q_1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & Q_1^{\mathsf T} \end{bmatrix}\begin{bmatrix} \lambda_1 & 0 \\ 0 & A_1 \end{bmatrix}\begin{bmatrix} 1 & 0 \\ 0 & Q_1 \end{bmatrix} = \begin{bmatrix} \lambda_1 & 0 \\ 0 & Q_1^{\mathsf T}A_1Q_1 \end{bmatrix} = D, so A = QDQ^{\mathsf T} with Q orthogonal and D diagonal, completing the induction.
Sanity check: take the symmetric matrix A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}. Its characteristic equation is (2-\lambda)^2 - 1 = 0, so \lambda_1 = 3 and \lambda_2 = 1 — both real, as guaranteed. Solving (A-3I)v=0 gives v_1 = (1,1), and (A-I)v=0 gives v_2 = (1,-1); these two are orthogonal (v_1 \cdot v_2 = 1 - 1 = 0), exactly as the theorem predicts for eigenvectors of distinct eigenvalues. Normalizing gives q_1 = \frac{1}{\sqrt2}(1,1) and q_2 = \frac{1}{\sqrt2}(1,-1), and setting Q = \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2 \\ 1/\sqrt2 & -1/\sqrt2 \end{bmatrix}, D = \begin{bmatrix} 3 & 0 \\ 0 & 1 \end{bmatrix}, a direct multiplication confirms QDQ^{\mathsf T} = A.
A symmetric matrix is the best-behaved object Linear Algebra hands back: real eigenvalues only, and a set of eigenvectors that always meet each other at right angles, no exceptions to check first. Multiplying that guarantee into optimization is what a quadratic form does. Picture Q(\mathbf x)=\mathbf x^{\mathsf T}A\mathbf x as a machine that takes in a direction and reports back a single curvature number for stepping that way from the origin. If every direction reports a positive number, you're standing at the bottom of a bowl no matter which way you look -- that's positive definite. If every direction reports negative, you're at the top of a dome. If some directions report positive and others negative, you're on a saddle, exactly the mountain-pass shape section 1 described, and it's the eigenvalues' mixed signs, not any single formula, that are actually responsible for that shape.
An eigenvalue \lambda of a square matrix A solves \det(A-\lambda I)=0, the characteristic equation -- the values for which A\mathbf x=\lambda\mathbf x has a nonzero solution. When A is symmetric (A=A^{\mathsf T}), two guarantees Linear Algebra proved in full generality collapse this course's whole optimality theory onto one number: every eigenvalue of A is real (never a complex pair to worry about), and A always has a full set of orthogonal eigenvectors, with no defective case to rule out first. The new piece is the quadratic form Q(\mathbf x)=\mathbf x^{\mathsf T}A\mathbf x, a scalar built from a symmetric matrix and a vector. Its sign for every nonzero \mathbf x is decided entirely by A's eigenvalue signs: all positive means Q is positive definite (a bowl, sitting entirely above the origin except at \mathbf x=\mathbf 0 itself); all negative means negative definite (a dome); mixed signs mean indefinite (a saddle). That is exactly the Hessian classification from section 1, restated for a matrix of any size with no 2\times2 shortcut required.
Where det(A - lambda I) crosses zero
Predict first: does det(A - lambda I) cross zero once, twice, or never across this domain? Drag lambda along the axis and watch the dashed guide's height (the value of det(A - lambda I)) shrink toward zero as you approach a crossing.
Loading interactive…
A symmetric matrix's eigenvectors, ready-made perpendicular
Before stepping through, predict: are these two eigenvectors of A perpendicular to each other? Step through placing them as columns of P and their eigenvalues into D, then compare P's two columns directly.
Loading interactive…
Reading definiteness from a quadratic form's level curve
Edit A's entries -- a positive definite A always closes into an ellipse, an indefinite A always opens into a hyperbola. Try flipping one diagonal entry negative and watch the shape change without touching anything else.
Loading interactive…
Worked example
Classifying the quadratic form of A=\begin{pmatrix}3&1\\1&3\end{pmatrix}
-
1
Find the eigenvalues of A=\begin{pmatrix}3&1\\1&3\end{pmatrix} and use them to classify the quadratic form Q(\mathbf x)=\mathbf x^{\mathsf T} A\mathbf x.
-
2
Characteristic equation: \det(A-\lambda I)=(3-\lambda)^2-1=0, so (3-\lambda)^2=1 and 3-\lambda=\pm1.
-
3no complex-pair case to check -- symmetry rules it out automatically
\lambda=3-1=2 or \lambda=3+1=4. Both roots are real, as guaranteed for any symmetric matrix.
no complex-pair case to check -- symmetry rules it out automatically
-
4
Both eigenvalues are positive (2>0 and 4>0), so Q is positive definite.
-
5
\boxed{\lambda=2,4;\ Q(\mathbf x)=\mathbf x^{\mathsf T}A\mathbf x\text{ is positive definite.}} Every nonzero input direction produces a positive output -- A's associated surface is a bowl, matching the ellipse the widget above draws for these exact entries.
Where this goes next
Second-order optimality conditions, in any number of variables
This section's whole payoff arrives the moment a Hessian has more than two rows: section 1's D-and-f_{xx} shortcut has nothing to say about a 3\times3 or 4\times4 Hessian, but "check whether every eigenvalue is positive" works at any size, because the Hessian is always symmetric and this section's guarantee always applies. Convexity, still several chapters away, is nothing more than this same test demanded to hold at every point in a function's domain at once, not just at one candidate optimum.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
If \lambda satisfies \det(A-\lambda I)=0, what does that say about the matrix A - \lambda I itself?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
A zero determinant only forces the matrix to be non-invertible; it does not force every individual entry to be 0. Plenty of nonzero singular matrices exist, e.g. \begin{bmatrix}1&1\\1&1\end{bmatrix}.
A-\lambda I = A would require \lambda = 0 specifically, not whatever root the characteristic equation happens to produce.
The sign of the entries has nothing to do with whether the determinant is zero.
Why
A square matrix has determinant zero exactly when it is singular, i.e. some nonzero vector lies in its null space — that null-space vector is the eigenvector belonging to \lambda.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Simplify \det(A - \lambda I) into a single polynomial in \lambda for A = \begin{bmatrix} 1 & 3 \\ 2 & 0 \end{bmatrix}.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Any equivalent form counts — 2x+3, 3 + 2·x, … Use the keypad below the field for fractions, roots, and other symbols.
Why
\det(A-\lambda I) = (1-\lambda)(0-\lambda) - (3)(2) = \lambda^2 - \lambda - 6.
Correct answer
Also accepted:
Why
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
For A = \begin{bmatrix} 5 & 4 \\ 1 & 2 \end{bmatrix}, expand \det(A - \lambda I) into standard polynomial form in \lambda.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Any equivalent form counts — 2x+3, 3 + 2·x, … Use the keypad below the field for fractions, roots, and other symbols.
Why
\det(A-\lambda I) = (5-\lambda)(2-\lambda) - (4)(1) = \lambda^2 - 7\lambda + 10 - 4 = \lambda^2 - 7\lambda + 6.
Correct answer
Also accepted:
Why
Write out A - \lambda I first, then expand its 2\times 2 determinant using (a-\lambda)(d-\lambda) - bc.
Quick review
Practice from earlier coursework, pulled in for this topic.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
What is \|(6, 8)\|, the length needed to normalize this vector into a unit eigenvector?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\|(6,8)\| = \sqrt{6^2+8^2} = \sqrt{36+64} = \sqrt{100} = 10 — dividing the vector by this length is exactly the normalization step that turns an eigenvector into a column of an orthogonal Q.
Correct answer
Also accepted:
Why
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
A symmetric matrix is written A = Q\Lambda Q^{\mathsf T} where \Lambda has diagonal entries 6 and -2. Find \det(A).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\det(A) = \det(Q)\det(\Lambda)\det(Q^{\mathsf T}), and for an orthogonal matrix \det(Q)\det(Q^{\mathsf T}) = \det(Q)^2 = 1 (since Q^{\mathsf T}Q=I forces \det(Q)=\pm1), leaving \det(A)=\det(\Lambda)=6(-2)=-12 — the product of the eigenvalues, as expected.
Correct answer
Also accepted:
Why
Recall that \det(A) = \det(Q)\det(\Lambda)\det(Q^{\mathsf T}), and that \det(Q)\det(Q^{\mathsf T}) = 1 for an orthogonal Q.
From Linear Algebra — Vectors, Matrices, and the Geometry of Data
Which of these matrices is symmetric?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Close, but the off-diagonal entries have opposite signs (5 and -5) — transposing swaps their positions but not their signs, so this matrix does not equal its own transpose.
Transposing this matrix gives \begin{bmatrix}0&0\\1&1\end{bmatrix}, which is not the same matrix — check entry (1,2) against entry (2,1).
Entry (1,2) is 3 but entry (2,1) is 5 — for a symmetric matrix those two positions must hold the same number.
Why
A matrix is symmetric exactly when A=A^{\mathsf T}, i.e. entry (i,j) equals entry (j,i) for every pair. Only the first matrix has matching off-diagonal entries (5 and 5).
Practice: From eigenvalues to a quadratic form's shape (3 problems · doesn’t affect your progress)
Find the smaller eigenvalue of A=\begin{pmatrix}5&2\\2&5\end{pmatrix}.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
\det(A-\lambda I)=(5-\lambda)^2-4=0\Rightarrow5-\lambda=\pm2\Rightarrow\lambda=3\text{ or }7; the smaller is 3.
Correct answer
Also accepted:
Why
Both eigenvalues of the symmetric matrix A from the previous problem (3 and 7) are positive. What does this say about Q(\mathbf x)=\mathbf x^{\mathsf T}A\mathbf x?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Negative definite needs every eigenvalue negative -- both of these are positive.
Indefinite needs mixed signs -- both eigenvalues here share the same sign.
For a symmetric matrix, the eigenvalue signs are exactly what determine definiteness -- nothing else is needed.
Why
Every eigenvalue of a symmetric matrix positive means the quadratic form is positive for every nonzero input -- positive definite, by definition.
Q is positive definite
For A=\begin{pmatrix}5&2\\2&5\end{pmatrix} and \mathbf x=(1,1), compute Q(\mathbf x)=\mathbf x^{\mathsf T}A\mathbf x.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
A\mathbf x=(5+2,2+5)=(7,7), so \mathbf x^{\mathsf T}A\mathbf x=(1)(7)+(1)(7)=14.
Correct answer
Also accepted:
Why