Recipe

Solve a generalized eigenvalue problem

Use when: you're asked to maximize or minimize x^{\mathsf T}Ax subject to x^{\mathsf T}Bx=1, or directly to solve Ax=\lambda Bx, with B symmetric positive definite

Goal: every generalized eigenvalue and its matching eigenvector, normalized against B rather than against the identity

Taught in Constrained quadratic optimization and the generalized eigenvalue problem

1. 1

   Scale every entry of B by \lambda, subtract the result from A entrywise, then set the determinant of what's left equal to zero.

   Example

   For A=\begin{bmatrix}3&0\\0&1\end{bmatrix}, B=\begin{bmatrix}2&0\\0&1\end{bmatrix}: A-\lambda B=\begin{bmatrix}3-2\lambda&0\\0&1-\lambda\end{bmatrix}.

   Why: rearranging Ax=\lambda Bx gives (A-\lambda B)x=0 directly, and a nonzero x solves that exactly when A-\lambda B is singular.

2. 2

   Expand the determinant from Step 1 and solve the polynomial equation it produces, the same way as an ordinary characteristic equation.

   Example

   \det(A-\lambda B)=(3-2\lambda)(1-\lambda)=0, giving \lambda=3/2 or \lambda=1.

   ⚡ Shortcut

   When A and B are both already diagonal, skip the determinant entirely: each generalized eigenvalue is just A_{ii}/B_{ii}, matching diagonal entries of the two matrices. Here 3/2 and 1/1=1, exactly matching.

   Why: every root of this determinant is, by construction, a \lambda where A-\lambda B collapses a direction, so the roots are exactly the generalized eigenvalues.

3. 3

   Solve (A-\lambda B)x=0 for each root found in Step 2; the nonzero solutions are the generalized eigenvectors.

   Example

   At \lambda=3/2: A-1.5B=\begin{bmatrix}0&0\\0&-0.5\end{bmatrix}, forcing x_2=0, so x=(1,0) up to scale.

   Why: this is the same null-space argument as the ordinary eigenvalue problem, just against A-\lambda B instead of A-\lambda I.

4. 4

   Divide the eigenvector from Step 3 by the right constant so the ORIGINAL constraint, x^{\mathsf T}Bx=1, holds exactly, not x^{\mathsf T}x=1.

   Example

   For x=(1,0), x^{\mathsf T}Bx=2, so the rescaled vector is (1,0)/\sqrt2, giving x^{\mathsf T}Bx=2(1/\sqrt2)^2=1.

   • the largest \lambda from Step 2 is wanted — this rescaled eigenvector achieves \max_{x^{\mathsf T}Bx=1}x^{\mathsf T}Ax exactly.
   • the smallest \lambda from Step 2 is wanted — the same rescaling on ITS eigenvector achieves the constrained minimum instead.

   Why: the constraint the whole problem was posed under is x^{\mathsf T}Bx=1, so only a B-normalized eigenvector actually satisfies it, unlike the ordinary Rayleigh quotient's unit-length normalization.