Recipe
Run Newton's method for a nonlinear system
Use when: you have formulas for every F_i and every partial derivative \partial F_i/\partial x_j, and a starting guess \mathbf{x}_0 reasonably close to a root of F(\mathbf{x})=\mathbf{0}
Goal: a sequence of vector guesses converging rapidly to a root of the system, each one built by solving one linear system
Taught in Newton's Method for Systems: the Jacobian
Quick recipe
-
1
Differentiate every F_i with respect to every x_j before running any numbers -- the same matrix gets reused, unevaluated, at every step.
Example
For F_1=x_1^2+x_2^2-4,\ F_2=x_1^2-x_2-1: J(\mathbf{x})=\begin{bmatrix}2x_1 & 2x_2\\ 2x_1 & -1\end{bmatrix}.
Why: Just as one derivative f'(x) was reused at every single-variable Newton step, this whole matrix is reused too. Only the numbers plugged into it change from step to step.
-
2
Plug the current guess into every F_i and every entry of J.
Example
\mathbf{x}_0=(1,1): F(1,1)=(1+1-4,\ 1-1-1)=(-2,-1). J(1,1)=\begin{bmatrix}2&2\\2&-1\end{bmatrix}.
Why: These are the only two objects the linear approximation from the Textbook tab needs: where F actually is, and how it's locally tilted there.
-
3
Solve this small linear system by elimination (Chapter 7's own partial-pivoting Gaussian elimination), exactly like any other linear system. Do NOT compute J^{-1} and multiply.
Example
\begin{bmatrix}2&2\\2&-1\end{bmatrix}\Delta\mathbf{x}=\begin{bmatrix}2\\1\end{bmatrix}. Eliminating: \Delta\mathbf{x}=\left(\dfrac23,\dfrac13\right).
- J(\mathbf{x}_n) is singular (no valid pivot survives elimination) — the linear system has no unique solution, and this step is undefined. This is exactly the failure mode Section 4's steepest-descent fallback exists for.
- J(\mathbf{x}_n) is nonsingular — proceed normally, as above.
Why: This step is where Newton's method for systems and "just invert the Jacobian" genuinely part ways. Forming J^{-1} explicitly costs roughly three times the arithmetic of one elimination solve. It is also more vulnerable to amplifying round-off (Chapter 7 §2's own lesson, unchanged here). And J(\mathbf{x}_n) changes every step, so the matrix has to be re-solved FRESH at every single iteration anyway. There is no point in the computation where an explicit inverse buys anything a direct solve doesn't.
-
4
Go back to Step 2 with \mathbf{x}_{n+1} in place of \mathbf{x}_n. Stop once \lVert F(\mathbf{x}_n)\rVert (or \lVert\Delta\mathbf{x}\rVert) is below the required tolerance.
Example
\mathbf{x}_1=(1,1)+\left(\tfrac23,\tfrac13\right)=\left(\tfrac53,\tfrac43\right)\approx(1.66667,\,1.33333). Continuing: \mathbf{x}_2\approx(1.52424,\,1.30303), \mathbf{x}_3\approx(1.51750,\,1.30278) -- already correct to 4 decimal places after only 3 steps, and a fourth step reaches (1.51749,\,1.30278), correct to 5.
Why: Each pass repeats Steps 2-3 at the newest point. The same quadratic convergence Chapter 2 §6 proved for one equation carries over here too (Section 2's own worked examples below show it holding step by step). That is exactly why so few repetitions are typically enough.
Common mistakes
- Computing J^{-1} explicitly and multiplying it by -F(\mathbf{x}_n), instead of solving the linear system directly. This costs more arithmetic and is more exposed to round-off. It gives the same answer in exact arithmetic, so nothing is gained.
- Reusing the same J(\mathbf{x}_n) at the next step instead of recomputing it at the NEW \mathbf{x}_{n+1}. This is a different, much weaker method -- not Newton's method with a shortcut taken. (Section 3 studies exactly this idea, deliberately, as its own separate method.)
- Forgetting that \Delta\mathbf{x} solves for the STEP, not the new point itself. The update is \mathbf{x}_{n+1}=\mathbf{x}_n+\Delta\mathbf{x}, not \mathbf{x}_{n+1}=\Delta\mathbf{x}.
- Trusting a starting guess far from any root to "find its way." Chapter 2 §6 gave this same warning for one equation. The Jacobian only describes F's LOCAL linear behavior, and there are more ways for a guess to be "too far" once there's more than one unknown (Section 4).