Recipe

Run one step of Newton's method for optimization

Use when: you have a differentiable $f$ with its gradient and Hessian available at the current iterate $x^{(k)}$

Goal: the next iterate $x^{(k+1)}$ , and eventually a stationary point once the stopping check passes

Taught in Newton's method for optimization

Quick recipe

Compute the gradient and Hessian at $x^{(k)}$
Solve $H\,\Delta x=-\nabla f$ for the step
Update $x^{(k+1)}=x^{(k)}+\Delta x$
Check a stopping criterion, then repeat from Step 1

1
Evaluate $\nabla f$ and $H$ at the current iterate. Both come from the same $f$ you started with, so no new function needs defining.

Example

For $f(x,y)=x^2+xy+y^2-4x-5y$ at $x^{(0)}=(0,0)$ : $\nabla f=(2x+y-4,\,x+2y-5)$ gives $g^{(0)}=(-4,-5)$ , and $H=\begin{bmatrix}2&1\\1&2\end{bmatrix}$ everywhere, since $f$ is already quadratic.

Why: Newton's method needs both the slope and the local curvature to build the quadratic model it's about to minimize.
2
Solve this linear system directly for $\Delta x$ — invert $H$ if that's convenient, or solve the system some other way — rather than guessing a step size.

Example

Continuing the same point: $H^{-1}=\dfrac13\begin{bmatrix}2&-1\\-1&2\end{bmatrix}$ , so $\Delta x=-H^{-1}g^{(0)}=-\dfrac13\begin{bmatrix}2&-1\\-1&2\end{bmatrix}\begin{bmatrix}-4\\-5\end{bmatrix}=-\begin{bmatrix}-1\\-2\end{bmatrix}=\begin{bmatrix}1\\2\end{bmatrix}$
- $H(x^{(k)})$ is invertible with all-positive eigenvalues — solve directly; the step points toward a genuine local minimum of the quadratic model.
- $H(x^{(k)})$ is singular, or has a negative eigenvalue — the system may have no solution, or may solve to a saddle or maximum instead; Section 3 covers diagnosing and repairing this, so watch for it and don't trust the result blindly.
Why: solving the system exactly jumps straight to the minimizer of the local quadratic model, instead of only stepping a little way downhill.
3
Add the step to the current iterate to get the next one.

Example

$x^{(1)}=x^{(0)}+\Delta x=(0,0)+(1,2)=(1,2)$ — solving $2x+y=4,\ x+2y=5$ directly confirms $(1,2)$ is exactly the minimizer.

Why: because $f$ is exactly quadratic here, the local model IS $f$ , so this single jump lands exactly on the minimizer; for a non-quadratic $f$ the same update only lands closer.
4
Check whether $\nabla f(x^{(k+1)})$ (or the step size) is small enough to stop. If not, treat $x^{(k+1)}$ as the new $x^{(k)}$ and return to Step 1.

Example

Switching to $f(x)=\dfrac{x^3}{3}-x+1$ from $x_0=2$ , where $g=f'=x^2-1$ : $x_1=2-\dfrac{4-1}{4}=1.25$ ( $g(x_1)=0.5625$ , not small — repeat); $x_2=1.25-\dfrac{0.5625}{2.5}=1.025$ ( $g(x_2)=0.050625$ , repeat); $x_3\approx1.000305$ ( $g(x_3)\approx0.00061$ , small enough to stop). $f''(1)=2>0$ confirms the stopping point is a genuine local minimum, not just any stationary point.

Why: without this check the iteration never stops, and skipping the Hessian-sign check at the end means accepting a saddle or maximum without noticing.

Common mistakes

Dropping the minus sign when solving $H\,\Delta x=-\nabla f$ , which turns a descent step into an ascent step.
Treating whatever stationary point the iteration reaches as a minimum without checking the Hessian's eigenvalue signs there.
Stopping once $x^{(k)}$ itself looks unchanged, without separately checking that $\nabla f$ is actually near $\mathbf 0$ .
Recomputing $H$ from scratch at every step even when $f$ is exactly quadratic and $H$ never changes.