Newton's method (optimization)

Applied to optimization, Newton's method repeats x^{(k+1)} = x^{(k)} - H(x^{(k)})^{-1}\nabla f(x^{(k)}) — the update that would land exactly on the minimum of the local quadratic (second-order Taylor) approximation to f at x^{(k)}. Near a well-behaved minimum it converges quadratically (see newtons-method-quadratic-convergence), far faster than plain gradient descent, but each step needs the Hessian and its inverse.