Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

descent direction

A direction d along which the objective is guaranteed to decrease, at least for a small enough step.

At a point x with \nabla f(x)\ne\mathbf 0, a direction d is a descent direction if \nabla f(x)^\top d < 0 — the directional derivative of f along d is negative, so f(x+\alpha d)<f(x) for all sufficiently small \alpha>0. Every first-order descent method (gradient descent, Newton's method, conjugate gradient, and more) is an instance of picking some descent direction, then a step size, at every iteration.

Read the full section →
Graph settings

Calculator