momentum (optimization)
Accumulating a running "velocity" from past gradients so consistent directions speed up and oscillating ones cancel out.
Classical (heavy-ball) momentum carries a velocity v^{(k)} forward across iterations: v^{(k+1)} = \beta v^{(k)} - \alpha\nabla f(x^{(k)}), x^{(k+1)} = x^{(k)} + v^{(k+1)}, for a momentum coefficient \beta\in[0,1). Consecutive gradients pointing roughly the same way reinforce each other in v, accelerating progress; consecutive gradients that flip sign (as in a narrow valley's zig-zag) partially cancel, damping the oscillation. Nesterov momentum evaluates the gradient at the look-ahead point x^{(k)}+\beta v^{(k)} instead of x^{(k)} itself, correcting the step using where the accumulated velocity is already carrying it rather than where it started.