adaptive gradient method
A gradient method that gives each parameter its own automatically-tuned step size from its own recent gradient history.
An adaptive gradient method scales each parameter's effective step size by that parameter's own accumulated squared-gradient history: Adagrad accumulates a running sum, RMSProp replaces the sum with an exponentially-weighted moving average so old history fades, and Adam layers a momentum-style running average of the gradient itself on top of RMSProp's scaling, with a bias correction on both averages for their first few steps.
Point table
Graph settings
Calculator