optimization
Stochastic gradient descent
taught
Taught in
-
Stochastic gradient descent
Optimization
-
Mini-batches and epochs
Optimization
-
Escaping saddle points with noise
Optimization
-
When a population method beats a gradient method
Optimization
-
Choosing an algorithm family, revisited
Optimization
-
Comparing algorithms on the same problem
Optimization
Requires
- descent-direction-framework required
Required by
- Adaptive gradient methods (Adagrad, RMSProp, Adam) required
- Choosing an optimization algorithm by problem structure required
- Learning-rate schedules required
- Minibatch SGD for linear regression required
- Minibatch and stochastic gradient descent helpful
- Optimization vocabulary for ML: gradient descent and constraints required
Local knowledge map
Local knowledge map
Click to expand
Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.