Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

deep-learning

Gradient descent: the update rule

taught

Taught in

Requires

Required by

Local knowledge map

Local knowledge map

Fitting a line with gradient descent Hyperparameters Learning-rate schedules and warmup The Adam optimizer The full training loop, assembled, with torch.manual_seed Gradient descent with more than one parameter Momentum Plotting a loss curve from a training run The learning rate Training discipline for vision models -- splits, augmentation,… What gradient descent and backpropagation do (primer) torch.optim.SGD and .step() Gradient descent: the update rule Backpropagation Drawing a straight line with plt.plot Partial derivatives: one knob at a time Reading a loss curve The chain rule -- slopes multiply through a chain The gradient -- every partial derivative in one list The slope of sigmoid and ReLU loss.backward() and gradient accumulation The derivative: a formula for a curve's slope The loss landscape as a search space A loss function -- one number for how wrong The slope of a curve at one point, via a secant

Click to expand

Solid arrows are required prerequisites; dashed are recommended or helpful. Lower rows are more foundational. Click any box to jump to that concept.