Chapter 11 builds a convolutional neural network for vision tasks. It
assumes you already know what a neuron computes. It also assumes you
know, in one sentence, what "training" actually does to one. This
course does not require deep-learning as a prerequisite. So this
section is a genuine, but deliberately shallow, primer: everything
Chapter 11 needs, and nothing more. The full mathematics lives
elsewhere — the calculus behind gradient descent, the chain-rule
derivation of backpropagation — in deep-learning chapters 2, 4, 9,
and 10. This section is where to start, not where that depth lives.
Think of one neuron as a tiny voting machine. It looks at several numbers coming in. It decides how much to trust each one — that's the weight. A bigger weight means "listen to this input more." It also adds a fixed opinion of its own, regardless of the inputs — that's the bias. It totals everything up. Then it runs that total through a fixed rule that squashes it into a final answer. That rule is the activation function; this section doesn't need to know which one. A whole network is just many of these tiny voting machines, arranged in layers. Each layer's votes feed into the next layer, as its own input. The last layer produces one final answer. That whole left-to-right pass is the "forward pass."
Training a network means adjusting every single weight and bias in the
whole network, until its final answers get good. "Good" is measured by
one number, the loss: large when the network is wrong, small when it's
right. Gradient descent is the idea "nudge every knob a little, in
whichever direction makes the loss number go down." Backpropagation is
just the bookkeeping trick that makes computing all those nudges
efficient. Instead of separately figuring out, from scratch, how much
each of possibly millions of weights affects the final loss, it works
backward one layer at a time. It reuses work it already did for the
layer after it. Nothing here proves that trick actually works. Nothing
here walks through how the backward arithmetic runs. deep-learning's
own chapters do that. This section only owes you what the words mean.
An artificial neuron takes several numeric inputs x_1,\dots,x_n. It multiplies each by its own weight w_1,\dots,w_n, and adds a bias b. It then passes the result through a fixed nonlinear activation function \varphi: a = \varphi\!\left(\sum_i w_i x_i + b\right). A layer is a group of neurons computing this same kind of sum. Each neuron works from the identical inputs, with its own weights and bias. A network's forward pass evaluates layer after layer in order. Each layer's outputs feed in as the next layer's inputs, ending at a final prediction.
Training means choosing every weight and bias so the network's
predictions are good. That's measured by a single number, the
loss: large when predictions are wrong, small when they're right.
Gradient descent nudges every weight a small step, in whichever
direction reduces the loss. Backpropagation is the specific
algorithm that computes how much to nudge each individual weight. It
applies the calculus chain rule, working backward from the output layer
to the input layer. It reuses each layer's own already-computed result,
rather than recomputing every weight's effect on the loss from scratch.
Nothing here derives why the chain rule gives the right nudge.
Nothing here shows how the backward pass is actually computed, layer
by layer. That derivation is deep-learning's own job, in the chapters
named above.
One forward pass, by hand
Two inputs from an image patch, x1=0.8 and x2=0.2. Before pressing anything, estimate whether the hidden sum z1 = 0.6(0.8) + (-0.4)(0.2) + 0.1 comes out positive or negative. Press Step to check, then click a hidden node to see its own sum spelled out with the real numbers. Step again for the output -- is the final prediction above or below 0.5?
Loading interactive…
Worked example
Computing one neuron's output by hand
-
1
A single neuron has inputs x_1=0.8, x_2=0.2, weights w_1=0.6, w_2=-0.4, and bias b=0.1. Compute its pre-activation sum z. Multiply each input by its own weight, then add the bias.
-
2z = w_1x_1 + w_2x_2 + b = 0.6(0.8) + (-0.4)(0.2) + 0.1 = 0.48 - 0.08 + 0.1 = 0.5
This is the widget's own first hidden unit — pressing Step once reveals this exact sum.
Loading interactive…
Worked example
Reading a forward pass's final prediction
-
1
Using the widget above, the two hidden units produce activations a_1\approx0.6225 and a_2\approx0.5349 after their own sigmoid. The output neuron's weights are 0.7 and 0.9 with bias 0.05. Compute the output's pre-activation sum z_{\text{out}} (its final activation, run through sigmoid, is the network's prediction). Treat a_1 and a_2 as this neuron's own two inputs.
-
2z_{\text{out}} = 0.7(0.6225) + 0.9(0.5349) + 0.05 \approx 0.4358+0.4814+0.05 \approx 0.9672
Passed through sigmoid, this gives a final prediction around 0.72 — above 0.5. Why 0.5 is a meaningful cutoff isn't this section's job to explain. Neither is how training would adjust these exact weights, if this prediction were wrong. That's gradient descent and backpropagation, named above, taught in full starting
deep-learningchapter 9.
Loading interactive…
Where this goes next
Convolutional neural networks for vision
Chapter 11 builds a CNN for image classification, directly on this section's vocabulary. A CNN is still a stack of layers, computing weighted sums and nonlinearities. It's still trained by gradient descent. It's still adjusted by backpropagation. Chapter 11 adds only one genuinely new idea. Some of those weighted sums are organized as a convolution (Section 2 of this chapter), instead of a plain weighted sum over every input at once. And the kernel's own numbers are learned by this same training process, rather than chosen by an engineer.
Check your understanding (counts toward your progress)
A neuron has inputs x_1=0.5, x_2=0.4, weights w_1=2, w_2=-1, and bias b=0.1. Compute its pre-activation sum z (before applying any activation function).
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
z = w_1x_1+w_2x_2+b = 2(0.5)+(-1)(0.4)+0.1 = 1.0-0.4+0.1 = 0.7.
Correct answer
Also accepted:
Why
What specific job does backpropagation do that plain "gradient descent" by itself does not explain?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
How many training examples to use is a separate training-setup decision, not something backpropagation determines.
The activation function is chosen when the network is designed, before any training (or backpropagation) runs at all.
Setting the initial weights happens before training begins -- backpropagation's job starts only once training is already underway.
Why
Gradient descent says "nudge every weight toward lower loss"; backpropagation is the specific algorithm that computes exactly how much to nudge each weight, via the chain rule, working backward through the layers.
The "loss" in a training setup is...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The number of layers is a fixed structural choice, unrelated to how good any particular prediction is.
How many training images exist says nothing about whether the network's CURRENT predictions are accurate.
The loss is specifically the number that training is trying to reduce -- it changes constantly as weights are adjusted.
Why
The loss is one number, large when predictions are bad and small when they're good, and it's exactly what training tries to shrink.
A network's "forward pass" refers to...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Every layer's own computation is needed -- a later layer depends on its inputs, which come directly from the layer before it.
Adjusting weights to reduce error is what TRAINING does, not the forward pass itself, which only produces a prediction from fixed weights.
Choosing the number of layers is a design decision made before training even starts -- it isn't part of running a forward pass.
Why
The forward pass evaluates every layer in sequence, each one's output becoming the next layer's input, ending in a final prediction.
An artificial neuron's output is computed as...
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Taking just the maximum ignores every weight and the bias entirely -- a neuron uses all of its inputs, weighted, not just the largest one.
Multiplying inputs together is not how a standard neuron combines them -- it computes a weighted SUM, not a product.
A neuron's output is entirely determined by its inputs, weights, and bias -- nothing about it is random.
Why
a=\varphi(\sum_i w_ix_i+b) -- a weighted sum plus a bias, then a fixed nonlinear activation function.
This course does not derive backpropagation. Explain, in one or two sentences, what a reader should still be able to say about it after this section, and what they should NOT claim to understand yet.
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Why
A reader should be able to say backpropagation computes how much to adjust every weight, working backward from the output layer using the chain rule, and that it reuses each layer's own computed result rather than starting over for every weight. They should not yet claim to understand HOW the backward pass is actually computed layer by layer, or why the chain rule gives the correct adjustment -- that derivation belongs to deep-learning's own chapters.
Practice: Neurons, forward passes, and what training does (4 problems · doesn’t affect your progress)
In one sentence, what does a single neuron compute from its inputs?
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Why
It multiplies each input by its own weight, adds a bias, sums the result, and passes that sum through a fixed nonlinear activation function.
A neuron has inputs x_1=1, x_2=2, weights w_1=3, w_2=-1, bias b=0. Compute its pre-activation sum z.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
z=3(1)+(-1)(2)+0=3-2=1.
Correct answer
Also accepted:
Why
Which statement correctly separates what gradient descent does from what backpropagation does?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Gradient descent is the general "move toward lower loss" idea; backpropagation is the specific, efficient algorithm that computes each individual weight's own nudge via the chain rule.
gradient descent nudges weights toward lower loss; backpropagation computes how much to nudge each one
Name one thing this section explicitly does NOT teach about backpropagation, and where the full treatment lives.
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Why
This section does not derive why the chain rule produces the correct weight adjustment, or walk through the layer-by-layer backward computation itself -- that full mathematical treatment is deep-learning's own chapter 10.