Recipe
Forward pass of one neuron
Use when: you are given a neuron's inputs, its weights, its bias, and the name of its activation function, and need its output
Goal: the neuron's output a, with the sum z written down on the way
Taught in A single neuron, start to finish
Quick recipe
-
1
Pair every input with the weight that belongs to it and multiply. Keep the products separate for now; do not add anything yet.
Example
For x = (2, 3) and w = (0.5, 0.2): 0.5 \times 2 = 1 and 0.2 \times 3 = 0.6.
Why: each weight scales exactly one input, so the pairing must be right before any adding happens.
-
2
Sum all the products from the previous step into one number.
Example
1 + 0.6 = 1.6.
Why: the neuron cares about the combined pull of every input, not each one alone.
-
3
Add b to the sum. Write the result as z.
Example
z = 1.6 + 1 = 2.6.
Why: the bias shifts the whole total regardless of the inputs, and it is the step most often skipped because no input reminds you of it.
-
4
Feed z into whichever rule the neuron uses.
Example
Sigmoid: \sigma(2.6) = 1/(1 + e^{-2.6}) \approx 1/1.074 \approx 0.931. Step: 2.6 \ge 0, so 1.
- step function — output 1 if z \ge 0, else 0.
- sigmoid — output \sigma(z) = 1/(1 + e^{-z}).
Why: without this step the neuron is just a weighted sum, and the last two sections showed a bend is what makes it useful.
-
5
The activation's result is the neuron's output. State it, with z beside it.
Example
a \approx 0.931 (sigmoid), with z = 2.6.
Why: z and a are different numbers with different jobs; reporting both makes the next chapter's layer computations checkable.
Common mistakes
- Forgetting to add the bias, and reporting z = 1.6 instead of 2.6.
- Skipping the activation, and reporting z = 2.6 as the output instead of a = 0.931.
- Multiplying the bias by an input, as if it were a third weight.
- Treating a sigmoid of a negative z as a negative output; it is a number between 0 and 0.5.