Convolution and kernels, recalled

Here's how to get one output pixel using correlation. First, lay the kernel down. Center it on the matching input pixel. Next, multiply each kernel number by the image pixel sitting right under it. Then add up all those products. That sum is your answer.

Convolution does the same laying-down-and-multiplying. The only difference: you flip the kernel first, end-to-end and top-to-bottom. Suppose the kernel looks the same after that flip — a plain averaging grid, say. Then flipping changes nothing. But for a lopsided kernel, it does change the result.

Here's a point that causes real confusion later, so it's worth stating plainly. A deep learning library often calls something a "convolutional layer." But strictly speaking, it's running correlation as this section defines it. The kernel never actually gets flipped. In practice, nobody who builds these networks worries about this. The network learns whatever numbers work, no matter which operation trained them. Still, it's good to know the label and the math don't quite agree. It's the same mismatch as a library's .shape order not quite matching how a person reads a photo's size out loud.

Now, the three named kernels. Identity does nothing at all — it's just a sanity-check baseline. The box kernel averages a neighborhood's pixels, all weighted equally. It's cheap to compute, but a little blocky. The Gaussian kernel also averages the neighborhood. But it lets nearby pixels count more than far-away ones. It tapers off smoothly, instead of cutting off sharply at the window's edge. That's why a Gaussian blur looks soft and natural. A box blur, by contrast, can leave faint rectangular traces.

Take an image I and a small weight grid H (a kernel) of size (2a+1)\times(2b+1). The 2D discrete correlation computes one output value per position: center H on that position, multiply each kernel entry by the image value underneath it, and sum the results. (I \star H)(x,y) = \sum_{i=-a}^{a}\sum_{j=-b}^{b} I(x+i,\, y+j)\, H(i,j). Convolution, (I*H)(x,y), computes the identical kind of sum, but first reads the kernel flipped 180°: replace H(i,j) with H(-i,-j) in the formula above. For a kernel that is symmetric under that flip — the box and Gaussian kernels below both are — the two operations produce identical output. For one that isn't, like Chapter 3's Sobel/Prewitt kernels, they don't, and the distinction matters there.

This is also the point where mathematics and software vocabulary part ways. It's worth stating plainly, once: most deep-learning frameworks label their convolutional layers' operation "convolution," but they actually implement correlation. The kernel is not flipped before sliding it across the input. A learned filter's own values absorb whatever difference that would have made during training, so the distinction has no practical consequence for a trained network. Still, it is a real naming mismatch between the framework's vocabulary and the mathematical operation this section actually defines — not a property of convolution itself.

Three named kernels recur throughout this course. The identity kernel has a single 1 at its center and 0 elsewhere, so it leaves the image unchanged. The box kernel has every entry equal to 1/K^2 for a K\times K grid, normalized so its entries sum to 1. It replaces each pixel with the plain average of its neighborhood. The Gaussian kernel H(i,j) = \frac{1}{2\pi\sigma^2} e^{-(i^2+j^2)/(2\sigma^2)} weights the neighborhood's center most heavily, tapering off smoothly along a bell curve controlled by \sigma (sigma). That produces a smoother, more natural blur than a box kernel's sharp-edged average. That's why it's the default smoothing step before almost every technique in Chapters 3–5.