Machine learning · extension lesson
Convolutional neural networks: from pixels to features
Can the same detector find a line on either side of a picture? Start by moving one line and watching where a response appears.
No coding or prior maths needed for the first experiment. The calculations are available afterwards. This standalone exploration and its checks do not save to your core lesson notebook.
Move the picture, keep the detector
live Computed here · invented six-by-six image · hand-set detector, no training
Predict first: if the line moves right, will the strongest response move too?
Picture
Where the detector responds
live The bright response is now in output column 2. The detector is unchanged; only the picture moved.
The same small detector checks each nearby group of pixels. Its response is strongest where that group matches a bright vertical line. This reuse is the central idea behind a convolution. The response picture is smaller because this example skips positions where the detector would run off the edge.
This is one fixed line detector, not an image recogniser. More complicated pictures can produce misleading responses; this experiment does not show that a trained network will handle every shift.
Explain it back: a shifted window
A photo shows a window on the left. In a second photo, the same window is on the right. Explain why a detector for its vertical frame can still help, and why that alone does not prove the picture contains a window.
Try saying it in your own words before opening the explanation. No calculation is needed.
Compare with one explanation
The same detector checks small patches throughout both photos, so it can respond to the frame in a new position. But a vertical edge could also belong to a door or a tree. A local response is useful evidence, not a complete identification.
Explore the calculations: pixels, filters and classification
A pixel is one brightness value in the picture. A filter is a small set of numbers used by the detector. The lab below exposes those numbers and lets you edit them indirectly by changing the picture.
One filter, many positions
A grayscale image is a grid of brightness values. A kernel is a small grid of weights. Here, nine weights cover a 3 × 3 patch. Multiply matching entries, add the products and a bias, then move the kernel. The results form a feature map.
Stride is how many pixels the kernel moves per step. Padding adds a border, here made of zeros. Try stride 2, then add padding: the highlighted patch and output dimensions change together.
As in common CNN libraries, the kernel is not flipped. The exact operation is cross-correlation, conventionally called convolution in this context. Source: PyTorch Conv2d.
Synthetic pixels · fixed hand-set filters · bias 0
Input image · 6 × 6
Click a pixel, or Tab to it and press Space. Brightness cycles 0 → 0.5 → 1. Blue borders mark the current patch; “pad” cells are added zeros.
| -1 | 2 | -1 |
| -1 | 2 | -1 |
| -1 | 2 | -1 |
Patch 1 / 16 · output row 1, column 1 live
Multiply each pixel by its matching kernel weight, then add all nine products. This is one output cell.
Nine products live
- 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1 - 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1 - 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1
Sum + bias 0 = -3 live
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
Output side = floor((input + 2 × padding − kernel) ÷ stride) + 1
floor((6 + 2 × 0 − 3) ÷ 1) + 1 = 4 live
Keep positive responses, then reduce the grid
ReLU replaces negative values with zero. It adds a nonlinear step: a negative line-contrast response does not cancel a positive one later. Max pooling then keeps the largest value in each 2 × 2 window, moving two cells at a time.
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 6 | 0 |
| 6 | 0 |
Pooling uses no padding. An incomplete final row or column is dropped. These smaller maps lose exact location information; pooling does not guarantee the same answer for every image shift.
From two feature maps to one label
This second experiment uses your same edited image and both filters at once. Its convolution always uses stride 1 and no padding, independently of the controls above. Each branch applies ReLU, 2 × 2 max pooling, then averages its four pooled values. The larger average supplies the label.
liveHand-set feature/classification pipeline · no training · scores are not probabilities
- 6 × 6 image
- → two 4 × 4 maps
- → ReLU
- → two 2 × 2 pools
- → means → label
| 6 | 0 |
| 6 | 0 |
Mean = (6 + 0 + 6 + 0) ÷ 4 = 3 live
| 0 | 0 |
| 0 | 0 |
Mean = (0 + 0 + 0 + 0) ÷ 4 = 0 live
Output: Vertical live
A tie means equal scores, including a blank image. Any unequal scores force a vertical or horizontal label, even for a scribble that fits neither class. This rule has no “other” category.
Try the horizontal preset, then erase or add pixels. Which branch changes? Try a blank image: no response is evidence for neither orientation, rather than a confident classification.
What a trained CNN adds
The lab’s two filters and score rule were written by hand. In a trained CNN, convolution weights and classification weights are parameters. A training image passes through the network, its scores are compared with the labelled class using a loss, and backpropagation supplies gradients. An optimiser updates parameters to reduce that loss over training examples.
A filter shares its weights across spatial positions. With one grayscale input channel, a 3 × 3 filter has nine weights and one bias, however many positions it visits. Multiple filters produce multiple output channels; colour inputs add red, green and blue channels. Later convolution layers combine local patches across earlier feature channels.
More layers can combine local responses over larger regions. They do not guarantee useful features: the data, labels, architecture and training procedure matter. Test performance on held-out images, including shifted or noisy examples, instead of judging from a few training images.
Source and next reading: PyTorch: training and testing an image classifier.