Curriculum

Deep Learning · Vision Architecture

Convolutional Neural Networks (CNN)

CNNs apply learnable filters across spatial positions, exploiting translation invariance. Weight sharing dramatically reduces parameters and encodes inductive bias that nearby pixels are related.

The idea

A convolutional layer applies a small filter (kernel) by sliding it across the input, computing a dot product at each position. The result is a feature map. Multiple filters produce multiple channels. Key insight: weight sharing—the same filter detects an edge everywhere in the image, not just where it was trained.

Padding controls output spatial size. Stride controls how far the filter moves each step. Pooling (max or average) subsamples the feature map, reducing spatial dimensions and adding translation invariance. After several conv+pool blocks, features are spatially small but channel-rich.

Classic architectures: LeNet (1998), AlexNet (2012, winner of ImageNet), VGG (uniform 3×3 filters), GoogLeNet (Inception blocks), ResNet (residual skip connections). Modern: ConvNeXt and EfficientNet compete with Vision Transformers on many benchmarks.

(f ★ g)[n] = Σₖ f[k]·g[n−k]

1D convolution as cross-correlation. In 2D CNNs, this extends to a 2D sliding dot product.

What to hold onto

  • Receptive FieldThe input region one output unit "sees". In a stack of 3×3 convolutions, the receptive field grows by 2 pixels per layer. Dilated convolutions expand it without extra parameters.
  • Depthwise Separable ConvSplit the standard convolution into depthwise (each channel independently) and pointwise (1×1 across channels). 8–9× fewer operations. Used in MobileNet.
  • Feature MapsA conv layer with 64 filters produces 64 feature maps, each detecting a different pattern. Visualising them reveals what the network has learned.

Live: Convolutional Kernel & Feature Extractor

Select 3x3 convolution kernels (Sobel, Edge Detect, Sharpen) and watch the sliding receptive field compute the feature map.

Receptive Field 3x3 Conv

Quick answers

What is a Convolutional Neural Network?

A CNN is a neural network architecture designed for data with spatial structure (images, audio). It applies small learnable filters across the input using convolution, sharing weights across positions. This encodes translation invariance, dramatically reduces parameters versus fully-connected networks, and builds hierarchical feature detectors.

What is the purpose of pooling in CNNs?

Pooling layers (max or average) reduce spatial dimensions by summarising a region into a single value. This makes feature detectors tolerant of small translations (max pooling selects the strongest activation in a region), reduces computation in subsequent layers, and widens the effective receptive field.