Convolutional Neural Networks (CNN)

A convolutional neural network replaces the dense layers of an MLP with small learned filters that slide across the input. Two structural assumptions — locality (nearby pixels are related) and translation equivariance (a feature means the same thing wherever it appears) — cut the parameter count by orders of magnitude and made deep learning work on images. CNNs remain the workhorse for vision on edge devices, medical imaging, and real-time detection, and they are the backbone of detectors such as YOLO.

This page covers the convolution operation, pooling, the standard architecture pattern, receptive fields, a PyTorch example, and how CNNs compare to Vision Transformers. It sits under Deep Learning (DL).

1. The Convolution Operation

A 2-D convolution layer (technically cross-correlation in every framework) computes each output value as the dot product of a small kernel with the input patch underneath it, plus a bias. The same kernel weights are reused at every spatial position — weight sharing — so a 3x3 filter has 9 weights per input channel regardless of image size. For an input of height H, kernel k, padding p and stride s, the output size is floor((H + 2p − k)/s) + 1.

Compare this to a dense layer mapping a 224x224x3 image to just 64 units: about 9.6 million weights. A 3x3 convolution producing 64 channels needs 3·3·3·64 + 64 = 1,792.

2. Channels, Stride, Padding and Pooling

Channels. Real inputs have Cin channels (3 for RGB) and a layer learns Cout filters, each of shape Cin×k×k. Each filter produces one output channel, so a conv layer has Cout·Cin·k2 + Cout parameters. A 1x1 convolution mixes channels without looking at neighbors and is used to cheaply change channel count.

Padding ("same" padding of p = (k−1)/2) preserves spatial size so borders are not eroded. Stride greater than 1 downsamples. Pooling (max or average) downsamples without parameters; global average pooling at the end collapses each channel to a single number and replaces large dense layers.

3. The Standard CNN Pipeline

Almost every CNN follows this template: a stem that quickly reduces resolution, a stack of stages where each stage halves spatial size and roughly doubles channels, then global average pooling and a linear classifier. The historical line runs LeNet (digits), AlexNet (2012, the ImageNet breakthrough, ReLU + GPUs), VGG (uniform 3x3 stacks), Inception (parallel branches), ResNet (residual connections) and EfficientNet / ConvNeXt (scaling and modernized blocks).

The convolutional body is a general-purpose feature extractor. For detection, a head like the one in YOLO reads multi-scale feature maps; for segmentation, a U-Net-style decoder upsamples back to pixel resolution.

4. Receptive Field and Feature Hierarchy

The receptive field of a unit is the region of the input that can influence it. Two stacked 3x3 convolutions see 5x5; three see 7x7, with fewer parameters and more nonlinearity than a single 7x7 kernel — the insight behind VGG. Downsampling multiplies the growth, so deep units cover most of the image. Visualizations of trained networks show a consistent hierarchy: early filters respond to edges and color blobs, middle layers to textures and parts, late layers to object-level patterns. That hierarchy is why ImageNet-pretrained backbones transfer so well to new tasks.

5. PyTorch Example

A compact CNN for CIFAR-10 (32x32 RGB). In practice, starting from a pretrained torchvision backbone and fine-tuning is usually better than training from scratch.

import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as T

tf_train = T.Compose([T.RandomCrop(32, padding=4), T.RandomHorizontalFlip(),
                      T.ToTensor(), T.Normalize((0.49, 0.48, 0.45), (0.25, 0.24, 0.26))])
tf_test = T.Compose([T.ToTensor(), T.Normalize((0.49, 0.48, 0.45), (0.25, 0.24, 0.26))])
train = torchvision.datasets.CIFAR10("data", train=True, download=True, transform=tf_train)
test = torchvision.datasets.CIFAR10("data", train=False, download=True, transform=tf_test)
dl_tr = torch.utils.data.DataLoader(train, batch_size=128, shuffle=True, num_workers=2)
dl_te = torch.utils.data.DataLoader(test, batch_size=256)

def block(c_in, c_out):
    return nn.Sequential(
        nn.Conv2d(c_in, c_out, 3, padding=1, bias=False), nn.BatchNorm2d(c_out), nn.ReLU(),
        nn.Conv2d(c_out, c_out, 3, padding=1, bias=False), nn.BatchNorm2d(c_out), nn.ReLU(),
        nn.MaxPool2d(2))

model = nn.Sequential(
    block(3, 64), block(64, 128), block(128, 256),   # 32 -> 16 -> 8 -> 4
    nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(256, 10))

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
opt = torch.optim.SGD(model.parameters(), lr=0.05, momentum=0.9, weight_decay=5e-4)
sched = torch.optim.lr_scheduler.OneCycleLR(opt, max_lr=0.1, epochs=20, steps_per_epoch=len(dl_tr))
loss_fn = nn.CrossEntropyLoss()

for epoch in range(20):
    model.train()
    for x, y in dl_tr:
        x, y = x.to(device), y.to(device)
        opt.zero_grad()
        loss_fn(model(x), y).backward()
        opt.step(); sched.step()

model.eval(); correct = 0
with torch.no_grad():
    for x, y in dl_te:
        correct += (model(x.to(device)).argmax(1).cpu() == y).sum().item()
print("test acc", correct / len(test))

6. CNN vs MLP vs Vision Transformer

CNNs are the safer default when labeled data is limited, latency matters, or the target is an embedded device. ViTs and hybrids win at large scale and in multimodal systems. Convolutions are also used on 1-D signals (audio, time series, text) and 3-D volumes (CT/MRI, video).

7. Common Pitfalls

• Training from scratch on small data. Fine-tune a pretrained backbone instead; freeze early layers if data is very small.
• Mismatched normalization. Pretrained weights expect the mean/std used in pretraining.
• Augmentation that breaks labels. Horizontal flips are wrong for text or digits like 6/9; rotations can be wrong for medical orientation.
• BatchNorm with tiny batches. Statistics become noisy; use GroupNorm or freeze BN stats.
• Shape bookkeeping. PyTorch is channels-first (N, C, H, W); Keras/TensorFlow default to channels-last.
• Shortcut learning. CNNs latch onto texture, watermarks or scanner artifacts; check with saliency maps and out-of-distribution test sets.

8. Related Topics