The simplest neural networks: the single-layer perceptron and the multi-layer perceptron that every deeper architecture builds on.
Multi-layer perceptrons: fully connected feedforward networks, forward pass, backpropagation, activations, initialization, and a PyTorch example.
The perceptron, the original single-neuron linear classifier: model, learning rule, convergence theorem, Python implementation, the XOR limitation, and its rela
A feedforward network passes information in one direction: from the inputs, through zero or more hidden layers, to the outputs, with no loops and no memory of previous inputs. The perceptron is the single-unit case. It computes a weighted sum of its inputs plus a bias and outputs 1 if the sum is positive. Its learning rule nudges the weights after every mistake and is guaranteed to converge when the classes are linearly separable, but a single perceptron can only draw one straight line (a hyperplane), so it cannot represent functions such as XOR.
The multi-layer perceptron (MLP) stacks layers of such units and replaces the step function with differentiable activations such as ReLU, tanh or sigmoid. Hidden layers learn intermediate features, which removes the linear-separability limit; with one sufficiently wide hidden layer an MLP can approximate any continuous function on a bounded domain. Training uses backpropagation to compute gradients of a loss and gradient descent to update the weights.
| Page | Model | Learning | Read it for |
|---|---|---|---|
| The Perceptron | One linear unit, step output | Perceptron update rule | Historical foundation, linear separability, why XOR fails |
| Multi-Layer Perceptron | Stacked dense layers with nonlinear activations | Backpropagation and gradient descent | The building block used inside almost every modern network |
Read the perceptron page first if you want the intuition of weights, bias and decision boundaries; go straight to the MLP page for forward passes, backpropagation, initialisation and a PyTorch example. MLPs remain a strong baseline for tabular data and appear as the feed-forward sublayer in every Transformer block and as the classification head on CNNs. When inputs have structure, specialised architectures usually beat a plain MLP: convolutional networks for images, recurrent networks and Transformers for sequences, and graph neural networks for graphs.