Long short-term memory networks — the recurrent architecture that made sequence learning practical before attention took over.
A long short-term memory network is a recurrent neural network whose cell carries a separate memory vector, the cell state, alongside the usual hidden state. Three learned gates decide what to forget, what to write, and what to expose at each time step. Because the cell state is updated additively rather than by repeated matrix multiplication, gradients can flow across many more time steps than in a vanilla RNN, which is what made speech recognition, handwriting recognition, and early neural machine translation work in the 2010s.
The pages in this folder build on each other. LSTM Networks explains the cell itself: the forget, input, and output gates and the equations that tie them together. Bidirectional LSTM runs two LSTMs over the sequence in opposite directions and concatenates their states, so every position sees both left and right context. Stacked LSTM feeds the output sequence of one LSTM layer into another to learn hierarchical features.
Read the core LSTM page first; the two variants assume familiarity with the gate equations.
| Variant | Use when | Avoid when |
|---|---|---|
| Single-layer LSTM | Small datasets, baselines, streaming prediction | Long-range structure needs more capacity |
| Bidirectional LSTM | The whole sequence is known: tagging, classification, NER | Real-time forecasting, where the future is not available |
| Stacked LSTM | Larger datasets, complex sequences | Little data; deeper stacks overfit and train slowly |
| GRU (sibling) | Fewer parameters, similar accuracy on many tasks | When you need the separate cell state |
For most new sequence work, transformers have replaced LSTMs because they parallelize across time steps during training. LSTMs remain a good fit for small models on devices, low-latency streaming where state is carried forward step by step, and time-series problems with modest data. See the sibling pages on vanilla RNNs and the GRU for comparison, or the RNN section for the wider family.