Graph neural networks learn from data whose structure is a graph — nodes connected by edges — rather than a grid (images) or a sequence (text). Molecules, social and citation networks, road maps, knowledge graphs, transaction networks and recommender interaction graphs all fit this shape. A GNN computes a vector representation for every node by repeatedly exchanging and aggregating information with its neighbors, so each embedding reflects both the node's own features and its local graph structure. The two most widely used layer types, GCN and GAT, each have their own deep-dive page.
This page sits under Deep Learning (DL).
Symmetric-normalized neighbor averaging, Â H W, derived from spectral graph convolution.
Learned, per-edge attention weights over neighbors with multi-head aggregation.
A Transformer is attention over a fully connected graph of tokens — the link between GNNs and Transformers.
A graph has a node set V and edge set E. Each node carries a feature vector (atom type, user profile, paper bag-of-words), stacked into X ∈ RN×d; edges may carry features too (bond type, transaction amount). Structure is the adjacency matrix A, which for real graphs is extremely sparse, so libraries store an edge list (COO format, edge_index of shape 2×|E|). Two properties shape everything else: graphs have no natural node ordering, and node degree varies widely. A GNN must therefore be permutation equivariant — relabeling nodes should relabel outputs identically — and must handle any number of neighbors.
Almost every GNN fits the message passing framework (Gilmer et al., 2017). In each layer, every node collects messages from its neighbors, combines them with a permutation-invariant aggregator, and updates its own embedding with a small neural network. Weights are shared across nodes, exactly as convolution kernels are shared across pixels, so the model works on graphs of any size. After k layers a node's embedding depends on its k-hop neighborhood — the graph analogue of a receptive field. Most practical models use 2–4 layers.
Choice of aggregator matters: sum preserves degree information (important for counting substructures in molecules), mean normalizes it away (robust on graphs with highly varied degrees), max picks out salient neighbors, and attention learns per-neighbor weights.
[hu; hv]): link prediction for recommendation, drug–target interaction, knowledge-graph completion. Train with negative sampling.| Layer | Aggregation | Notes |
|---|---|---|
| GCN | Degree-normalized sum with self-loop | Simple, fast, strong baseline; fixed neighbor weights |
| GraphSAGE | Mean / max / LSTM over sampled neighbors, concatenated with self | Neighbor sampling scales to very large graphs; inductive |
| GAT | Learned attention weights per edge | Handles noisy or heterogeneous neighborhoods |
| GIN | Sum followed by MLP | As expressive as the 1-WL test; good for graph classification |
| MPNN / edge-conditioned | Messages depend on edge features | Chemistry, physics simulations |
| R-GCN / HGT | Separate weights per relation or node type | Knowledge graphs, heterogeneous graphs |
| Graph Transformers | Attention over all nodes plus structural encodings | Long-range interactions; quadratic cost |
Node classification on the Cora citation graph (2,708 papers, 7 topics) with a two-layer GCN. Swapping GCNConv for SAGEConv or GATConv is a one-line change.
import torch
import torch.nn.functional as F
from torch_geometric.datasets import Planetoid # pip install torch_geometric
from torch_geometric.nn import GCNConv
data = Planetoid(root="data", name="Cora")[0]
print(data) # Data(x=[2708, 1433], edge_index=[2, 10556], y=[2708], train_mask=..., ...)
class GNN(torch.nn.Module):
def __init__(self, d_in, d_hid, n_cls):
super().__init__()
self.conv1 = GCNConv(d_in, d_hid)
self.conv2 = GCNConv(d_hid, n_cls)
def forward(self, x, edge_index):
x = F.dropout(x, p=0.5, training=self.training)
x = F.relu(self.conv1(x, edge_index))
x = F.dropout(x, p=0.5, training=self.training)
return self.conv2(x, edge_index)
model = GNN(data.num_features, 16, int(data.y.max()) + 1)
opt = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)
for epoch in range(200):
model.train(); opt.zero_grad()
out = model(data.x, data.edge_index) # full-graph forward
F.cross_entropy(out[data.train_mask], data.y[data.train_mask]).backward()
opt.step()
model.eval()
pred = model(data.x, data.edge_index).argmax(1)
acc = (pred[data.test_mask] == data.y[data.test_mask]).float().mean()
print(f"test accuracy {acc:.3f}")For graph-level tasks, batch many small graphs with torch_geometric.loader.DataLoader and apply global_mean_pool(x, batch) before the classifier. DGL is the main alternative library.
edge_index; use to_undirected where appropriate.