Message
passing.

Convolutional networks exploit the grid of an image; recurrent nets and Transformers exploit the order of a sequence. But an enormous amount of real data has neither. A molecule is atoms joined by bonds; a citation network is papers linked by references; a social graph is people connected by follows. These are graphs — sets of nodes joined by edges — and they have no canonical ordering and no fixed neighborhood size. There's nothing for a convolution to slide over.

A graph neural network learns on this structure directly. The Graph Convolutional Network (Kipf & Welling, 2017) is the canonical, minimal form. Its promise: give each node a feature vector and a list of neighbors, and it produces a new feature vector per node that blends in information from the neighborhood — useful for classifying nodes, predicting missing edges, or scoring an entire graph. The whole thing reduces to one operation applied over and over: a node updates itself from its neighbors.

The engine of every GNN is message passing. In one layer, each node gathers messages from its neighbors, aggregates them into a single summary, and combines that with its own current feature to produce a new one. For a GCN the aggregation is a normalized weighted sum; other GNNs use a max, a mean, or an attention-weighted sum, but the shape is always the same: gather, aggregate, update.

Because every node runs the same shared function, the network handles graphs of any size and any connectivity — a property images and sequences don't demand. And depth buys reach:

Stacking layers grows the receptive field one hop at a time. But there's a tension: go too deep and repeated averaging causes over-smoothing, where every node's feature converges to nearly the same value and the network loses its ability to tell nodes apart. Most GCNs are shallow — two or three layers — for exactly this reason.

Two details separate a naive "sum my neighbors" from a GCN layer that actually trains, and both live inside the normalized adjacency Â:

The self-loops matter because without A + I a node's new feature would be built only from its neighbors, discarding what it already knew. The symmetric normalization matters because raw neighbor-sums are dominated by high-degree hubs — a node with 1,000 neighbors would accumulate a huge vector and drown out low-degree nodes, and repeated layers would make magnitudes explode or vanish. Dividing each edge weight by √(dᵢ·dⱼ) turns every aggregation into a stable, degree-balanced average that behaves well no matter the graph's structure. That single normalized matrix is the whole trick.

One layer of a Graph Convolutional Network is a single line:

H⁽ˡ⁾ is the node-feature matrix at layer l;  mixes each node with its neighbors (normalized); W⁽ˡ⁾ is the learned weight; σ is a nonlinearity (ReLU). ·H is the message passing; ·W and σ are an ordinary neural layer on top.

Read right to left, it does exactly three things:  H replaces every node's feature with the degree-normalized average of its neighborhood (the message passing); · W applies a learned linear transformation shared across all nodes; and σ adds a nonlinearity. Stack two of these with a softmax on top and you have a node classifier that, on citation benchmarks, beat far more complicated methods. The runnable version below builds  from a raw adjacency and propagates features one hop — the entire heart of the model in a few lines of plain Python.

The GCN made learning on graphs simple, fast, and strong:

It has real limits — shallow depth to avoid over-smoothing, a fixed (non-learned) aggregation weight, and the full-graph version doesn't scale to billions of edges without sampling tricks. Later work (GraphSAGE's neighbor sampling, GAT's learned attention weights) addressed each. But the GCN's contribution was to distill graph learning to its essence: one normalized message-passing step, repeated.

Because so much important data is a graph, GNNs became a standard tool across science and industry. They predict molecular properties for drug discovery, estimate travel times in mapping apps by treating road networks as graphs, catch fraud in transaction networks, power recommendation systems over user–item graphs, and simulate physical systems of interacting particles. Anywhere relationships matter as much as the entities, message passing is the natural model.

There's also a deep connection to the rest of modern AI. Self-attention — the core of the Transformer — is essentially message passing on a fully connected graph, where the attention weights play the role of learned edge strengths. Graph attention networks made that link explicit, and the two families remain mathematically close cousins. Meanwhile the "learn node representations" goal connects back to embedding methods like word2vec (node2vec and DeepWalk borrow its skip-gram idea for graphs). The GCN's clean, degree-normalized layer is still the clearest doorway into all of it — one matrix multiply that taught neural networks to reason over structure.