Transformer
The neural-network architecture behind virtually every modern language model.
A Transformer stacks layers of attention and feed-forward networks with residual connections and layer normalization. Unlike RNNs it processes a whole sequence in parallel, which is what made training on internet-scale text practical. Decoder-only Transformers (GPT-style) generate text left-to-right; encoder-only ones (BERT) build representations for understanding.
Worked example: GPT is decoder-only — a causal mask lets each token attend only to earlier tokens, so it can generate the next word; BERT is encoder-only — every token sees every other, which suits classification and search but not generation. Gotcha: “processes in parallel” describes training (all positions at once). At inference, a decoder still generates one token at a time, each conditioned on all previous — that sequential decode, not attention math, is the latency bottleneck.