RNN

An RNN (Recurrent Neural Network) processes a sequence one element at a time, maintaining a hidden state that carries information forward — so each step’s output depends on everything before it. It was the standard for text, speech, and time series before transformers.

Worked example: reading a sentence word by word, the RNN updates its hidden state at each word, so by the end it has (in principle) a summary of the whole sentence to predict the next word. Gotcha: RNNs struggle with long-range dependencies because the signal must pass through every step — gradients vanish or explode over long sequences (which LSTMs partly fix) — and their inherent sequentiality prevents parallel training, the exact limitations attention and transformers removed.