Residual Connection
also: skip connection
Adding a layer’s input directly to its output, so gradients have a direct path back through very deep networks.
A residual connection computes output = layer(x) + x instead of just layer(x), giving the gradient an unobstructed path straight back to earlier layers during backpropagation. This one architectural trick is a major reason networks with dozens or hundreds of layers (including every modern Transformer) can be trained at all — without it, gradients tend to vanish long before reaching the earliest layers.
Worked example: add a layer’s input directly to its output (out = x + f(x)) so gradients flow straight through the addition, letting very deep networks train without the signal vanishing — the ResNet idea that made 100+ layer models possible. Gotcha: it works because the network only learns the RESIDUAL (the change from the input), easier than relearning the whole mapping; without it transformers this deep would not train — every block is `x + attention(x)` then `x + mlp(x)` for exactly this reason.