Residual Connection
also: skip connection
Adding a layer’s input directly to its output, so gradients have a direct path back through very deep networks.
A residual connection computes output = layer(x) + x instead of just layer(x), giving the gradient an unobstructed path straight back to earlier layers during backpropagation. This one architectural trick is a major reason networks with dozens or hundreds of layers (including every modern Transformer) can be trained at all — without it, gradients tend to vanish long before reaching the earliest layers.