Sigmoid
The classic S-shaped activation that squashes any number into (0,1) — used for probabilities and gates.
The sigmoid function maps any real number to a value between 0 and 1 with a smooth S-shape (large negatives → 0, large positives → 1, 0 → 0.5). It turns a raw score into something interpretable as a probability, and historically was a common neural-network activation.
Worked example: logistic regression passes its weighted sum through a sigmoid to output a probability; and LSTM gates use sigmoids to produce values near 0 or 1 that act as soft on/off switches for remembering or forgetting. Gotcha: sigmoid causes vanishing gradients — for large positive or negative inputs its slope is nearly 0, so gradients barely flow and deep networks train poorly — which is why ReLU/GELU replaced it in hidden layers; it survives mainly at the output (binary probability) and in gating.