Tanh
also: hyperbolic tangent
An S-shaped activation like sigmoid but centered at zero, squashing inputs to (−1, 1).
The tanh (hyperbolic tangent) function is an S-shaped activation that squashes any input to the range (−1, 1). It is like sigmoid but zero-centered, which historically made it preferable for hidden layers because zero-centered activations keep gradients better-behaved than sigmoid’s all-positive outputs.
Worked example: an RNN or LSTM uses tanh to squash its cell state and candidate values into (−1, 1), keeping activations bounded as information flows across many time steps. Gotcha: tanh shares sigmoid’s core flaw — it saturates at the extremes, so gradients vanish for large-magnitude inputs and deep stacks train slowly — which is why modern feed-forward layers use ReLU/GELU; tanh persists mainly inside recurrent cells and some output heads.