Softmax
Turns a vector of raw scores (logits) into a probability distribution that sums to 1.
Softmax exponentiates each score and normalizes, mapping arbitrary logits to probabilities. It sits at the output of a classifier or language model (over the next token) and inside attention (over the keys). The numerically stable form subtracts the max before exponentiating.