Perplexity
How surprised a model is by text — the classic intrinsic measure of language-model quality.
Perplexity is the exponentiated average negative log-likelihood of a sequence: a perplexity of 10 means the model was, on average, as uncertain as choosing among 10 equally likely tokens. Lower is better. It tracks pretraining quality well but correlates loosely with downstream usefulness — which is why evals moved to task benchmarks and human preference.