Override Rate
The share of a system’s outputs that a human changes before acting on them — simultaneously a quality signal, a trust signal, and a source of labels.
Override rate is logged whenever a user edits or rejects what the system produced. It is the most useful single number in a deployed AI product because it measures the thing that matters — whether the humans doing the work trust the output enough to use it unchanged.
Worked example: an override rate falling from 34% to 11% over eight weeks is the most persuasive chart available in a quarterly business review, because it shows users choosing to trust the system rather than being told to. Gotcha: capture the corrected value, not just the fact of an override — six weeks of real usage then produces a better golden set than any labelling exercise.