Eval Gate
A CI check that blocks a change when evaluation results fall below agreed thresholds — the point at which an eval stops being a document.
An eval gate runs the golden set on every change, including prompt changes, and fails the build on four conditions: overall score below the agreed baseline, high-cost error rate above budget, any per-case regression, and cost or latency breaches. The per-case regression check is the one that earns its keep.
Worked example: a prompt tweak raises overall accuracy from 74% to 77% while re-routing three legal-review invoices to auto-pay; only the regression assertion catches it, because the average improved. Gotcha: prompts are code — a prompt edit that skips the gate is an unreviewed production deploy, and it is the most common source of “it got worse and nobody knows when”.