A test suite
for quality.

Code has a safety net: unit tests run on every change and block the merge if something breaks. LLM apps usually have nothing equivalent — quality lives in prompts, model versions, and retrieval configs, all of which change often, and all of which can silently degrade output. A one-line prompt edit that helps one case can wreck another, and without an automated check you find out from users.

Evals in CI close that gap. You maintain a golden set of representative cases, run the current build's outputs through a scorer, and gate the pull request on the result — merge only if quality holds. It's the same discipline as testing, applied to a fuzzier target. The one thing you can't reuse from normal testing is the assertion: LLM output isn't exact-match, so the gate has to be built from scores, and scoring non-deterministic output introduces the wrinkle that makes eval gates their own craft.

A golden set is a curated collection of representative test cases — inputs paired with either reference answers or a scoring rubric — that stands in for "does this system work." It should cover the common paths, the tricky edge cases, and the failures you've fixed before (so they can't silently return). It's the LLM equivalent of a test suite, and its quality determines everything: a golden set that misses a scenario can't catch a regression in it.

Because outputs aren't exact-match, each case is scored, not asserted. The scorer might be an LLM judge against a rubric, embedding similarity to a reference, a task metric like token-level F1, or a hard check like "does the generated code pass its tests." Every case yields a number in a comparable range, and the eval run produces a vector of scores — one per case — for the current build. That vector, compared against a baseline, is what the gate reasons over. Keep the golden set in version control next to the code, and grow it every time a bug escapes: each production failure becomes a permanent test.

The naive gate is "did the average score stay above the bar?" — and it's dangerously incomplete. Averages hide things. A change can lift several cases while quietly destroying one, and the mean stays comfortably above threshold. You merge, the dashboard is green, and a real capability is gone — the one case you broke just happened to be outvoted by the ones you helped.

So a real eval gate has two conditions: the aggregate score must clear a threshold (catches broad quality drops), and no individual case may score meaningfully below its baseline (catches narrow, hidden regressions). The first guards the overall bar; the second guards every case you already got right. Only when both hold should the change merge. That two-part gate is the core idea, and the math makes it precise.

Let the current build's per-case scores be c_i and the baseline's be b_i. The aggregate gate checks the mean against a threshold θ; the regression gate flags any case that dropped more than a tolerance τ:

The threshold guards the overall bar; the tolerance band τ keeps normal run-to-run noise from flagging a case falsely.

The pull request passes only if both hold — the mean clears the bar and no case regressed:

A high mean alone isn't enough — one regressed case blocks the merge even if the average looks great. That's the property the average-only gate lacks. The runnable version below runs a suite, flags regressions, and shows the two-part gate blocking a hidden regression a mean-only gate would pass.

Turning the gate into a CI check is a short pipeline:

A few practicalities. Keep the baseline in version control and update it deliberately when an intended change legitimately shifts scores — never auto-adopt the current run as the new baseline, or regressions ratchet in silently. Budget the eval run: it costs tokens, so a fast smaller golden set can gate every PR while a larger suite runs nightly. And post a readable diff — the gate says pass/fail, but a reviewer needs to see which case moved and by how much to decide if a failure is a real regression or an intended change. This is evaluation wearing a CI hat: the same scoring, wired into the merge.

The one that bites first is a flaky gate. LLM scores fluctuate run to run, so a zero-tolerance per-case check flags noise as regression and the gate flaps — developers learn to ignore it, which is worse than no gate. Set a sensible tolerance band, reduce variance (fix seed/temperature where possible, average a few runs per case), and only then does the gate earn trust. The opposite failure is a golden set that's too small or stale: it passes everything because it doesn't cover the cases that break, giving false confidence. Grow it from real failures.

Two more. Trusting the judge blindly — if you score with an LLM judge, the judge has its own biases and errors, so validate it against human labels before you let it gate merges (an unvalidated judge is a random number generator with opinions). And gaming the metric: teams optimize prompts to score well on the golden set and drift from real quality, so refresh the set and watch production, not just the gate. Done right, evals in CI make LLM quality a first-class, automated concern — you stop hoping a change didn't break anything and start knowing, on every merge.