Long-horizon
agents.

If each step of a task succeeds with probability p, and the task needs k steps to all go right in a row, the chance of finishing clean is p^k. That exponent is brutal. It's the same pass^k curve that agent evaluation measures — and it means length, not difficulty, is the enemy.

A 95%-reliable step sounds great until you chain a hundred of them and land under 1%. The math of long horizons is unforgiving to any single-shot run.

The full treatment of measuring this — pass@k versus pass^k, and how reliability scores over many attempts — lives in the agent-evals handbook. This handbook picks up where measurement ends: given that p^k collapses, how do you build an agent that survives it?

The first move is to stop treating the task as one indivisible chain. Checkpointing saves the agent's state at intervals — completed subtasks, intermediate artifacts, verified progress — so a failure at step 87 doesn't throw away steps 1 through 86. Without checkpoints, one stumble anywhere means starting over; with them, a stumble costs only the work since the last save.

A good checkpoint is resumable state: enough that the agent can pick up exactly where it left off, not a vague summary that forces guesswork. The art is choosing what to persist — the durable artifacts and decisions, not the entire scratchpad — so resume is faithful and cheap.

Checkpoints unlock the real win: retry the failed segment, not the whole task. Split the k steps into segments of length c. A segment succeeds with probability p^c, and if you can retry a failed segment up to r times, its effective success rises to 1−(1−p^c)^r. The whole task is then those segments in series:

Same model, same per-step p. At p=0.95, k=100: the naive run finishes 0.6% of the time; checkpoint every 10 steps with 3 retries per segment and it finishes ~51%. The exponent that was killing you now works segment-by-segment, where retries can beat it.

Two dials fall out of the formula. Shorter segments (frequent checkpoints) make each segment more likely to pass on the first try; more retries per segment raise each segment's effective reliability. Both cost overhead, so you tune them against the horizon. The runnable model below lets you turn each dial.

The field now tracks an agent's time horizon: the length of task it can complete at some success rate (say, the duration it finishes half the time). It's a headline number because it's been climbing fast — roughly doubling on a regular cadence — and it captures what "more capable agent" actually means in practice: it can carry a longer task before reliability decay wins.

The lesson of this handbook is that horizon isn't only a property of the model. Raw per-step reliability sets a baseline horizon; checkpointing, retries, and verification extend it. An agent that saves progress and retries locally has a longer usable horizon than its bare p would predict. So when the METR-style curve doubles, part of that is better models — and part is better harnesses doing exactly what's above.

The agents that will run for hours — refactoring a codebase, conducting deep research, driving a multi-stage workflow — don't win by having a magically reliable model. They win on reliability engineering: checkpoint often, make state resumable, verify between steps, retry the smallest failing unit. It's the same discipline distributed systems learned decades ago (idempotency, retries, checkpoints), now applied to an agent's step chain.

For a builder the takeaway is liberating: you don't have to wait for a perfect model to ship a long-running agent. You have to engineer around imperfect steps. Measure the decay with agent-evals, guard each step with the patterns in agent patterns, and structure the whole loop with a harness that checkpoints and resumes. The horizon you can reach is mostly up to you.