Long-horizon
agents.

If each step of a task succeeds with probability p, and the task needs k steps to all go right in a row, the chance of finishing clean is p^k. That exponent is brutal. It's the same pass^k curve that agent evaluation measures — and it means length, not difficulty, is the enemy.

A 95%-reliable step sounds great until you chain a hundred of them and land under 1%. The math of long horizons is unforgiving to any single-shot run.

The full treatment of measuring this — pass@k versus pass^k, and how reliability scores over many attempts — lives in the agent-evals handbook. This handbook picks up where measurement ends: given that p^k collapses, how do you build an agent that survives it?

The first move is to stop treating the task as one indivisible chain. Checkpointing saves the agent's state at intervals — completed subtasks, intermediate artifacts, verified progress — so a failure at step 87 doesn't throw away steps 1 through 86. Without checkpoints, one stumble anywhere means starting over; with them, a stumble costs only the work since the last save.

A good checkpoint is resumable state: enough that the agent can pick up exactly where it left off, not a vague summary that forces guesswork. The art is choosing what to persist — the durable artifacts and decisions, not the entire scratchpad — so resume is faithful and cheap.

Checkpoints unlock the real win: retry the failed segment, not the whole task. Split the k steps into segments of length c. A segment succeeds with probability p^c, and if you can retry a failed segment up to r times, its effective success rises to 1−(1−p^c)^r. The whole task is then those segments in series:

Same model, same per-step p. At p=0.95, k=100: the naive run finishes 0.6% of the time; checkpoint every 10 steps with 3 retries per segment and it finishes ~51%. The exponent that was killing you now works segment-by-segment, where retries can beat it.

Two dials fall out of the formula. Shorter segments (frequent checkpoints) make each segment more likely to pass on the first try; more retries per segment raise each segment's effective reliability. Both cost overhead, so you tune them against the horizon. The runnable model below lets you turn each dial.

The retry math above makes two quiet assumptions: every failure is noticed, and every retry is an independent draw. Long-running agents break both. A step can fail while reporting success, with a test it skipped, a file it thinks it wrote, or "done" on a feature that doesn't work, and every later step builds on the mistake. A retry only fires on a failure somebody detected. And when the cause is the model's own misunderstanding, the retry tends to make the same mistake, because nothing about the second attempt is different.

Illustrative, at p = 0.95 and k = 100. If 10% of failures are silent, the ceiling is about 61%; at 20% it's 37%; at 50% it's 8%. No number of retries moves it. Only detection does.

So the cheapest reliability work on a long-horizon agent is usually verification, not retries: a check after each step that can tell "claimed done" from "done". Anthropic's write-up on harnesses for long-running agents names exactly these failure modes in its coding agents: trying to one-shot the whole app and running out of context mid-implementation, a later session seeing progress and declaring the project finished, and marking features complete without end-to-end testing. Its fix for the last one was to have the agent use browser automation and "do all testing as a human user would." Correlated failures need a different kind of retry, one that changes something (the prompt, the context, the approach, or the model), so the next attempt isn't the same attempt again.

A checkpoint is only as good as what it saves, and for an LLM agent the hard part is that the working state lives in the context window, which fills up and degrades. Anthropic's post on context engineering (September 2025) calls the degradation context rot and describes three ways to work past it:

The harness post (November 2025) turns this into a checkpoint protocol for work that spans many context windows. An initializer session writes an init.sh that starts the dev server, a claude-progress.txt log, and a JSON feature list in which every feature (over 200 in their example) starts out marked failing. Then it makes an initial git commit. Every later coding session starts the same way: read the progress file and the git log, run init.sh and a basic end-to-end test to catch damage from the last session, pick the highest-priority failing feature, and finish by committing and updating the progress file, leaving the environment clean.

Map that onto this handbook's model and each file has a job. The git commit is the checkpoint, the progress file is the resumable state, and the feature list with its passes field is the verifier that keeps "done" honest. Even the file format is a reliability decision: the authors chose JSON because the model was less likely to inappropriately rewrite it than a Markdown file.

Progress files are checkpoints the agent keeps for itself. A durable execution engine keeps them for the program running the agent, so a crashed process can resume instead of restarting. In Temporal, a workflow's steps are recorded in an Event History. After a crash, a worker replays that history to rebuild state and carries on from the last recorded event. In the docs' words, "completed Activities will not re-execute as part of a Workflow Replay." LangGraph's checkpointers do the analogous job for agent graphs: they "persist a thread's graph state as checkpoints," keyed by a thread_id, to support human-in-the-loop pauses, time travel and fault tolerance.

The gap that bites is between doing a side effect and recording it. Temporal notes that an Activity isn't written to history until it returns or errors, and that "Temporal recommends that Activities be idempotent" because retried Activities "may be executed more than once." For an agent that sends email, files tickets or moves money, a crash after the action but before the record means the retry does it twice, unless the action carries a key that is the same on every attempt:

A sketch. The downstream API must honor the key too, because the crash can land between charge() and record(). Practice it in the Idempotency Key Handler challenge.

The field now tracks an agent's time horizon, defined by METR in "Measuring AI Ability to Complete Long Tasks" (March 2025) as the time humans typically take to complete the tasks a model finishes with 50% success. At publication, frontier models such as Claude 3.7 Sonnet sat around 50 minutes, and the frontier horizon had doubled roughly every seven months since 2019. By METR's Frontier Risk Report of May 2026, the public frontier's 50% horizon was about 12 hours, with a wide interval of 5 to 61 hours. METR's Time Horizon 1.1 suite can't reliably measure past 16 hours, so the yardstick itself is running out.

The same report puts the 80% horizon at about 1.5 hours. That gap is this handbook's opening math in the wild: ask for more reliability and the task length a model can carry shrinks by roughly a factor of eight. A team that needs a long-running agent to succeed four times in five is working with a much shorter horizon than the headline number.

The lesson of this handbook is that the horizon isn't only a property of the model. Raw per-step reliability sets a baseline; checkpointing, retries and verification extend it. An agent that saves progress, verifies each step and retries locally has a longer usable horizon than its bare p would predict. So when METR's curve doubles, part of that is better models, and part is better harnesses doing exactly what's above.

The agents that will run for hours — refactoring a codebase, conducting deep research, driving a multi-stage workflow — don't win by having a magically reliable model. They win on reliability engineering: checkpoint often, make state resumable, verify between steps, retry the smallest failing unit. It's the same discipline distributed systems learned decades ago (idempotency, retries, checkpoints), now applied to an agent's step chain.

For a builder the takeaway is liberating: you don't have to wait for a perfect model to ship a long-running agent. You have to engineer around imperfect steps. Measure the decay with agent-evals, guard each step with the patterns in agent patterns, and structure the whole loop with a harness that checkpoints and resumes. The horizon you can reach is mostly up to you.