Reflexion,
explained.

A ReAct-style agent can act, observe, and adapt within one attempt. But let it fail the whole task and try again, and something embarrassing happens: it repeats the same mistake. Nothing carried over. The standard fixes were both bad — fine-tune on failures (expensive, impossible via API, and one task's lessons overwrite another's) or hope a bigger context magically helps (it doesn't; raw failed trajectories are noise, not lessons).

What's missing is what humans do between attempts: compress the failure into a lesson. Not "here are the 4,000 tokens of my failed attempt" but "I forgot to handle the empty-list case."

Reflexion splits the retry loop into three roles. The Actor attempts the task — writing code, navigating a household simulator, searching for answers. The Evaluator grades the attempt: unit tests for code, task success for environments, an LLM judge where nothing crisper exists. And the Self-Reflection model turns the graded failure into a short, specific verbal lesson.

The lessons accumulate in an episodic memory buffer that's prepended to every new attempt. Attempt three doesn't just know the task — it knows exactly how attempts one and two died.

Classic RL compresses feedback into a single number — reward 0. Everything about why is discarded, and the algorithm spends thousands of episodes rediscovering it through trial and error. A verbal reflection keeps the why: which assumption was wrong, which edge case was missed, what to try instead. It's credit assignment in prose — precise, immediate, and executable by the very next attempt.

The whole loop is one line of state that survives between attempts — the memory of what went wrong:

Delete that memory arrow and every retry is attempt one again. The runnable version below shows it fix the boundary bug on attempt two — and a memoryless retry failing forever.

The coding result carried the headline: with nothing but self-generated unit-test feedback and verbal retries, GPT-4 jumped eleven points on the benchmark everyone watched. The breadth mattered more — the same three-role loop worked on navigation, search and code, unchanged.

Reflexion learns nothing durable — lessons live in a context buffer, bounded by the window, gone when the episode ends (unless you persist them into real memory). It leans on the Evaluator: crisp signals like unit tests make it soar; fuzzy self-evaluation can reinforce confident nonsense. And reflection quality is capped by the model's own diagnostic ability — an agent that misunderstands why it failed writes itself bad advice and then dutifully follows it.

It's also, quietly, a test-time method: multiple attempts cost multiple rollouts. Like self-consistency and ToT, it buys capability with inference compute.

Reflexion named and validated the pattern that now lives inside nearly every serious agent: try → check → diagnose in words → retry with the diagnosis. Every coding agent that reads its own test failures and patches, every harness with a critique-then-revise step, every eval loop that feeds errors back as guidance — that's the Reflexion loop, productionized.

Its deeper claim aged even better: a model's language channel is a legitimate substrate for learning. That idea threads through self-critique training, agent post-mortems written to memory, and the whole self-improvement research line. Weights are one place to store lessons; words, it turns out, are another.

Reflexion splits the agent into three roles, and getting each right is what makes it work. The actor attempts the task; the evaluator judges the attempt (a test suite, a reward signal, an exact-match check); and the self-reflection step turns a failure into a short natural-language lesson stored in memory. The next attempt reads those reflections back into context. The whole method lives or dies on the evaluator: if it can't tell success from failure, the reflections are noise and the loop just churns.

That is why the method shines where feedback is concrete and cheap. Coding tasks with unit tests, decision-making tasks with a clear win/lose signal, and puzzles with a checkable answer all give the evaluator something real to grade, so the verbal lesson ("I forgot to handle the empty-list case") is grounded and actionable. Tasks with vague or delayed feedback give it little to reflect on, and the gains shrink.

The verbal signal is the clever part. Classic reinforcement learning nudges weights with a scalar reward; Reflexion instead writes an explicit sentence about what went wrong and what to try next, and feeds it back through the context window. It needs no gradient updates or fine-tuning — the "learning" is entirely in memory and prompt — which is what let it work on a frozen model. But that also bounds it: the lesson only helps while it stays in context, and a reflection that doesn't generalize can misfire on the next attempt.

Modern agent loops inherited the core idea directly. The pattern of act → evaluate → reflect → retry, with lessons carried in external memory, is now standard in agent harnesses that iterate on tests, retry failed tool calls with the error fed back, and keep a running scratchpad of what has and hasn't worked. Reflexion was an early, clean statement of the loop that most self-improving agents now run.