Reflexion,
explained.

A ReAct-style agent can act, observe, and adapt within one attempt. But let it fail the whole task and try again, and something embarrassing happens: it repeats the same mistake. Nothing carried over. The standard fixes were both bad — fine-tune on failures (expensive, impossible via API, and one task's lessons overwrite another's) or hope a bigger context magically helps (it doesn't; raw failed trajectories are noise, not lessons).

What's missing is what humans do between attempts: compress the failure into a lesson. Not "here are the 4,000 tokens of my failed attempt" but "I forgot to handle the empty-list case."

Reflexion splits the retry loop into three roles. The Actor attempts the task — writing code, navigating a household simulator, searching for answers. The Evaluator grades the attempt: unit tests for code, task success for environments, an LLM judge where nothing crisper exists. And the Self-Reflection model turns the graded failure into a short, specific verbal lesson.

The lessons accumulate in an episodic memory buffer that's prepended to every new attempt. Attempt three doesn't just know the task — it knows exactly how attempts one and two died.

Classic RL compresses feedback into a single number — reward 0. Everything about why is discarded, and the algorithm spends thousands of episodes rediscovering it through trial and error. A verbal reflection keeps the why: which assumption was wrong, which edge case was missed, what to try instead. It's credit assignment in prose — precise, immediate, and executable by the very next attempt.

The coding result carried the headline: with nothing but self-generated unit-test feedback and verbal retries, GPT-4 jumped eleven points on the benchmark everyone watched. The breadth mattered more — the same three-role loop worked on navigation, search and code, unchanged.

Reflexion learns nothing durable — lessons live in a context buffer, bounded by the window, gone when the episode ends (unless you persist them into real memory). It leans on the Evaluator: crisp signals like unit tests make it soar; fuzzy self-evaluation can reinforce confident nonsense. And reflection quality is capped by the model's own diagnostic ability — an agent that misunderstands why it failed writes itself bad advice and then dutifully follows it.

It's also, quietly, a test-time method: multiple attempts cost multiple rollouts. Like self-consistency and ToT, it buys capability with inference compute.

Reflexion named and validated the pattern that now lives inside nearly every serious agent: try → check → diagnose in words → retry with the diagnosis. Every coding agent that reads its own test failures and patches, every harness with a critique-then-revise step, every eval loop that feeds errors back as guidance — that's the Reflexion loop, productionized.

Its deeper claim aged even better: a model's language channel is a legitimate substrate for learning. That idea threads through self-critique training, agent post-mortems written to memory, and the whole self-improvement research line. Weights are one place to store lessons; words, it turns out, are another.