ReAct,
explained.

Rung 1, and the comparison is against reasoning in a closed loop. A model reasoning step by step with no access to anything outside itself can reason fluently from a false premise — each step follows plausibly from the last, and nothing in the process can discover that the second step was wrong. The failure is confident and internally consistent, which is the worst combination.

The change is to interleave reasoning with actions that return results: a thought, then an action in the world, then the observation it produced, then the next thought conditioned on that observation. The loop runs both ways, and both directions matter. Reasoning makes the action better — the model decides what to look up rather than searching blindly. And the observation makes the reasoning better, because a claim that contradicts what just came back from a tool is corrected against something external rather than elaborated.

Stated in the units the ladder cares about: acting without reasoning gives a model no way to plan a sequence of lookups, and reasoning without acting gives it no way to be wrong about facts. The pattern is grounding — every few steps, the chain is anchored to something that was not generated. And it costs nothing to adopt, because like rung 1 of the reasoning ladder it is a prompting pattern rather than a model: no training, no parameters, no gradient.

By 2022 there were two ways to get more out of a language model, and each broke in its own way. Reasoning-only (chain-of-thought) let the model think in steps — but with no access to the outside world, it would reason its way to a confident, wrong answer, hallucinating facts it had no way to check.

Acting-only approaches let the model call tools — search, APIs — but with no reasoning between calls, it flailed: it couldn't plan a sequence of lookups, recover from a dead end, or decide what to do with what it found.

ReAct — short for Reasoning + Acting — simply interleaves them. The model is prompted to alternate between thinking about what to do and doing it, feeding each result back into the next thought. Reasoning decides the next action; the action's result grounds the next round of reasoning.

It's how a person actually researches something: think about what you need, look it up, read what you found, adjust your plan, look up the next thing. Not pure thought, not blind clicking — a conversation between the two.

Concretely, the model cycles through three moves until it can answer:

The observation isn't the model's guess — it's a real result from a real tool. That's the whole point: each loop injects a fact from outside the model's memory.

This trace is a condensed illustration, not a transcript from the paper (in the paper's own Apple Remote example the agent's search[Front Row] fails and it retries with Front Row (software)). No single fact was memorized. The model reasoned about what to fetch, fetched it, and let each observation steer the next step to the answer.

The magic is the synergy. Reasoning makes acting smarter — the model plans which tool to call and why, and recovers when a search comes back empty. Acting makes reasoning honest — every observation is a fact from the world, so the chain of thought can't quietly drift into fiction.

The whole method is one repeated step, and the arrow that matters is the last one — the observation feeds the next thought:

Strip the Observations out and you have plain reasoning — which, on a multi-hop question, has nothing to look up. The runnable version below solves the Apple Remote question both ways.

ReAct is the conceptual backbone of the entire agent era. Function calling, web-browsing assistants, coding agents, research agents — nearly all of them run some version of the Thought–Action–Observation loop this paper named. Frameworks like LangChain and the tool-calling APIs of every major model provider are, at heart, machinery for running ReAct loops.

It gave the field its simplest, most general recipe for turning a next-token predictor into something that can pursue a goal: let it think, let it act, and let what it observes shape what it thinks next.

ReAct is a prompting method, so the whole procedure lives in the prompt and in a small harness around a frozen model. The paper's main runs use PaLM-540B with greedy decoding.

1. Write a handful of trajectories by hand. For HotpotQA the authors picked 6 training questions at random, and for FEVER 3, and wrote out a complete solution for each: every thought, every action and every observation the API returned. They report that more exemplars did not help. The prompt is those trajectories followed by the new question.
2. Expose three actions and nothing else. The knowledge tasks talk to a deliberately simple Wikipedia API. search[entity] returns the first five sentences of that entity's page, or the five most similar titles if no page matches. lookup[string] returns the next sentence on the current page that contains the string, like Ctrl+F. finish[answer] ends the episode. There is no calculator and no free-text search engine; the authors chose an interface weaker than real retrievers so the model has to reason about exactly which title to ask for.
3. Generate one step, then stop. The model writes Thought n: and Action n:; the harness halts generation, executes the action, appends Observation n: with the result and lets the model continue. A thought touches nothing outside the model and returns no observation. It only adds text that later steps condition on.
4. Recover by rewording. When a search misses, the observation lists similar titles and the next thought picks one. One of the hand-written HotpotQA exemplars shows the full pattern: search Colorado orogeny, lookup[eastern sector] to jump to the sentence that matters, land on a disambiguation page for High Plains, search High Plains (United States), and finish with 1,800 to 7,000 ft.
5. Cap the steps, then fall back. ReAct gets 7 steps on HotpotQA and 5 on FEVER. Correct trajectories that needed that many steps were under 1.5 percent of the total, and more steps did not raise the score. Two back-off rules connect ReAct to self-consistency (CoT-SC: 21 chain-of-thought samples at temperature 0.7, majority vote). ReAct hands over to CoT-SC when it runs out of steps without calling finish; CoT-SC hands over to ReAct when its top answer gets fewer than half the votes, a sign that the model's own knowledge is not settled.
6. Thin out thoughts for long tasks. In ALFWorld and WebShop an episode can run to dozens of actions, so thoughts are sparse and the model decides when to write one: to break the goal into subgoals, track progress, or guess where an object is likely to be. ALFWorld prompts use 2 of 3 annotated trajectories per task type, which gives 6 prompt variants; WebShop uses a single example.
7. Optionally fine-tune on its own successes. The authors also kept 3,000 ReAct trajectories that reached the correct answer and fine-tuned PaLM-8B and PaLM-62B to generate whole trajectories, thoughts, actions and observations, from the question alone.

The prompt format

Every exemplar and every live episode has the same plain-text shape: Question:, then numbered Thought, Action and Observation lines, ending in finish[answer]. There is no tool schema and no special training. The action grammar is carried entirely by the exemplars, which is why the same recipe carried over from Wikipedia questions to a text household game and a shopping website by changing only the actions and the exemplars.

• ct — the context: every observation and action so far, which is exactly what the prompt holds.
• π — the policy; here a frozen PaLM-540B reading a few exemplars.
• A — actions that query or change the environment: search, lookup and finish on the knowledge tasks.
• L — the space of language. A thought is an action in L: free, unbounded and invisible to the environment.
• votes(y) — how many of the n sampled chains end in answer y.

At the extremes the equations recover the baselines. Remove every thought and Â collapses to A, which is the Act baseline. Remove every action and only L remains, which is chain of thought. Set the vote threshold to zero and CoT-SC never hands over; set it above n and every question goes to search. The authors note the catch built into the second line: because L is unlimited, learning a policy over it needs strong language priors. Consistent with that, prompted ReAct was the weakest of the four methods at 8B and 62B and worked well only at 540B.

All prompting results use PaLM-540B. HotpotQA is scored by exact match and FEVER by accuracy. The baselines are ablations of the same exemplars: Standard (no thoughts, actions or observations), CoT (thoughts only), CoT-SC (21 sampled chains, majority vote) and Act (actions and observations, no thoughts).

ReAct alone trails both CoT and CoT-SC on HotpotQA and edges them on FEVER, where a claim can hinge on one detail that has to be looked up. The combinations are the best prompts on each task, and both reach CoT-SC's 21-sample score with only 3 to 5 samples.

A hand-labeled error analysis explains the HotpotQA gap: 50 correct and 50 incorrect trajectories from each method. Hallucination caused 56 percent of CoT's failures and none of ReAct's, and 14 percent of CoT's correct answers rested on hallucinated reasoning against 6 percent for ReAct. But 47 percent of ReAct's failures were reasoning errors, including loops that repeat earlier thoughts and actions, 23 percent were searches that returned nothing useful, and 29 percent were label ambiguity.

On the interactive tasks thoughts pay off. In ALFWorld (134 unseen games) the best of six ReAct prompts solved 71 percent, against 45 percent for the best Act prompt and 37 percent for BUTLER, an imitation learner trained on 100,000 expert trajectories per task type; even the worst ReAct prompt, at 48, beat both. A variant whose thoughts only narrated environment feedback, in the style of Inner Monologue, reached 53 at its best. On WebShop, ReAct's success rate was 40.0 against 28.7 for imitation plus reinforcement learning and 59.6 for human experts.

Fine-tuning reverses the small-model result: on 3,000 of its own correct trajectories, PaLM-8B ReAct beat every 62B prompting method and PaLM-62B ReAct beat every 540B prompting method.

Being a prompting pattern is also the limitation. Reliability is entirely the underlying model's, and the characteristic failures were visible immediately: agents that loop, repeating the same search with the same query and the same result; agents that take an action whose observation they then ignore; agents that stop early. Nothing in the pattern prevents any of these, because the pattern is a suggestion about output format.

It is also memoryless across episodes. Whatever the agent works out is lost when the task ends, so the same mistake is available to be made again tomorrow. Rungs 3 to 6 are each an answer to that one sentence.

How it held up: this is the agent loop. Every framework, every tool-calling API, every deployed agent on this site runs thought, action, observation, repeat — sometimes with the labels hidden inside a structured tool-call interface, which is the same loop with better plumbing. Its adoption was so complete that the paper's contribution is easy to understate: before it, "language model uses a tool" and "language model reasons" were separate research directions, and the useful observation was that each one repairs the other's main weakness.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

Give the model a tool, then a loop, then a memory, then a world. Ends on the 2026 paper measuring how badly the whole stack degrades over many turns.

1. 1ReAct2023
2. 2Toolformer2023
3. 3Reflexion2023
4. 4Voyager2023
5. 5Generative Agents2023
6. 6MemGPT2023
7. 7LLMs Get Lost in Multi-Turn Conversation2026

All 110 papers, in 19 ladders →