Voyager,
explained.

Rung 3 stores a sentence about what went wrong. This rung stores a program that works, and the difference is larger than it sounds. A verbal lesson has to be re-interpreted every time it is read, cannot be checked, and does not combine with other lessons. A function can be executed, so its correctness is testable rather than asserted, and it can be called by later functions, so capability compounds into a hierarchy instead of accumulating as advice.

Three parts make that into lifelong learning. An automatic curriculum proposes the next task based on what the agent can currently do and what it has already learned, so difficulty tracks ability without a human setting goals. A skill library holds the self-written, self-verified functions, indexed by a description so the right one can be retrieved when a situation resembles one solved before. And an iterative prompting loop feeds execution errors and environment feedback back in until the code runs and a GPT-4 critic judges that the goal was achieved — so a skill is finished when that self-verification passes, not when a human says so.

Stated in the ladder's units: rung 3 improves within a task and starts over on the next one. This never resets — the library persists, each new skill can be built from existing ones, and the curriculum keeps proposing work at the edge of what the library supports. That is a qualitatively different trajectory from repeated attempts at a fixed task.

Benchmarks hand agents a goal and grade the finish. Open worlds don't work like that — Minecraft has no finish line, thousands of possible pursuits, and progress that compounds: you can't smelt iron before you've built a furnace, mined the ore, crafted the pickaxe, chopped the wood. An agent for a world like this needs three things benchmarks never test: it must choose what to do next, keep what it learns, and build new abilities on old ones.

Classic RL approaches ground away at Minecraft for years (MineRL diamond challenges) with hard-coded rewards and billions of environment steps. Voyager's bet: GPT-4 already knows what a furnace is for — use that knowledge, and let learning happen in code and text instead of weights.

Voyager's first component asks GPT-4, given the agent's current state — inventory, nearby biome, skills already mastered, past failures — one question on repeat: "what's the most useful next task that's neither trivial nor impossible?" Fresh spawn with bare hands → punch a tree. Has wood → craft a table. Iron gear and nightfall → fight a zombie.

Voyager acts by writing JavaScript programs against the Mineflayer API — and every program that verifiably works gets saved: code plus an embedding of a GPT-3.5-written description of it. Facing a new task, the agent retrieves the most relevant stored skills and composes them into the new program.

This is the paper's deepest choice: skills as code. Code is compositional (programs call programs), verifiable (it worked or it didn't), interpretable (you can read what the agent knows), and permanent — immune to context-window amnesia and the catastrophic forgetting that plagues weight updates. The library is a curriculum vitae the agent writes for itself.

Two rules keep the compounding honest, and they're exactly what makes a growing library trustworthy rather than a pile of broken code:

You can't skip up the tech tree, and unverified code never enters — so each new skill stands on solid, tested ground. The runnable version below climbs the tree and enforces both rules.

First drafts of skills usually fail — wrong API call, missing prerequisite, zombie interrupts. Voyager runs a tight repair loop: execute the program, collect three streams of feedback — environment state, execution errors, and a GPT-4 self-verification check on whether the goal was actually achieved — and revise the code. Repeat until the verifier passes or the attempt budget runs out.

It's the Reflexion insight pointed at code, with environment feedback and a GPT-4 critic standing in for a unit test. Only verified programs enter the library — which is what keeps compounding from compounding garbage.

Voyager is the proof-of-concept for agents that improve by accumulating artifacts instead of updating weights. Every modern agent that writes a helper script and reuses it, saves a workflow, or maintains a library of its own tools is running Voyager's play. The pattern even echoes in how humans now package expertise for agents — a skill library, curated by hand instead of discovered by a curriculum, but the same shape: capabilities as retrievable, composable, verified units.

Alongside Reflexion (learning from failure in words) and Generative Agents (memory and reflection for believable behavior), it completed 2023's agent trilogy — the year LLM agents stopped being chatbots with tools and started being systems that grow.

Voyager drives a Minecraft bot through MineDojo and the Mineflayer JavaScript library, with three OpenAI models in fixed roles: gpt-4-0314 for the curriculum, the code and the checking; gpt-3.5-turbo-0301 for cheaper language tasks; and text-embedding-ada-002 for retrieval. Every temperature is 0 except the curriculum's, which is 0.1 to keep proposals varied.

1. Read the world as text. The GPT-4 API available was text-only, so there are no pixels. Each prompt describes the agent's state: inventory, equipment, nearby blocks and entities, biome, time, health and hunger, and position.
2. Propose a task. The curriculum prompt combines a standing directive, to discover as many diverse things as possible without picking tasks too hard for current resources and skills, with the state, the lists of completed and failed tasks, and extra context from GPT-3.5, which asks and answers its own questions about the situation. GPT-4 returns the next task.
3. Retrieve relevant skills. GPT-3.5 writes a general suggestion for solving the task; that suggestion plus the latest environment feedback is embedded and used to pull the five most similar skills from the library.
4. Write a program. GPT-4 reasons step by step, then writes an async JavaScript function. Its prompt holds code-writing guidelines, the control primitives, the retrieved skills, the previous round's code with its feedback, errors and critique, and the state. The authors wrote the primitives on top of Mineflayer: mineBlock collects a given block within 32 blocks, craftItem needs a crafting table nearby, smeltItem needs a furnace, and placeItem, killMob, exploreUntil and two chest functions cover the rest, alongside raw calls such as pathfinding to a goal.
5. Run it and collect three kinds of feedback. Environment feedback arrives as chat messages the primitives emit, such as needing seven more iron ingots for a chestplate. Execution errors come from the interpreter, for example asking for an item that does not exist. Self-verification is a second GPT-4 acting as critic: given the state and the task, it says whether the task succeeded and, if not, how to fix it.
6. Refine for up to four rounds. The next round's prompt carries all three signals. If the critic confirms success, the program is stored and the curriculum proposes a new task; after four rounds without success the task goes on the failed list and the curriculum moves on, free to propose it again later.
7. Index the skill. GPT-3.5 writes a description of the working program; the embedding of that description is the key and the code is the value. Later programs call stored skills by name, so abilities stack.

What the primitives hide

Mining, crafting, smelting and fighting are single function calls, and pathfinding is delegated to Mineflayer, so Voyager plans at the level of collecting three logs rather than holding a mouse button on each block. The authors are explicit about the consequence: they do not compare Voyager with agents that read screen pixels and output low-level controls, because that would not be a like-for-like comparison. The paper is about lifelong learning with GPT-4 as the planner, not about perception or motor control.

• E — the text-embedding-ada-002 embedding; descs is the GPT-3.5 description that keys skill s.
• ⊕ — joining the suggested plan and the latest chat feedback into one query.
• sim — embedding similarity in the vector store; the five best matches enter the code prompt.
• critic — a separate GPT-4 call that reads the state and the task and returns success plus a critique.

The rules bound what can go wrong. An empty library leaves GPT-4 working from primitives alone, which still solved the paper's transfer tasks, only more slowly. Retrieval was checked directly: across 309 samples the right skill was ranked first 80.2 percent of the time and appeared in the top five 96.5 percent of the time, which is why five are passed rather than one. Admission is only as good as the critic, because it rests on a model's judgment, and nothing later re-tests or removes a stored skill.

The baselines were re-implemented for MineDojo, and every method ran three times with a budget of 160 prompting iterations. In that budget Voyager discovered 63 unique items, 3.3 times as many as the baselines, and traveled 2.3 times farther. On the tech tree it unlocked wooden tools 15.3 times faster than the baselines in prompting iterations, stone tools 8.5 times faster and iron tools 6.4 times faster, and it was the only agent to reach diamond tools.

The transfer test cleared the inventory, put the agent in a new world and gave it four unseen tasks. Entries are mean prompting iterations to success, with successful runs out of three; a dash means no run succeeded within 50 iterations.

The library is portable: AutoGPT, which otherwise solved nothing, succeeded on three of the four tasks once given it.

The ablations put a number on each part. Replacing the automatic curriculum with a random one cut discovered items by 93 percent, because many tasks are impossible out of order; a hand-designed curriculum also fell short. Without the skill library, progress plateaued in later stages. Self-verification was the most important feedback signal, and removing it cut items by 73 percent. Using GPT-3.5 instead of GPT-4 as the code writer cost a factor of 5.7 in unique items.

The authors list the limits plainly. GPT-4 cost about 15 times as much as GPT-3.5. The critic sometimes missed a success, for example not treating spider string as evidence that a spider had been beaten. The curriculum sometimes proposed items that do not exist, such as a copper sword, and the code writer tried cobblestone as furnace fuel and called functions outside the provided API. With no vision, building structures such as a Nether portal needed a human acting as critic or curriculum.

The domain does a great deal of the work, and the paper is clear about the setting. The environment offers a programmatic interface and textual feedback, so "write code, run it, read the error" is available; the world is forgiving of failed experiments; and success is often mechanically checkable from the game state. Most real environments supply none of those, and it is the feedback channel rather than the agent design that does not transfer.

Self-verification is another model call, so a skill is admitted to the library on a judgement that can be wrong, and a wrong skill is worse than a wrong reflection because later skills will call it. Nothing prunes or re-validates the library as it grows, which is a maintenance problem the paper does not need to solve at its scale and anything larger would.

How it held up: skills as code is the durable contribution and it became a standard pattern — an agent that writes, tests and reuses its own tools rather than re-deriving behaviour each time. The reason it keeps reappearing is the asymmetry at the top of this block: among all the things an agent might remember, executable memory is the only kind whose correctness can be checked mechanically, and the only kind that composes.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

Give the model a tool, then a loop, then a memory, then a world. Ends on the 2026 paper measuring how badly the whole stack degrades over many turns.

1. 1ReAct2023
2. 2Toolformer2023
3. 3Reflexion2023
4. 4Voyager2023
5. 5Generative Agents2023
6. 6MemGPT2023
7. 7LLMs Get Lost in Multi-Turn Conversation2026

All 110 papers, in 19 ladders →