Voyager,
explained.

Benchmarks hand agents a goal and grade the finish. Open worlds don't work like that — Minecraft has no finish line, thousands of possible pursuits, and progress that compounds: you can't smelt iron before you've built a furnace, mined the ore, crafted the pickaxe, chopped the wood. An agent for a world like this needs three things benchmarks never test: it must choose what to do next, keep what it learns, and build new abilities on old ones.

Classic RL approaches ground away at Minecraft for years (MineRL diamond challenges) with hard-coded rewards and billions of environment steps. Voyager's bet: GPT-4 already knows what a furnace is for — use that knowledge, and let learning happen in code and text instead of weights.

Voyager's first component asks GPT-4, given the agent's current state — inventory, nearby biome, skills already mastered, past failures — one question on repeat: "what's the most useful next task that's neither trivial nor impossible?" Fresh spawn with bare hands → punch a tree. Has wood → craft a table. Iron gear and nightfall → fight a zombie.

Voyager acts by writing JavaScript programs against the Mineflayer API — and every program that verifiably works gets saved: code plus an embedding of its docstring. Facing a new task, the agent retrieves the most relevant stored skills and composes them into the new program.

This is the paper's deepest choice: skills as code. Code is compositional (programs call programs), verifiable (it worked or it didn't), interpretable (you can read what the agent knows), and permanent — immune to context-window amnesia and the catastrophic forgetting that plagues weight updates. The library is a curriculum vitae the agent writes for itself.

First drafts of skills usually fail — wrong API call, missing prerequisite, zombie interrupts. Voyager runs a tight repair loop: execute the program, collect three streams of feedback — environment state, execution errors, and a GPT-4 self-verification check on whether the goal was actually achieved — and revise the code. Repeat until the verifier passes or the attempt budget runs out.

It's the Reflexion insight pointed at code, with the environment as an incorruptible unit test. Only verified programs enter the library — which is what keeps compounding from compounding garbage.

The transfer test sealed it: dropped into a brand-new world with its skill library intact, Voyager solved novel tasks from scratch markedly faster than baselines — the library generalized, not just memorized. Ablations were equally blunt: remove the skill library or the curriculum, and progress flatlines.

Voyager is the proof-of-concept for agents that improve by accumulating artifacts instead of updating weights. Every modern agent that writes a helper script and reuses it, saves a workflow, or maintains a library of its own tools is running Voyager's play. The pattern even echoes in how humans now package expertise for agents — a skill library, curated by hand instead of discovered by a curriculum, but the same shape: capabilities as retrievable, composable, verified units.

Alongside Reflexion (learning from failure in words) and Generative Agents (memory and reflection for believable behavior), it completed 2023's agent trilogy — the year LLM agents stopped being chatbots with tools and started being systems that grow.