Toolformer,
explained.

Rung 1 gets tool use from a prompt at inference time, which means the model was never taught when a tool helps — it is being asked to improvise a judgement. This rung's change is to train that judgement in, and to do it with no human annotation whatsoever, which requires a criterion for whether a tool call was a good idea.

The criterion is the rung's whole content and it is unusually elegant. Take ordinary text. Insert a candidate API call at some position, execute it, and splice the result in. Then ask a purely mechanical question: does having that result make the text that follows easier to predict? If the call's answer lowers the loss on the subsequent tokens, it was useful; if not, it was noise. Keep the useful ones, discard the rest, and fine-tune on the surviving text with the calls in place.

Stated in the ladder's units: rung 1 needs a human to write a prompt describing when to use each tool, and this needs a corpus and an executor. The supervision signal is exactly the thing you want — a tool is worth calling when its result carries information the model did not already have — and it is free, because it is the model's own loss. Nobody has to decide in advance which sentences deserve a calculator.

An LLM is a next-word predictor trained on frozen text. That makes it brilliant at language and unreliable at anything text-prediction doesn't guarantee: exact arithmetic (it approximates rather than computes), current facts (its knowledge stops at training time), precise lookups, and faithful translation. It will confidently produce a plausible-but-wrong number or an out-of-date fact.

Humans solve this by using tools — we reach for a calculator, a search engine, a reference. The obvious fix is to let the model do the same: call an external system that's actually correct at the thing the model is bad at. The hard part isn't the tools; it's teaching the model when to call them and what to ask, without hand-annotating thousands of examples.

Toolformer's trick is beautifully simple: represent a tool call as just more text. Interrupt the generation with a special API-call notation, run the tool, splice its result back into the sequence, and carry on. To the model, a calculator call is a token pattern like any other — something it can learn to produce in the right spot.

Because the call and its result live inline, the model reads the tool's answer as ordinary context and predicts the following words with it in hand. This framing means no new architecture is needed — teaching tool use becomes teaching the model to generate the right call text at the right moment, which is exactly what fine-tuning on the right examples can do.

Where do the training examples come from if humans don't label them? This is the paper's real contribution. Toolformer generates its own data and keeps only what helps, judged by a single, elegant criterion: does the tool's result make the next words easier to predict?

Concretely, at many positions in ordinary text the model proposes candidate API calls. Each candidate is actually executed to get a real result. Then it checks: with the call's result inserted, is the model's loss on the subsequent tokens lower than it was without the call (or with a call whose result was withheld)? If the result genuinely helps prediction, the call is useful and is kept; if not, it's discarded. No human ever says "call the calculator here" — reduced prediction loss is the automatic label.

Written out, the filter is one inequality. Keep the call only if its result cuts the loss on the following tokens by at least a margin τ, versus both not calling and calling-but-hiding-the-result:

The "withheld" term is the catch: the call must help because of what it returned, not just because a call is sitting there. The runnable version below applies exactly this test.

Put together, the method is a self-improving loop that starts from just a few hand-written example calls per tool:

The result is a model that, at inference, spontaneously emits an API call when it would help, pauses to let the tool run, reads the result, and continues — all learned, all self-supervised, from a tiny seed of examples.

The paper equipped the model with five external systems, each an antidote to a known LLM weakness:

Each is something the model is unreliable at but a purpose-built system nails. And nothing about the method is specific to these five — any capability you can wrap in a simple API could, in principle, be learned the same way.

Toolformer substantially improved zero-shot performance on tasks that lean on its tools — math word problems, factual question answering, and multilingual queries — often letting a modest-sized model beat much larger models that had no tools. Critically, learning to use tools didn't degrade its core language ability; it stayed a fluent language model that had simply gained new reach.

The demonstration mattered more than any single benchmark: it proved an LLM can be taught, without human supervision, to recognize its own limits and reach outside itself for help. That's a qualitatively different kind of capability from "predict the next word better."

Toolformer's tool calls are essentially one-shot and independent — it doesn't learn to chain tools, use one tool's output as another's input, or run an interactive multi-step loop. That sequential, compositional tool use is exactly what later agent frameworks (like ReAct) tackled. Each tool also needs a handful of seed examples and its own integration, so adding tools isn't free.

And the self-supervised filter, while clever, is a heuristic: "lowers next-token loss" is a proxy for "is genuinely useful," and the two can diverge. The method works cleanly for tools with crisp inputs and outputs, and gets murkier for tools whose value is diffuse or hard to score against text prediction.

Toolformer is a foundational step toward the AI agents that now define the frontier. It established the core idea underneath function calling and tool-using assistants: a language model can autonomously decide when to invoke external software and what to ask it. Every agent that searches the web, runs code, queries a database, or calls an API is exercising the capability Toolformer showed could be learned.

Its self-supervised angle is the lasting lesson — you don't have to hand-label tool use; let the model generate candidate calls and keep the ones that measurably help. Combined with reasoning (chain-of-thought) and interactive loops (ReAct), tool use turned the LLM from a closed oracle into an open system that can act on the world. Toolformer is where "the model can reach outside itself" began.

The method's shape constrains what it can learn. Each call has to be insertable into a span of text and independent — its usefulness is scored locally, against the next few tokens — so it cannot learn chains of dependent calls, where one tool's output determines the next tool's input and neither is individually useful. That is most of what a real agent does. It also needs a pass over the corpus per tool, and the tool set is fixed at training time.

How it held up: the practice went a different way. Tool use is now taught by instruction tuning on demonstrations of tool-calling conversations — including, at the top of the open-weights ladder, on synthesised agentic trajectories at enormous scale — which handles dependent multi-step calls that this method cannot express. So the recipe was superseded fairly quickly.

What survives is the criterion, and it deserves to outlive the paper. "Was this retrieval worth doing" is a question people usually answer with a human judgement or a benchmark, and this rung shows it can be answered mechanically by asking whether the retrieved information reduced the model's uncertainty about what it was about to say. That measurement is available in any system that retrieves anything, and it is still under-used.

Answer each one out loud before you open it. Recalling an answer is what makes it stick; recognising one you have just read is not. Your score is kept on this device.

Give the model a tool, then a loop, then a memory, then a world. Ends on the 2026 paper measuring how badly the whole stack degrades over many turns.

1. 1ReAct2023
2. 2Toolformer2023
3. 3Reflexion2023
4. 4Voyager2023
5. 5Generative Agents2023
6. 6MemGPT2023
7. 7LLMs Get Lost in Multi-Turn Conversation2026

All 110 papers, in 19 ladders →