Toolformer,
explained.

Rung 1 gets tool use from a prompt at inference time, which means the model was never taught when a tool helps — it is being asked to improvise a judgement. This rung's change is to train that judgement in, and to do it with no human annotation whatsoever, which requires a criterion for whether a tool call was a good idea.

The criterion is the rung's whole content and it is unusually elegant. Take ordinary text. Insert a candidate API call at some position, execute it, and splice the result in. Then ask a purely mechanical question: does having that result make the text that follows easier to predict? If the call's answer lowers the loss on the subsequent tokens, it was useful; if not, it was noise. Keep the useful ones, discard the rest, and fine-tune on the surviving text with the calls in place.

Stated in the ladder's units: rung 1 needs a human to write a prompt describing when to use each tool, and this needs a corpus and an executor. The supervision signal is exactly the thing you want — a tool is worth calling when its result carries information the model did not already have — and it is free, because it is the model's own loss. Nobody has to decide in advance which sentences deserve a calculator.

An LLM is a next-word predictor trained on frozen text. That makes it brilliant at language and unreliable at anything text-prediction doesn't guarantee: exact arithmetic (it approximates rather than computes), current facts (its knowledge stops at training time), precise lookups, and faithful translation. It will confidently produce a plausible-but-wrong number or an out-of-date fact.

Humans solve this by using tools — we reach for a calculator, a search engine, a reference. The obvious fix is to let the model do the same: call an external system that's actually correct at the thing the model is bad at. The hard part isn't the tools; it's teaching the model when to call them and what to ask, without hand-annotating thousands of examples.

Toolformer's trick is beautifully simple: represent a tool call as just more text. Interrupt the generation with a special API-call notation, run the tool, splice its result back into the sequence, and carry on. To the model, a calculator call is a token pattern like any other — something it can learn to produce in the right spot.

Because the call and its result live inline, the model reads the tool's answer as ordinary context and predicts the following words with it in hand. This framing means no new architecture is needed — teaching tool use becomes teaching the model to generate the right call text at the right moment, which is exactly what fine-tuning on the right examples can do.

Where do the training examples come from if humans don't label them? This is the paper's real contribution. Toolformer generates its own data and keeps only what helps, judged by a single, elegant criterion: does the tool's result make the next words easier to predict?

Concretely, at many positions in ordinary text the model proposes candidate API calls. Each candidate is actually executed to get a real result. Then it checks: with the call's result inserted, is the model's loss on the subsequent tokens lower than it was without the call (or with a call whose result was withheld)? If the result genuinely helps prediction, the call is useful and is kept; if not, it's discarded. No human ever says "call the calculator here" — reduced prediction loss is the automatic label.

Written out, the filter is one inequality. Keep the call only if its result cuts the loss on the following tokens by at least a margin τ, versus both not calling and calling-but-hiding-the-result:

The "withheld" term is the catch: the call must help because of what it returned, not just because a call is sitting there. The runnable version below applies exactly this test.

Put together, the method is a self-improving loop that starts from just a few hand-written example calls per tool:

The result is a model that, at inference, spontaneously emits an API call when it would help, pauses to let the tool run, reads the result, and continues — all learned, all self-supervised, from a tiny seed of examples.

The paper equipped the model with five external systems, each an antidote to a known LLM weakness:

Each is something the model is unreliable at but a purpose-built system nails. And nothing about the method is specific to these five — any capability you can wrap in a simple API could, in principle, be learned the same way.

Toolformer's tool calls are essentially one-shot and independent — it doesn't learn to chain tools, use one tool's output as another's input, or run an interactive multi-step loop. That sequential, compositional tool use is what prompting-based agent frameworks such as ReAct — published a few months before Toolformer — were built around. Each tool also needs a handful of seed examples and its own integration, so adding tools isn't free.

And the self-supervised filter, while clever, is a heuristic: "lowers next-token loss" is a proxy for "is genuinely useful," and the two can diverge. The method works cleanly for tools with crisp inputs and outputs, and gets murkier for tools whose value is diffuse or hard to score against text prediction.

Toolformer is a foundational step toward the AI agents that now define the frontier. It established the core idea underneath function calling and tool-using assistants: a language model can autonomously decide when to invoke external software and what to ask it. Every agent that searches the web, runs code, queries a database, or calls an API is exercising the capability Toolformer showed could be learned.

Its self-supervised angle is the lasting lesson — you don't have to hand-label tool use; let the model generate candidate calls and keep the ones that measurably help. Combined with reasoning (chain-of-thought) and interactive loops (ReAct), tool use turned the LLM from a closed oracle into an open system that can act on the world. Toolformer is where "the model can reach outside itself" began.

Toolformer turns a pretrained model into its own data generator. The base model M is GPT-J, with 6.7 billion parameters, and the text C is a subset of CCNet. The procedure runs once per tool, and the results are merged at the end.

1. Write a call as text. A call is a tool name and an input string, linearized as <API> Calculator(400 / 1400) → 0.29 </API>. In practice the markers are ordinary tokens, square brackets and an arrow, so the vocabulary does not change.
2. Narrow the corpus per tool. Annotating everything is expensive, so heuristics pick likely text. For the calculator: documents with at least three numbers within 100 tokens where one is the result of an operation on two others, or phrases such as equals or total of followed by a number. For the calendar: documents whose URL yields a creation date, about 18 percent of them.
3. Find where a call might go. A tool-specific prompt P(x) shows a few hand-written example calls. For every position i, the model's probability of opening a call there is computed; positions above a sampling threshold τ_s are kept, up to the top k.
4. Sample candidate calls. At each kept position the model generates up to m calls, prompted with P(x), the text so far and the opening token, until it closes the call. Samples that never close are discarded.
5. Execute them. Question answering is Atlas, a retrieval-augmented model fine-tuned on Natural Questions. Wikipedia search is BM25 over the KILT Wikipedia dump. The calculator does the four basic operations and rounds to two decimals. The calendar returns the current date. Translation is the 600M-parameter NLLB model into English, with fastText detecting the source language.
6. Keep only calls whose answer helps. The filter measures the loss on the tokens after position i three ways: with the call and its result, with no call, and with the call but no result. The call is placed in front of the text as a prefix rather than at position i, because the untrained model has never seen a call mid-sentence, and splicing one in would raise its loss for the wrong reason. A call survives if the result beats the better alternative by τ_f; both thresholds are set per tool.
7. Merge and fine-tune. Surviving calls are inserted at their positions to form C*, which holds exactly the texts of C plus calls, so fine-tuning should not shift the model's language distribution. GPT-J is fine-tuned on C* with batch size 128 and learning rate 1×10^⁻⁵, warming up over the first 10 percent of training.
8. Decode with live tools. At inference the model decodes normally until it emits the arrow, meaning it expects a result. Decoding pauses, the API runs, the response and closing token are inserted, and generation resumes. In evaluation a call starts whenever the opening token is among the 10 most likely next tokens, not only the most likely, and each input gets at most one call so the model cannot loop.

• z — the prefix placed before the text: nothing, the call alone, or the call with its result.
• e(ci, ri) — the linearized call for position i together with the result it returned; e(ci, ε) is the same call with the result left empty.
• wt — the weight on the token t places after the call position; it falls by 0.2 per token and is zero from the sixth token on.
• τf — the minimum loss reduction, set per tool so enough examples survive.

The minimum inside L^⁻ is the point. Comparing only against no call would credit calls whose text alone gives the answer away, such as a question that restates the fact; requiring the result to beat the bare call means the help has to come from what the tool returned. The weights decide where calls land: a result that only becomes useful six tokens later earns nothing, so surviving calls sit right before the words they explain. Set τ_f near zero and almost every harmless call survives; raise it and only calls whose answers are nearly copied into the next tokens remain.

Every evaluation is zero-shot, with lenient scoring because the models are not told the answer format. The comparisons are GPT-J; Toolformer with calls disabled at inference, which separates what the fine-tuning taught from what the tools supply; and GPT-3 (175B), about 25 times larger. Tools that would make a task trivial were switched off: Wikipedia search on LAMA, whose facts come from Wikipedia, and the QA tool on the question-answering sets, since it was fine-tuned on Natural Questions.

The model learned when to call: the calculator on 97.9 percent of math examples, the QA tool on 98.1 percent of LAMA, Wikipedia search on 99.3 percent of the question sets. On math and dates, a 6.7B model with tools beats GPT-3 outright.

The losses are as instructive. Toolformer stays behind GPT-3 on all three question-answering sets; the authors blame a weak BM25 search engine and the model's inability to reformulate a query or browse past the top result. On the multilingual MLQA benchmark the translation tool helped in every language, but fine-tuning on CCNet hurt enough that Toolformer did not consistently beat plain GPT-J. On TempLAMA the gain did not come from the calendar, because the right strategy, a calendar call followed by a QA call, is ruled out by the one-call limit.

Language modeling was unharmed: with calls disabled, perplexity matched GPT-J fine-tuned on the same text without calls, 10.3 on WikiText and 10.5 on held-out CCNet. Size matters, though. Run on GPT-2 models of 124M, 355M, 775M and 1.6B parameters with three tools, the ability to use them only appeared at around 775M; smaller models scored about the same with and without tools.

The method's shape constrains what it can learn. Each call has to be insertable into a span of text and independent — its usefulness is scored locally, against the next few tokens — so it cannot learn chains of dependent calls, where one tool's output determines the next tool's input and neither is individually useful. That is most of what a real agent does. It also needs a pass over the corpus per tool, and the tool set is fixed at training time.

How it held up: the practice went a different way. Tool use is now taught by instruction tuning on demonstrations of tool-calling conversations — including, at the top of the open-weights ladder, on synthesised agentic trajectories at enormous scale — which handles dependent multi-step calls that this method cannot express. So the recipe was superseded fairly quickly.

What survives is the criterion, and it deserves to outlive the paper. "Was this retrieval worth doing" is a question people usually answer with a human judgement or a benchmark, and this rung shows it can be answered mechanically by asking whether the retrieved information reduced the model's uncertainty about what it was about to say. That measurement is available in any system that retrieves anything, and it is still under-used.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

Give the model a tool, then a loop, then a memory, then a world. Ends on the 2026 paper measuring how badly the whole stack degrades over many turns.

1. 1ReAct2023
2. 2Toolformer2023
3. 3Reflexion2023
4. 4Voyager2023
5. 5Generative Agents2023
6. 6MemGPT2023
7. 7LLMs Get Lost in Multi-Turn Conversation2026

All 110 papers, in 19 ladders →