MemGPT,
explained.

Rung 5's agent has memory and no control over it. A retrieval policy picks what enters the context, scored by rules the designer fixed, and the agent is a passive recipient — it cannot decide that something matters, cannot choose to write a conclusion down, cannot go and look for what it knows it needs.

The change is an analogy carried through completely: treat the context window as RAM and external storage as disk, and make the model the memory manager. It reads and writes its own memory through ordinary function calls — page a document in, write a fact out, search the archive, edit its own working notes. When the context approaches full, memory pressure raises an interrupt and the model is given the chance to summarise and evict before anything is lost, rather than having the oldest content silently fall off the end.

Stated in the units the ladder compares on: rungs 1 to 5 have a bounded effective memory — whatever fits, plus whatever a fixed retrieval policy happens to surface. This has an unbounded one from a bounded context, in exactly the way a program addresses more memory than it has physical RAM. And the decision about what is worth keeping is made by the only component that knows what the task needs.

In 2023, the context windows most people used were around 4–8K tokens (GPT-4 Turbo reached 128K late that year), and even today every window is finite. The failure it causes is intimate: tell an assistant in January that you're vegetarian, and by March — three hundred messages later — it cheerfully recommends a steakhouse. The fact scrolled out of the window, and out of existence. Naive fixes each fail: bigger windows fill eventually (and degrade as they fill); summarizing everything blurs details; RAG over history retrieves but never updates — the model can look things up, but it can't remember that a fact changed.

What long-lived assistants need isn't a bigger window. It's memory management: deciding what stays instantly visible, what gets archived, what gets revised, and what comes back when relevant.

Operating systems solved exactly this problem sixty years ago. Programs address vast virtual memory; the OS keeps hot pages in small fast RAM, parks cold pages on big slow disk, and swaps on demand. The program never knows. MemGPT maps the hierarchy one-to-one: main context (the window) is RAM; external context (a database) is disk; function calls are the paging mechanism.

The split encodes a theory of importance: identity-level facts deserve permanent residence in the window ("vegetarian" should never need retrieval); everything else earns its way back in through search. Getting that boundary right is most of the art of agent memory to this day.

All of it runs on ordinary function calling. MemGPT hands the model tools like core_memory_replace, archival_memory_insert, conversation_search — and the system prompt teaches it to be its own librarian.

Note what happened: the fact wasn't appended, it was revised — the old value is gone from core memory. That's the operation RAG-over-history fundamentally can't do, and it's why MemGPT agents don't recommend steakhouses to vegetarians in March.

And the paging rule is exactly an OS's: when the in-window tokens exceed the budget, evict the oldest buffer messages to out-of-window recall until it fits again — core memory never evicts:

Recent chat ages out to searchable storage; identity facts and revisions stay in the window. The runnable version below pages, revises, and searches.

The OS analogy runs deeper than storage. MemGPT is event-driven: user messages, timers, and function results all arrive as interrupt-like events that wake the processor (the LLM). A heartbeat mechanism lets the model chain multiple steps — search, then read, then answer — without the user prompting each one.

And when the message buffer nears capacity, the system raises a memory-pressure warning — the model is told, in-band, that eviction is coming, and it responds like a good process: summarizes what matters, writes the durable bits to archival storage, and lets the rest page out. Forgetting becomes a managed operation instead of a silent catastrophe.

MemGPT wrote the vocabulary of agent memory. Core vs recall vs archival tiers, memory-exposed-as-tools, summarize-on-pressure, the agent as its own memory manager — that architecture is now visible in nearly every serious agent framework and every "the assistant remembers you" product. The project itself became Letta, a runtime built around persistent self-editing agents.

The deeper legacy is the framing: LLM limitations are systems problems. The model is a processor; intelligence about memory, scheduling and I/O can live in the architecture around it. That lens — the LLM-as-OS-process view — now shapes agent memory design, context engineering, and the whole harness discipline.

MemGPT treats a fixed-context model as a processor and builds an operating-system-style memory around it. The paper's experiments use GPT-4 (an 8,192-token window), GPT-4 Turbo (128,000) and GPT-3.5 Turbo (16,385) as that processor.

1. Divide the prompt into three regions. System instructions are read-only and describe the control flow, the memory tiers and the function schemas. Working context is a fixed-size block of text the model can change only through function calls; in conversation it holds key facts about the user and the persona the agent plays. The FIFO queue holds the rolling message history, including system messages such as memory warnings and the inputs and outputs of function calls, and its first slot is a recursive summary of everything already evicted.
2. Keep two stores outside the prompt. Recall storage is a database of every message the system has processed. Archival storage is a read-write database for text of any length, such as documents or facts the agent decides to save.
3. Run the queue manager on every message. It appends the new message to the queue, assembles the prompt, triggers inference, and writes both the input and the output to recall storage.
4. Warn at 70 percent. When the prompt passes a warning threshold, 70 percent of the window in the paper's example, the queue manager inserts a memory-pressure system message, giving the model a chance to save what matters to working context or archival storage.
5. Flush at 100 percent. When the prompt reaches the window, the manager evicts a block of the oldest messages, half the window in the example, and generates a new recursive summary from the old summary and the evicted messages. The evicted messages leave the prompt but stay searchable in recall storage.
6. Execute the model's function calls. Output is parsed as a function call and its arguments checked; valid calls run, and results and errors, such as trying to write to a full working context, come back as messages so the model can correct itself. Searches return paginated results so one call cannot overflow the window.
7. Chain steps with heartbeats. A call can set request_heartbeat=true, and the processor runs again as soon as the function returns. That is how the agent searches, reads a page of results, searches again and only then answers. Without the flag the call is a yield, and the agent waits for the next event: a user message, a system alert such as a document upload, or a scheduled timer.

Apart from the forced eviction at the limit, nothing moves unless the model decides to move it, which is what the paper means by self-directed memory. The paper's example conversation shows the agent replacing a stored fact about the user when the user's situation changes, instead of keeping both versions side by side. The paper calls the editable block working context; the released code, now part of Letta, calls it core memory.

• W — the backbone's context window, 8,192 tokens in the GPT-4 runs.
• 0.7, 1.0, 0.5 — the paper's example warning, flush and eviction fractions.
• summary′ — the new recursive summary, which folds the old one together with what was evicted.
• recall — the message database; eviction moves messages out of the prompt, never out of the system.

The fractions set the trade-offs. Warn too late and there is no room left to save anything before the flush; evict too little and the system flushes and re-summarizes constantly, compounding the loss in each summary of a summary. Since nothing is deleted, the cost of a bad summary is not forgetting but searching: a fact that survives only in recall storage is used only if the model thinks to look for it.

The conversation tests extend Multi-Session Chat, in which crowdworkers kept consistent personas across five sessions of about a dozen messages each. In deep memory retrieval the user asks, in a new sixth session, a question that only an earlier session can answer. Answers are graded by an LLM judge for accuracy and by ROUGE-L recall. The fixed-context baselines see a lossy summary of the five past sessions; MemGPT sees none of them directly and must page them in by searching recall storage.

A second chat task scores the agent's opening message for a new session by its similarity to the user's persona facts; MemGPT's openers scored above the human-written one on two of the three similarity measures with every backbone, but below it on the third, which compares directly against the human opener; storing facts in working context was what made them work.

Document QA follows the multi-document setup of the lost-in-the-middle study over Wikipedia passages, with 50 sampled questions judged by an LLM. Fixed-context baselines are capped by how many retrieved documents fit and by the retriever itself; MemGPT pages through results in archival storage, so a larger document set does not cap it. Two weaknesses showed: it often stopped paging before exhausting the results, and with GPT-3.5 its accuracy dropped sharply because of weak function calling.

The nested key-value task packs 140 UUID pairs, about 8,000 tokens, into the context, and values can themselves be keys, up to four levels deep. GPT-3.5 fell to 0 percent at one level of nesting, usually returning the first value it found, and GPT-4 and GPT-4 Turbo reached 0 percent by three. MemGPT with GPT-4 was unaffected by depth. With GPT-4 Turbo or GPT-3.5 it beat the baselines but began failing at two levels by not making enough lookups, so the stronger GPT-4 Turbo baseline did not make the stronger MemGPT agent.

The whole design rests on the model being good at paging decisions, which is a hard judgement badly served by the training these models get: knowing what will matter later is a prediction about a future the model cannot see. The characteristic failures are exactly what the analogy predicts — a fact that was never written out and is therefore gone, a summary that discarded the one detail that mattered, and search calls for something that was never stored.

It also costs calls. Memory management is itself model inference, so an agent spends part of its budget on bookkeeping, and that overhead is paid on every interaction whether or not anything needed remembering.

How it held up: the operating-system framing became the dominant way to talk about agent memory, which is a large contribution for an analogy — it supplied the vocabulary (working context versus archive, paging, eviction, pressure) that the field now uses by default, and it continued as a maintained system rather than staying a paper. Read rungs 5 and 6 as the same disagreement that operating systems had: automatic management is more reliable, explicit control is more powerful, and neither has won.

Work each one out before you open it — a number, a prediction or a reason, not a definition. Producing the answer is what makes it stick. Your progress is kept on this device.

Give the model a tool, then a loop, then a memory, then a world. Ends on the 2026 paper measuring how badly the whole stack degrades over many turns.

1. 1ReAct2023
2. 2Toolformer2023
3. 3Reflexion2023
4. 4Voyager2023
5. 5Generative Agents2023
6. 6MemGPT2023
7. 7LLMs Get Lost in Multi-Turn Conversation2026

All 110 papers, in 19 ladders →