MemGPT,
explained.

In 2023, context windows were 4–8K tokens, and even today every window is finite. The failure it causes is intimate: tell an assistant in January that you're vegetarian, and by March — three hundred messages later — it cheerfully recommends a steakhouse. The fact scrolled out of the window, and out of existence. Naive fixes each fail: bigger windows fill eventually (and degrade as they fill); summarizing everything blurs details; RAG over history retrieves but never updates — the model can look things up, but it can't remember that a fact changed.

What long-lived assistants need isn't a bigger window. It's memory management: deciding what stays instantly visible, what gets archived, what gets revised, and what comes back when relevant.

Operating systems solved exactly this problem sixty years ago. Programs address vast virtual memory; the OS keeps hot pages in small fast RAM, parks cold pages on big slow disk, and swaps on demand. The program never knows. MemGPT maps the hierarchy one-to-one: main context (the window) is RAM; external context (a database) is disk; function calls are the paging mechanism.

The split encodes a theory of importance: identity-level facts deserve permanent residence in the window ("vegetarian" should never need retrieval); everything else earns its way back in through search. Getting that boundary right is most of the art of agent memory to this day.

All of it runs on ordinary function calling. MemGPT hands the model tools like core_memory_replace, archival_memory_insert, conversation_search — and the system prompt teaches it to be its own librarian.

Note what happened: the fact wasn't appended, it was revised — the old value is gone from core memory. That's the operation RAG-over-history fundamentally can't do, and it's why MemGPT agents don't recommend steakhouses to vegetarians in March.

And the paging rule is exactly an OS's: when the in-window tokens exceed the budget, evict the oldest buffer messages to out-of-window recall until it fits again — core memory never evicts:

Recent chat ages out to searchable storage; identity facts and revisions stay in the window. The runnable version below pages, revises, and searches.

The OS analogy runs deeper than storage. MemGPT is event-driven: user messages, timers, and function results all arrive as interrupt-like events that wake the processor (the LLM). A heartbeat mechanism lets the model chain multiple steps — search, then read, then answer — without the user prompting each one.

And when the message buffer nears capacity, the system raises a memory-pressure warning — the model is told, in-band, that eviction is coming, and it responds like a good process: summarizes what matters, writes the durable bits to archival storage, and lets the rest page out. Forgetting becomes a managed operation instead of a silent catastrophe.

Two benchmark families, both built to punish goldfish memory. In deep-retrieval conversation tests (recall a fact from far outside the window; answer consistently with an evolving persona), MemGPT with a small window beat fixed-context baselines — including ones given far larger windows. In document analysis, it handled question-answering and nested key-value lookups over document sets vastly exceeding the window, paging chunks in as needed — including multi-hop lookups where each retrieved value is the key to the next search.

The pattern across both: managed small memory beat unmanaged big memory. The window size mattered less than what the system did with it.

MemGPT wrote the vocabulary of agent memory. Core vs recall vs archival tiers, memory-exposed-as-tools, summarize-on-pressure, the agent as its own memory manager — that architecture is now visible in nearly every serious agent framework and every "the assistant remembers you" product. The project itself became Letta, a runtime built around persistent self-editing agents.

The deeper legacy is the framing: LLM limitations are systems problems. The model is a processor; intelligence about memory, scheduling and I/O can live in the architecture around it. That lens — the LLM-as-OS-process view — now shapes agent memory design, context engineering, and the whole harness discipline.

MemGPT's core analogy reframed how to think about context limits: treat the context window like RAM and external storage like disk, with the LLM acting as its own operating system paging information between them. When something important won't fit in the window, it gets written out; when it's needed again, the model calls a function to page it back in. Suddenly a fixed context is a cache over unbounded memory, not a hard ceiling.

The mechanism is function calling. The model decides — via tool calls — what to save, what to retrieve, and what to evict, managing its own memory hierarchy the way an OS manages pages. That self-management is the clever bit: the model isn't just given memory, it operates it, choosing what deserves to stay resident.

This matters because it directly attacks the two things a plain context window can't do: hold a conversation or task longer than the window, and remember across sessions. By spilling to and recalling from external storage, a MemGPT-style agent can carry state across days and far exceed the token limit of any single call — the foundation for assistants with genuine long-term memory.

The pattern is now standard in agent frameworks: a working context plus an external memory store, with explicit read/write/retrieve operations the agent controls. Whether the backing store is a vector database, a document, or a key-value log, the MemGPT idea — the model as manager of a tiered memory system — is how long-running agents keep from forgetting.