Gemini 1.5,
explained.

Every earlier leap in language models was about the model. Gemini 1.5's leap is about the prompt: how much you can put in front of the model at once. Its context window reaches 1 million tokens in production and up to 10 million in research — enough to hold entire books, whole codebases, or hours of audio and video in a single request. That changes what a model can do: instead of retrieving snippets and hoping, you can hand it the whole document and ask.

Architecturally Gemini 1.5 is a sparse mixture-of-experts transformer — only a fraction of its parameters fire for any given token, which is part of how a million-token context stays affordable. But a huge window is only useful if the model can actually use all of it. A prompt box that holds a million tokens but only pays attention to the last few thousand is a bigger box, not a longer memory. The proof that Gemini 1.5's memory is real is a deceptively simple test.

The needle-in-a-haystack test makes "does it really use the whole context?" measurable. Take a huge block of filler text (the haystack), hide one small, specific fact inside it (the needle — say, a random passkey), then ask the model to retrieve just that fact. Sweep the needle across many depths (start, middle, end) and many context lengths, and you get a grid of recall scores: green where the model found it, red where it didn't.

For most long-context models that grid lights up red in the early depths of long contexts — the needle placed near the beginning of a very long document is lost. Gemini 1.5's grid was near-uniformly green, ~99%+ recall across the whole range up to a million tokens. Depth barely mattered; length barely mattered. That uniformity is the claim: not "a big window" but "a window it can search end to end."

To see why this is hard, imagine a model whose effective attention reaches only the last W tokens — a sliding window. Anything older than W tokens has scrolled out of view. Put the needle near the end of the context and it's inside the window: found. Put it near the start of a long context and it's beyond W: gone, no matter how big the prompt box is.

So the interesting number isn't the size of the prompt box — it's the usable span. A model advertising a million-token window but effectively attending to only the last 8K would ace needles at the end and fail everything earlier. Gemini 1.5's achievement is that its usable span really does stretch across the entire window, which is exactly the condition the next section makes precise.

Place a needle at depth-fraction f (0 = start, 1 = end) in a context of length L. Its index is about f·(L−1), so its distance from the end is:

A needle at the very end (f=1) is distance 0; at the very start (f=0) it is nearly the whole length away.

A window model attending to the last W tokens retrieves the needle exactly when that distance fits inside the window:

To guarantee recall at every depth, including the very start (f=0), the span must cover the whole context: W ≥ L. That single inequality is the whole long-context game — and it comes at a price, because both self-attention (naively quadratic in L) and the KV cache that stores past tokens (linear in L, ~tens of GB at a million tokens) grow with the length you're trying to span. Gemini 1.5's sparse MoE and serving optimizations are what make W ≥ L ≈ 10⁶ affordable. The runnable version below builds the recall grid and shows a small window collapsing while a full-span one stays perfect.

Gemini 1.5 turned long context from a benchmark curiosity into a usable capability:

The film and rare-language demos made the point vivid: given a grammar book for a language with almost no web presence, in-context, Gemini 1.5 could translate it — learning entirely from the material in the prompt. That's the payoff of a span that reaches the whole context: the prompt becomes the model's working knowledge.

Gemini 1.5 reframed a running debate: do you retrieve relevant chunks and feed a short context (RAG), or do you just put everything in a long one? A million-token window doesn't end retrieval — retrieval is still cheaper and scales past any fixed window — but it dramatically raises the bar for when you need it. Many tasks that used to demand a retrieval pipeline become "paste the whole thing and ask." The engineering shifts from chunking and indexing to managing one very large, very expensive context.

It also made long-context evaluation serious. Needle-in-a-haystack is only the floor — it tests retrieval, not reasoning across a long context — and the field quickly moved to harder tests of multi-fact synthesis over long inputs. But the simple passkey grid remains the honest first question to ask any long-context model: not "how big is your window?" but "can you actually find something at the far end of it?" Gemini 1.5 was the model that made the answer, convincingly, yes.