Lost in the Middle

Lost in the middle is the observed failure mode where a language model attends well to information at the beginning and end of a long context but is more likely to miss facts placed in the middle — accuracy sags for mid-context content.

Worked example: put the one relevant passage among many distractors and a model often answers correctly when that passage is first or last, but wrongly when it sits in the middle of the prompt. Gotcha: it is why simply stuffing more into a long context can hurt — order matters. Put the most important retrieved content near the question (start or end), and do not assume a big context window means the model reads all of it equally.