The big idea
What is a news feed?
Open the app and you see a personalized stream of posts from everyone you follow, freshest and most relevant on top. Simple to use — but building one reader’s feed means merging posts from hundreds or thousands of authors, ranking them, in a few hundred milliseconds, for billions of readers.
The whole game is when you do the work: merge everyone’s posts at read time (simple, slow) or pre-build each feed at write time (fast reads, heavy writes). We’ll start naive and evolve into the hybrid that real feeds use.
How to read this: We add one piece at a time, problem then fix, and the diagram grows. Hit Begin.
Step 1 · The skeleton
Post and read
Two operations: an author publishes a post, and a reader asks for their feed. Both go through one place, and the post content has to live somewhere durable. How should feeds refer to posts?
Design decision: A post can appear in millions of feeds. How should those feeds store it?
The call: Store each post once; feeds reference it by ID. — One durable copy in the Post Store; feeds hold only IDs and hydrate bodies at read time. Edits, deletes and privacy changes resolve correctly everywhere, for free.
Stand up a Feed API with a Post Store behind it. Writing a post saves a row; reading a feed will (for now) go figure out what to show. This is the spine everything else hangs off.
Separate content from feeds: Posts are stored once, centrally. Feeds will reference posts by ID, never copy their bodies — so an edited or deleted post stays correct everywhere it appears.
Step 2 · The naive read
Pull (fan-out on read)
To build a feed on demand, you’d look up everyone the reader follows in the Social Graph, fetch each one’s recent posts, merge and sort them — every single time they refresh.
This pull model is dead simple and always fresh, and writes are trivial (just save the post). But a reader following 2,000 accounts triggers a 2,000-way query and merge on every open — far too slow at scale.
Pull = cheap writes, costly reads: Read-time fan-out keeps posting instant but makes reading expensive and repetitive. Since reads dwarf writes, paying the cost on every read is exactly backwards for a busy feed.
Step 3 · Flip the work
Push (fan-out on write)
Reads are the hot path, yet pull does the most work there. We want reading a feed to be a single cheap lookup — which means the answer must already exist before the reader asks. When do you build the feed?
Design decision: Reads dominate but pull does its heaviest work on reads. How do you make a feed read O(1)?
The call: Pre-build each follower’s feed at write time (fan-out on write). — When an author posts, push the ID into every follower’s feed cache. Reading is then O(1) — grab your precomputed list. Work moves from many reads to one write.
When an author posts, Fan-out Workers push the post ID into the Feed Cache of every follower. Reading is now O(1): grab your precomputed list of IDs and hydrate the bodies. Work moved from many reads to one write.
Push = fast reads, costly writes: Write-time fan-out makes reads trivial at the cost of write amplification — one post becomes thousands of cache inserts. Great until the author has millions of followers.
Step 4 · Newest isn’t best
Ranking
A purely chronological feed buries the post you’d most want to see under noise. Engagement craters when relevance is left to luck and timestamps. How do you order the feed?
Design decision: A chronological feed buries the best post. How do you order what the reader sees?
The call: Score candidates by affinity, recency and predicted engagement. — Two stages: cheaply gather candidates, then a model orders just those by how close you are to the author, freshness and likely engagement. The same retrieve-then-rank pattern powers search.
Run candidates through a Ranking stage that scores each by affinity (how close you are to the author), recency, and predicted engagement. The feed cache holds candidates; ranking decides their order at (or near) read time.
Candidates, then ranking: Split retrieval from ranking: fan-out cheaply gathers a candidate set, then a model orders just those. The same two-stage pattern powers search and recommendations.
Step 5 · The celebrity problem
Go hybrid
Push breaks for the mega-popular: one post by a celebrity with 50M followers means 50M cache writes — a thundering, slow, expensive fan-out that also wastes effort on inactive followers. What do you do for them?
Design decision: A celebrity with 50M followers posts. Push to all 50M feeds, or something else?
The call: Hybrid: push for normal authors, pull celebrities at read time. — Skip fan-out for the few mega-accounts and merge their recent posts into the precomputed feed on read. The merge set is tiny (a handful of celebs) so reads stay fast and writes stay sane.
Go hybrid. Push for ordinary authors; for celebrities, skip fan-out and let readers pull their recent posts at read time, merging them into the precomputed feed. Route new posts through an event stream so fan-out can absorb spikes and run async.
Best of both: Most authors are cheap to push; a few are ruinous. Detect the high-fan-out accounts and pull them instead — the merge is small (a handful of celebrities) and reads stay fast for everyone.
Step 6 · More than friends
Mixing sources
A modern feed isn’t only people you follow — it’s followed pages, recommended posts you don’t follow yet, and ads. These come from different systems but must feel like one coherent stream. How do you combine them?
Design decision: Friends, pages, recommendations and ads all need to share one feed. How?
The call: Treat each source as a candidate generator feeding one ranker. — Friends, pages, recs and ads each emit candidates; a shared ranker scores and interleaves them under one policy (with ad-spacing and diversity rules). Adding a new source is a new generator, not a new feed.
Gather candidates from each source — friends (feed cache), pages, recommendations, ads — then rank and interleave them under one policy, with rules for ad spacing and diversity so the feed doesn’t clump.
One feed, many candidate sources: Treat every source as a candidate generator feeding a shared ranker. Adding a new source (Reels, suggested groups) is a new generator, not a new feed — the blending logic stays in one place.
Step 7 · The sharp edges
Pagination, dedup & freshness
Readers scroll for ages, so feeds must page without showing the same post twice or skipping new ones that arrive mid-scroll. And a post deleted after fan-out shouldn’t haunt a million feeds. How do you page an ever-changing feed?
Design decision: Infinite scroll, with posts constantly inserted and deleted. How do you paginate?
The call: Cursor-based pagination + per-session dedup + ID hydration. — A stable cursor marks your position so inserts can’t cause dupes or gaps; dedup drops already-seen IDs; and because feeds store IDs, hydration re-checks the Post Store so deleted/edited posts resolve correctly at read time.
Use cursor-based pagination (a stable position, not page numbers) so inserts don’t cause dupes or gaps. De-dupe already-seen IDs per session. Because feeds store IDs, hydration re-checks the Post Store — so deleted or edited posts resolve correctly at read time.
IDs save you again: Storing references instead of copies makes deletes, edits and privacy changes a read-time concern, not a fan-out cleanup nightmare. Cursors keep an infinite scroll consistent under constant change.
You did it
You just designed a news feed.
- A Feed API over a Post Store; feeds reference posts by ID, never copy bodies.
- Pull (fan-out on read) is simple and fresh but slow for big follow counts.
- Push (fan-out on write) precomputes feeds for O(1) reads at write-time cost.
- A ranking stage orders candidates by affinity, recency and engagement.
- A hybrid model pushes for normal authors and pulls celebrities to dodge mega-fan-out.
- Multiple candidate sources (pages, recs, ads) blended under one ranker.
- Cursor pagination, per-session dedup and ID hydration keep an infinite scroll correct.