The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What happens when you open Twitter?
You follow hundreds of accounts. The instant you open the app, you want a feed of their latest posts — newest and most relevant first — in well under a second.
Naively, you’d ask the database for the latest tweets from everyone you follow and merge them on the spot. That works for ten followers and collapses for three hundred million users. The whole art of a feed is deciding when to do that merge: at read time, or ahead of time.
What the new pieces do
- Compose UIclient
- Someone posting a tweet. Hits send once, then expects all their followers to see it within seconds.
- Home Timelineclient
- Anyone opening the app. Expects a fresh, ranked feed of the people they follow — instantly, every time.
Step 1 · The skeleton
Post a tweet, store a tweet
Strip it down and there are two jobs: write a tweet, and read a feed of them. Both need a home for the tweet itself.
Two jobs: post a tweet and read a feed of them. What’s the minimal home for a tweet?
That’s fan-out (step 4) — premature here, and it still needs one canonical copy of the tweet. First you need a single source of truth for the tweet itself.
One durable row per tweet is the immutable spine; everything else is about getting that tweet’s id in front of the right people. A posted tweet never changes, so it’s perfectly cacheable and safe to copy into many feeds.
In-memory tweets vanish on restart and can’t be shared across servers. Tweets must be durably stored; memory is for caching feeds later.
Put a server in the middle and a Tweet Store behind it. To post, the Tweet Service writes a row tweet_id → {author, text, ts}. Everything else we build is about getting that tweet in front of the right people, fast.
Why this piece earns its place
Worth naming what everything downstream will actually copy around: the tweet’s id, not the tweet. Copying the id is what makes it safe to put one post in millions of places, because the row in the Tweet Store stays the single copy anyone reads. The cost is that the store lands on the read path too. Every feed load has to turn a list of ids back into tweets, so the merge we spend the next few steps removing comes back as a hydrate — a scatter of point lookups, one per tweet, on every refresh. That is the load the Tweet Store really feels, and it is why it is the first thing to shard and the first thing that wants a cache in front of it. Reads outnumber writes here roughly a hundred to one, and this is where that ratio quietly lands.
What the new pieces do
- App Serveredge
- The single front door for every request. Grows into an API Gateway behind a load balancer.
- Tweet Serviceservice
- Handles the (relatively rare) write: validate, store the tweet, and kick off delivery to followers.
- Tweet Storestore
- Durable home of every tweet's text, author and timestamp. The source of truth you hydrate from.
Step 2 · The graph
Who follows whom?
A feed is defined entirely by the follow graph: your home timeline is "the tweets of everyone I follow." So we need a fast way to answer two questions — who do I follow? and who follows me?
A feed is "tweets of everyone I follow." What do you need from the follow graph, and how do you store it?
You need BOTH directions: "who do I follow?" to build my feed, and "who follows this poster?" to deliver their tweet on fan-out. Storing only one forces a full scan for the other.
A JOIN over billions of edges per post is far too slow at this scale. Follows are a massive directed graph — use a specialized graph/KV store indexed for the two lookups, not a relational JOIN.
Follows are one-way edges (follower → followee). Both directions are queried constantly — your feed needs your followees, fan-out needs a poster’s followers — so store and index both as adjacency lists.
Add a Graph Service backed by an adjacency store. Following is just an edge follower → followee. The two directions get queried constantly, so we store and index both: a forward list and a reverse list.
Why this piece earns its place
Indexing both directions means a follow is two writes — append to my forward list, append to their reverse list — and those can land separately. The gap is not academic, because the fan-out worker reads the reverse list. A follow that only landed forward means the new follower silently receives nothing until something rebuilds; an unfollow that only landed forward means we keep delivering tweets from someone they dropped, because the poster’s reverse list still names them and the worker keeps writing into an inbox that no longer wants it. I’d make one direction authoritative — the forward list, since it records the user’s own action — write it first, and treat the reverse list as a derived index that can be rebuilt from it. Then divergence has a repair path instead of a support ticket, and anything the read path hydrates gets filtered against the authoritative side. Follows are also wildly uneven in size, which is what the last step is entirely about.
- ~500Musers
- billionsfollow edges
- both waysindexed
What the new pieces do
- Graph Serviceservice
- Knows who follows whom. The fan-out worker asks it 'who follows this person?' on every post.
Step 3 · Read-time merge
The naive feed: pull
First instinct: when you open the app, look up everyone you follow, fetch their recent tweets, merge by time, return. This is the pull (fan-out on read) model.
Add a Timeline Service that does exactly this. It’s simple and writes are cheap — posting just stores one row. But each read now fans out to hundreds of accounts and merges thousands of tweets, every single time someone refreshes. Reads outnumber writes ~100:1, so we’ve put the heavy work on the hottest path.
Why this piece earns its place
Pull doesn’t disappear once the next step arrives — it becomes the fallback. Kill the timeline cache and this is exactly the path every feed read takes, so it has to keep working, and it has to stay bounded. The thing to bound is the fan of followees, because cost here scales with how many accounts a person follows: the p99 belongs to the heaviest users, not the average one. Three hundred accounts is the typical case, and the people following many times that are the ones who time out first. So I’d cap what pull reads — a fixed recency window, a few tweets per followee, a hard limit on the merge — so a degraded feed is short and late rather than a request that never returns. A slow feed for everyone is survivable; a feed that hangs for your heaviest users is how a cache failure becomes an availability incident.
- 100 : 1reads : writes
- 300+accounts merged / read
- slowp99 feed load
What the new pieces do
- Timeline Serviceservice
- Serves the home feed: read the precomputed inbox, hydrate the tweets, hand back a ranked list.
Back of the envelope
- ~100:1 reads : writes
- pull puts the heavy work on the hot path
- 300+ accounts merged / read
- every refresh, every user
- cheap writes, brutal reads
- backwards for a read-dominated product
Step 4 · Fan-out on write
Precompute every inbox
Reads are killing us. But notice: a tweet is written once and read by all a person’s followers. What if we did the expensive merge at write time instead?
Reads outnumber writes ~100:1, and a tweet is written once but read by all followers. How do you fix the hot read path?
A merged feed goes stale the instant anyone you follow posts, so you’d constantly invalidate and recompute — still doing the expensive merge. Precompute at write time, not on reads.
A Kafka event triggers a worker that reads the poster’s followers and writes the tweet id into each follower’s Redis inbox. Reading a feed becomes reading one list — O(1), sub-millisecond. Trade a burst of writes for cheap reads — the right call for a read-dominated product.
Replicas spread read load but every feed read still merges hundreds of accounts’ tweets — the work itself is the problem, not throughput. Precompute the inbox so the read does almost nothing.
When someone posts, drop an event on a stream (Kafka). A Fan-out Worker asks the Graph Service for the poster’s followers and pushes the new tweet id into each follower’s Timeline Cache — a precomputed inbox in Redis. Now reading a feed is just reading one list: O(1), sub-millisecond.
Why this piece earns its place
Delivery is now a stream consumer, so plan on it delivering the same event twice. A worker that dies halfway through a follower list re-reads the event and writes the first half again, and the recovery case on this page — the worker comes back and drains its backlog — is that same replay at scale. If an inbox is an append-only list, a replay hands people duplicate tweets, and a backlog drained out of order hands them a feed that isn’t in time order at all. So the inbox shouldn’t be an append. Make it a keyed structure — the tweet id as the member, the tweet’s own timestamp as the sort — so a repeated write is a no-op and ordering comes from the tweet rather than from whenever the worker got there. That also makes trimming cheap, and it makes replaying the stream converge on the same inbox instead of a longer one.
- O(1)feed read
- asyncdelivery
- <5msinbox fetch
What the new pieces do
- Fan-out Workerworker
- Reads a poster's follower list and writes the new tweet id into each follower's precomputed timeline.
- Timeline Cachecache
- A per-user list of recent tweet ids — the precomputed inbox. Reading a feed is just reading this list.
- Fan-out Streambus
- Every new tweet becomes an event here, letting delivery happen asynchronously instead of blocking the poster.
Back of the envelope
- 500M tweets/day ÷ 86,400 s ≈ 5,800 tweets/sec
- the write rate, straight from Step 5’s own number
- × ~300 followers each ≈ 1.7M inbox writes/sec
- the merge Step 3 did per read, moved to write time and multiplied
- written once, read by all followers
- that 1.7M/sec is the price — and reads become a single list read
- feed read = read one list
- O(1), sub-millisecond
- burst of writes per post
- traded for cheap reads — a huge win
Step 5 · Scale the stores
Billions of tweets and edges
At hundreds of millions of tweets a day, no single database holds the Tweet Store or the follow graph, and no one Redis box holds every inbox.
No single box holds the tweet store, the follow graph, or every inbox. How do you partition them?
A single shared key doesn’t match how each store is read — a feed read keyed by region would still scatter. Shard each store by its own access key so a read touches one shard.
Caching helps reads but the authoritative stores still outgrow one machine’s disk and write capacity. You must partition the data itself across machines.
Pick the shard key to match the read: a feed read keyed by user id touches just one timeline shard, and random-ish ids spread load evenly — no hotspots (until a celebrity, next). Add nodes to grow.
Shard everything. Partition the Tweet Store by tweet id, the graph by user id, and the timeline cache by user id — so each user’s inbox lives on a predictable node. Every component scales out horizontally; add machines as you grow.
Why this piece earns its place
Sharding inboxes by user id is the right key for reads and the worst possible key for this write path. A poster’s followers are spread across every timeline shard by construction — that’s what an even key does — so one tweet becomes a few hundred writes scattered over the whole cluster. Done naively that’s a network round trip per follower, and the fan-out worker becomes the bottleneck long before Redis does. The worker should group the follower list by shard first and send one batched write per shard, so calls per tweet track the number of shards rather than the number of followers. The other thing this key buys is cheap resharding: inboxes are derived, and the graph plus the Tweet Store can rebuild any of them, so adding nodes doesn’t have to migrate data — a moved inbox can simply be rebuilt on next read.
- 500M+tweets / day
- shardedby id
- +nodesto grow
Step 6 · Rank it
Newest isn’t best
A pure reverse-chronological inbox buries the good stuff under noise. People engage far more with a feed that surfaces what they actually care about.
A reverse-chronological inbox buries the good stuff. How do you surface what people care about — without scoring all of Twitter?
Ranking all of Twitter per feed load is astronomically expensive and pointless — most tweets could never appear in your feed. You only need to score what’s plausibly relevant.
The precomputed inbox is a cheap candidate set (a few hundred ids); a heavier ranking model scores only those by predicted engagement (affinity, recency, media, behaviour) and reorders them. Retrieve cheap, rank narrow — every modern feed works this way.
Manual sorting doesn’t scale to hundreds of tweets and most users won’t do it. Engagement comes from automatically surfacing relevant tweets — a ranking model over the candidate set.
Insert a Ranking Service on the read path. It pulls the candidate tweet ids from the inbox, scores each by predicted engagement (author affinity, recency, media, past behaviour), and returns the reordered list. The inbox becomes a candidate set, not the final order.
Why this piece earns its place
Putting the ranker on the read path means the feed’s latency is now the model’s latency. Everything up to here was built so that a refresh is one list read, and this step puts a scoring service in front of the user and waits on it synchronously. So it needs a timeout, and the timeout needs somewhere to go. The good news is the fallback is free: candidates arrive already in time order, so a ranker that’s slow, overloaded or down degrades to a reverse-chronological feed — the product this site shipped for years — instead of an error. I’d make that an explicit path with its own alarm rather than an accident of a caught exception, because a silently chronological feed looks completely fine and surfaces days later as an engagement dip. Candidate count is the cost dial: score more and the feed improves while every single refresh gets slower.
What the new pieces do
- Ranking Svcservice
- Reorders the raw timeline by predicted engagement so the best tweets surface first, not just the newest.
Step 7 · The sharp edges
The celebrity problem
Fan-out on write breaks for a star with 100M followers: one tweet would trigger 100M inbox writes — a "fan-out storm" that lags delivery for everyone.
A star with 100M followers posts — fan-out-on-write means 100M inbox writes, a "fan-out storm." How do you avoid it?
Even fast, 100M writes per celebrity tweet lags delivery for everyone and wastes enormous work — most of those inboxes won’t be read before the next tweet. More workers don’t fix the wrong model for mega-accounts.
Pure pull reintroduces the brutal read cost for all the normal accounts that fan-out-on-write made fast. You only want to change behavior for the rare mega-accounts.
Don’t fan out a celebrity’s tweets; pull their recent posts when a follower loads their feed and merge into the precomputed inbox. Fast feeds for the common case, no storm for the rare one. Knowing when each model wins — and combining them — is the senior-interview answer.
Go hybrid. For normal accounts, push (fan-out on write). For a handful of celebrities, don’t fan out — instead, pull their recent tweets at read time and merge them into your precomputed feed. You get fast feeds for the common case and avoid the storm for the rare one. Layer on rate limits, abuse checks, and graceful handling of deletes.
Why this piece earns its place
The real design question is where the line sits, because celebrity isn’t a category the data hands you — it’s a follower-count threshold someone picks. Set it too high and merely large accounts still storm the fan-out. Set it too low and you’ve moved a big slice of the graph onto the read path, so every feed load merges a growing pile of pull-lists, which is the previous pull model leaking back in one account at a time. The harder part is that accounts cross the line, in both directions. At the moment of the flip a follower’s inbox still holds that account’s already-pushed tweets while the read path is also pulling them, so the merge has to deduplicate by tweet id or people see doubles. I’d make it a per-account flag with a deliberate transition rather than a live comparison against a number, so a crossing is an event you can watch instead of a boundary accounts oscillate across.
- 100Mceleb followers
- pushfor most
- pullfor stars
Back of the envelope
- 100M followers × 1 tweet
- = 100M inbox writes (a storm)
- push for the many
- O(followers) writes, fine for most
- pull for the few stars
- merge their tweets at read time
You did it
You just designed Twitter’s feed.
Everything you assembled, in order
- Tweet Service + Tweet Store — the immutable spine of every post.
- A directed follow graph, indexed in both directions.
- Pull (fan-out on read) is simple but punishes the hot read path.
- Fan-out on write precomputes each inbox in Redis — O(1) feed reads.
- Sharded tweet store, graph and timeline cache to scale horizontally.
- Retrieve from the inbox, then rank by predicted engagement.
- Hybrid fan-out — push for most, pull for celebrities — solves the storm.