The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What is Instagram?
Upload a photo; billions of people scroll endless photos. It sounds like a feed, but the twist is media: images are big, immutable, and viewed far more than they’re posted — so the bytes, not the rows, dominate the design.
Two big moves: keep heavy media out of the database (store pixels in a blob store + CDN, keep only pointers in the DB), and precompute feeds so scrolling is a cheap lookup. Everything else builds on those.
What the new pieces do
- Userclient
- Uploads photos and, far more often, scrolls a feed of them. The app is read-heavy and media-heavy — both shape every decision here.
Step 1 · The skeleton
Bytes here, rows there
A post is two very different things: a big image and a little bit of structured data (caption, author, timestamp). Cram the image into the database and it bloats, slows, and gets expensive fast.
A post = a big image + a little caption/author/timestamp. Where do the bytes go?
A multi-MB binary in every row bloats the DB, wrecks backups, and makes queries crawl. Databases are tuned for small structured rows, not megabytes of pixels.
Base64 inflates size ~33% and still drags the binary through your query layer and caches. You’ve made the bytes problem bigger and slower.
Each store does what it’s best at — the blob store holds cheap durable bytes, the DB holds small queryable rows pointing at them. They scale and cost out independently.
Split them. The API Gateway stores the image in an Object Store and writes a row to the Metadata DB holding the caption and the image’s location — a pointer, not the pixels. Each store does what it’s best at.
Why this piece earns its place
Splitting a post across two systems means one post is now two writes with no transaction spanning them, and the interesting question is what happens when the second one fails. Write the row first and a crash before the upload lands leaves a post in someone’s feed pointing at bytes that were never stored — a permanent broken image a user can see. Write the bytes first and the same crash leaves an object nobody references, which costs storage and nothing else. So the order is bytes, then row, and the leftovers get swept: a job that finds objects older than some window with no matching row and reclaims them. Deletion runs the same way in reverse — drop the row, mark the object, let the sweeper unlink it — because the two stores can’t be made to commit together either. It’s worth saying out loud that the pointer is the only thing tying them together: nothing in the object store knows it belongs to a post, so if a metadata row is lost the bytes are unrecoverable in practice even though they are perfectly durable.
What the new pieces do
- API Gatewaybackend
- The single entry point for posting and browsing. Routes uploads to storage and feed requests to the timeline, behind a load balancer.
- Object Storestore
- Cheap, durable blob storage holding the actual image bytes. The database stores only a pointer to the object, never the pixels.
- Metadata DBstore
- Posts, captions, the follow graph and image locations. Small rows, queried constantly — the structured backbone behind the media.
Step 2 · Serve images fast
Push photos to the edge
Serving every image from a central object store adds cross-region latency to every photo and hammers one place. In a media app, slow images are a slow app.
Hundreds of millions of viewers need the same immutable images. How do you serve them?
One origin adds cross-region latency to every photo and becomes a bandwidth choke point. Object stores are for durability, not global low-latency serving.
You’d copy petabytes of cold photos nobody in that region views. You only need the hot slice near each viewer — that’s exactly what a cache is for.
Immutable images cache at the edge with near-infinite TTL — the first viewer pulls from origin, everyone nearby is served from the edge. ~95%+ of bytes never touch origin.
Put a CDN in front of the object store. The first request for an image pulls it from the store; the edge then serves everyone nearby. Since an uploaded image never changes, it caches at the edge with a very long TTL.
Why this piece earns its place
A near-infinite TTL is only safe because the URL and the bytes are the same fact, and that stops being true the moment anything is edited in place. Re-crop a photo, replace a profile picture, or re-encode a variant and write it back to the same path, and every edge that already holds it keeps serving the old pixels until its TTL runs out — which we just said is effectively never. A global purge is the wrong tool here: it is slow, it is rate-limited, and it only works if you know every path involved. The fix is naming. Put a content hash or a version into the object key so a changed image is a different URL and the old one simply ages out unreferenced. That turns the hardest cache problem back into an ordinary write. It is also why the processing step should write each variant to its own derived key instead of overwriting anything — it preserves the property the CDN is leaning on. When that discipline slips the symptom is never an error; it is one user insisting their new photo is up and their friend still seeing the old one.
- immutableimages
- ~95%+from edge
- longTTL
What the new pieces do
- CDNcache
- Serves photos from servers near the viewer. Images are immutable once uploaded, so they cache at the edge essentially forever.
Back of the envelope
- 1 photo → ~4 stored sizes
- thumb, feed, detail, original
- immutable ⇒ TTL = forever
- no edits means no invalidation — the hardest cache problem vanishes
- ~95%+ served from edge
- origin sees only the long-tail miss traffic
Step 3 · One photo, many sizes
Process uploads async
A phone shouldn’t download a 12-megapixel original to show a 150px thumbnail. But generating every size during upload would make posting slow and block the user on heavy CPU work.
A 12MP original is too heavy for a 150px thumbnail. When do you make the other sizes?
You block the user on heavy CPU for every size — posting feels slow and the upload path can’t scale past your encoder fleet.
The post appears instantly; thumbnails and variants finish a beat later, off the request path. Heavy CPU work scales on its own fleet.
On-the-fly transforms redo work for every cache miss and couple your CDN to image logic. Precomputing a few fixed sizes once is far cheaper at this scale.
After storing the original, hand off to Image Processing workers that generate thumbnails and display variants asynchronously and write them back to the blob store. The post appears immediately; its optimized sizes finish a beat later.
Why this piece earns its place
Moving the work off the request path creates a window I have to answer for: the post row exists and its variants don’t. If a feed asks for a display size that hasn’t been written yet, the honest choices are to serve the original and let the client scale it down, or to hold the post out of feeds until processing reports done. I’d do both — hold it out of other people’s feeds, and show authors their own original immediately, so the person who just posted never stares at a gap. Async also buys me a poison message. One photo that kills the encoder — a corrupt file, an unexpected animation, a frame size that blows the memory limit — comes back on the queue and kills the next worker too, and with unlimited retries a single upload can occupy the fleet. So retries are capped, the failure lands in a dead-letter queue for a human, and that post falls back to serving its original indefinitely rather than sitting invisible. Queue depth is the metric that matters here: a spike fails nothing, it just stretches the delay between posting and a sharp thumbnail, and no error rate will tell you.
What the new pieces do
- Image Processingworker
- Asynchronously generates the many sizes and formats each photo needs (feed, thumbnail, full), so the device downloads exactly what it’ll display.
Step 4 · The feed
Precompute the timeline
Building a feed by querying every account a user follows, every time they refresh, is far too slow — reads are the hottest path in the whole app.
Reads dominate. How do you build a user’s feed on refresh?
That’s the hottest path in the app doing a huge fan-in read every scroll. It collapses under load — feeds must be cheap to read.
Do the expensive merge once when someone posts, not on every scroll. Opening the app becomes a single cheap cache read.
Feeds are personalized, ranked and constantly changing — a rendered blob is stale instantly and impossible to update granularly. Cache post IDs, not the rendered page.
Maintain a Feed Cache: a precomputed list of recent post IDs per user, updated by fan-out when people they follow post. Opening the app reads that list and hydrates image URLs from the CDN — fast and cheap.
Why this piece earns its place
Fan-out on write is a knob, not a switch, and the two settings are who gets a list and how long it is. The write cost is followers times posts, and a large share of it is spent on people who won’t open the app today — an account dormant for months still collects every post from everyone it follows into a list nobody reads. So I’d fan out only to recently active followers and rebuild a dormant user’s feed on demand the first time they come back; it is the same merge, just paid once, by someone who is definitely there to see it. Length is the other direction. Keep the list short and it is cheap to write and cheap to hold, but a determined scroller reaches the end and drops into the slow path — on the hottest surface in the product, at the moment that person is most engaged. Keep it long and you pay to maintain pages most sessions never reach. I’d size it to cover a typical session comfortably and treat the fall-through as a real paginated query, not an edge case I can pretend away.
What the new pieces do
- Feed Cachecache
- A ready-made list of post IDs per user so opening the app is one fast lookup, not a live query across everyone they follow.
Step 5 · Likes at scale
Counters the async way
A viral post can take millions of likes in minutes. Doing a synchronous UPDATE count = count + 1 per tap creates brutal write contention on one row and can stall everything.
A viral post takes millions of likes in minutes. How do you count them?
Every tap contends on one row — the classic hot-row write storm. That single post becomes a global lock and everything queues behind it.
Counting millions of rows on every view is even worse than the write. You need a maintained running total, not a scan.
The tap returns instantly (optimistic bump); the authoritative total is aggregated from the stream into sharded counters that spread the hot key. A sliver of staleness buys surviving virality.
Emit likes as events onto a stream and fold them into Counters asynchronously (sharded/approximate counters for the hottest posts). The user sees an instant optimistic bump; the authoritative total catches up moments later.
Why this piece earns its place
What this step quietly bakes in is that a like is a number. It isn’t — it is two facts with different requirements. The total can be approximate and seconds late and nobody can tell. Whether I liked this post is a single bit that has to be exact and durable, because it drives the heart in front of me: if that write is lost, or folded in asynchronously and read back before it lands, the heart fills on the tap and empties on the next scroll past, which reads as a broken app in a way a slightly stale total never does. So membership is a real write keyed by post and user, and it is what makes the event stream safe too — replay a like event twice and the count drifts upward unless the fold deduplicates against that same key. The other consequence is that a split counter is a read cost as well as a write relief: a total spread over N parts is N reads and a sum, so I wouldn’t shard every post by default. Promote a post to a sharded counter once it runs hot, and cache the summed total briefly for everything else.
What the new pieces do
- Countersstore
- Aggregated like and view counts, updated asynchronously so a viral post’s flood of taps never slows posting or reading.
- Eventsbus
- A stream of posts and likes. Feed fan-out, counters and analytics each consume it independently and asynchronously.
Back of the envelope
- 1 viral post × millions/min
- one row = one hot key, the write bottleneck
- shard the counter into N parts
- each tap hits part = hash(user) % N, summed on read
- optimistic UI + async fold
- user sees +1 now; the true total converges in seconds
Step 6 · Here then gone
Stories & ephemerality
Stories vanish after 24 hours. Sweeping the database for expired posts and pulling them from every feed would be a constant, expensive chore.
Stories vanish after 24h. How do you make old ones disappear?
A sweep over billions of rows is a constant expensive chore, and stories linger until the job runs. Expiry should be automatic, not scheduled.
Lifetime becomes a property of the data — the store reclaims it for free and feeds just stop showing it. The same trick powers sessions and OTPs.
Soft-deletes pile up unbounded, bloat every query with a “not deleted” filter, and waste storage on data no one can ever see again.
Give each story a TTL so it auto-expires in storage and the cache; feeds simply stop showing it once it’s gone. No cleanup job — expiry is a property of the data, and the system reclaims it for free.
Why this piece earns its place
A TTL expires one copy, and by now a story exists in three places with three different lifetimes. The row expires on its TTL. The pixels sit in the object store, which reclaims nothing unless a lifecycle rule tells it to. And the image is cached at the edge under the very long TTL we chose for immutable media, so that URL keeps returning a perfectly good photo after the story has vanished from every feed. That gap is fine as “it disappeared from my friends’ view” and not fine as a deletion guarantee, so I want to be careful which one I promise. Story media gets a short-lived link and its own lifecycle rule on the bucket, and expiry is enforced on read — the read path filters by expires-at rather than trusting the store to have already reclaimed the row, because reclamation is lazy and an expired row can stay readable for a while. Put plainly: the TTL is what makes cleanup free, and free cleanup is not the same thing as a delete.
What the new pieces do
- Storiesservice
- Posts that vanish after 24h. A TTL on each story auto-expires it, so storage and feeds clean themselves up without a sweep.
Step 7 · Scale & the sharp edges
Shard everything
Billions of users and posts overflow any single database, and celebrity accounts create hotspots in both the feed fan-out and the counters.
Billions of users overflow one DB, and celebrities create hotspots. What’s the plan?
Replicas help reads, but every write still funnels to one primary that eventually overflows. Vertical scaling has a ceiling; sharding doesn’t.
Hash-sharding spreads the median load across many shards; the handful of extreme accounts get bespoke handling (pull not push, split counters) so no single shard or row is the bottleneck.
Geo-sharding skews badly — follows and virality cross regions freely, creating cross-shard reads and lopsided load. Hash-by-id spreads more evenly for a global social graph.
Shard the Metadata DB (by user/post id) behind the gateway and load-balance the stateless services. Handle celebrities with the hybrid feed (pull, not push) and hot-key counter sharding, so no single row or shard becomes the bottleneck.
Why this piece earns its place
Hashing by id buys an even spread by giving up locality, and the bill arrives on the read. A feed is a list of post ids from many different authors, so hydrating one screen scatters across most of the shards and the user waits on the slowest of them — the request’s latency stops being the average shard and becomes the worst one in the batch, which is where every slow disk, every compaction and every noisy neighbour now surfaces. So hydration goes out in parallel under a deadline, hot post metadata sits in a cache in front of the shards so the common screen never touches them, and anything needing a cross-shard join or a global ordering gets designed out rather than tuned. The second thing I’d settle before it bites is the bucketing. Taking an id modulo the number of shards means the day you add one, nearly every key moves — a migration of the whole dataset to buy a little headroom. Hash into a large fixed set of logical buckets and map buckets onto machines instead, and growth moves a slice while everything else stays put.
You did it
You just designed Instagram.
Everything you assembled, in order
- Split media from metadata: pixels in a blob store, a pointer in the DB.
- A CDN serves immutable images from the edge, so media loads fast.
- Async image processing makes the many sizes without slowing uploads.
- A precomputed feed cache turns scrolling into a cheap lookup.
- Likes/views aggregated asynchronously via events and sharded counters.
- Stories expire automatically via TTLs — no cleanup job needed.
- Sharding plus hybrid feeds and hot-key handling absorb celebrity scale.
