The whole design, in writing
Learn system design by building a photo-sharing app like Instagram step by step. An interactive guide covering media upload, blob storage and CDN delivery, image processing, the timeline feed, like/view counters, ephemeral stories, and sharding for scale.
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What is Instagram?
Upload a photo; billions of people scroll endless photos. It sounds like a feed, but the twist is media: images are big, immutable, and viewed far more than they’re posted — so the bytes, not the rows, dominate the design.
Two big moves: keep heavy media out of the database (store pixels in a blob store + CDN, keep only pointers in the DB), and precompute feeds so scrolling is a cheap lookup. Everything else builds on those.
What the new pieces do
- Userclient
- Uploads photos and, far more often, scrolls a feed of them. The app is read-heavy and media-heavy — both shape every decision here.
Step 1 · The skeleton
Bytes here, rows there
A post is two very different things: a big image and a little bit of structured data (caption, author, timestamp). Cram the image into the database and it bloats, slows, and gets expensive fast.
A post = a big image + a little caption/author/timestamp. Where do the bytes go?
A multi-MB binary in every row bloats the DB, wrecks backups, and makes queries crawl. Databases are tuned for small structured rows, not megabytes of pixels.
Base64 inflates size ~33% and still drags the binary through your query layer and caches. You’ve made the bytes problem bigger and slower.
Each store does what it’s best at — the blob store holds cheap durable bytes, the DB holds small queryable rows pointing at them. They scale and cost out independently.
Split them. The API Gateway stores the image in an Object Store and writes a row to the Metadata DB holding the caption and the image’s location — a pointer, not the pixels. Each store does what it’s best at.
What the new pieces do
- API Gatewaybackend
- The single entry point for posting and browsing. Routes uploads to storage and feed requests to the timeline, behind a load balancer.
- Object Storestore
- Cheap, durable blob storage holding the actual image bytes. The database stores only a pointer to the object, never the pixels.
- Metadata DBstore
- Posts, captions, the follow graph and image locations. Small rows, queried constantly — the structured backbone behind the media.
Step 2 · Serve images fast
Push photos to the edge
Serving every image from a central object store adds cross-region latency to every photo and hammers one place. In a media app, slow images are a slow app.
Hundreds of millions of viewers need the same immutable images. How do you serve them?
One origin adds cross-region latency to every photo and becomes a bandwidth choke point. Object stores are for durability, not global low-latency serving.
You’d copy petabytes of cold photos nobody in that region views. You only need the hot slice near each viewer — that’s exactly what a cache is for.
Immutable images cache at the edge with near-infinite TTL — the first viewer pulls from origin, everyone nearby is served from the edge. ~95%+ of bytes never touch origin.
Put a CDN in front of the object store. The first request for an image pulls it from the store; the edge then serves everyone nearby. Since an uploaded image never changes, it caches at the edge with a very long TTL.
- immutableimages
- ~95%+from edge
- longTTL
What the new pieces do
- CDNcache
- Serves photos from servers near the viewer. Images are immutable once uploaded, so they cache at the edge essentially forever.
Back of the envelope
- 1 photo → ~4 stored sizes
- thumb, feed, detail, original
- immutable ⇒ TTL = forever
- no edits means no invalidation — the hardest cache problem vanishes
- ~95%+ served from edge
- origin sees only the long-tail miss traffic
Step 3 · One photo, many sizes
Process uploads async
A phone shouldn’t download a 12-megapixel original to show a 150px thumbnail. But generating every size during upload would make posting slow and block the user on heavy CPU work.
A 12MP original is too heavy for a 150px thumbnail. When do you make the other sizes?
You block the user on heavy CPU for every size — posting feels slow and the upload path can’t scale past your encoder fleet.
The post appears instantly; thumbnails and variants finish a beat later, off the request path. Heavy CPU work scales on its own fleet.
On-the-fly transforms redo work for every cache miss and couple your CDN to image logic. Precomputing a few fixed sizes once is far cheaper at this scale.
After storing the original, hand off to Image Processing workers that generate thumbnails and display variants asynchronously and write them back to the blob store. The post appears immediately; its optimized sizes finish a beat later.
What the new pieces do
- Image Processingworker
- Asynchronously generates the many sizes and formats each photo needs (feed, thumbnail, full), so the device downloads exactly what it’ll display.
Step 4 · The feed
Precompute the timeline
Building a feed by querying every account a user follows, every time they refresh, is far too slow — reads are the hottest path in the whole app.
Reads dominate. How do you build a user’s feed on refresh?
That’s the hottest path in the app doing a huge fan-in read every scroll. It collapses under load — feeds must be cheap to read.
Do the expensive merge once when someone posts, not on every scroll. Opening the app becomes a single cheap cache read.
Feeds are personalized, ranked and constantly changing — a rendered blob is stale instantly and impossible to update granularly. Cache post IDs, not the rendered page.
Maintain a Feed Cache: a precomputed list of recent post IDs per user, updated by fan-out when people they follow post. Opening the app reads that list and hydrates image URLs from the CDN — fast and cheap.
What the new pieces do
- Feed Cachecache
- A ready-made list of post IDs per user so opening the app is one fast lookup, not a live query across everyone they follow.
Step 5 · Likes at scale
Counters the async way
A viral post can take millions of likes in minutes. Doing a synchronous UPDATE count = count + 1 per tap creates brutal write contention on one row and can stall everything.
A viral post takes millions of likes in minutes. How do you count them?
Every tap contends on one row — the classic hot-row write storm. That single post becomes a global lock and everything queues behind it.
Counting millions of rows on every view is even worse than the write. You need a maintained running total, not a scan.
The tap returns instantly (optimistic bump); the authoritative total is aggregated from the stream into sharded counters that spread the hot key. A sliver of staleness buys surviving virality.
Emit likes as events onto a stream and fold them into Counters asynchronously (sharded/approximate counters for the hottest posts). The user sees an instant optimistic bump; the authoritative total catches up moments later.
What the new pieces do
- Countersstore
- Aggregated like and view counts, updated asynchronously so a viral post’s flood of taps never slows posting or reading.
- Eventsbus
- A stream of posts and likes. Feed fan-out, counters and analytics each consume it independently and asynchronously.
Back of the envelope
- 1 viral post × millions/min
- one row = one hot key, the write bottleneck
- shard the counter into N parts
- each tap hits part = hash(user) % N, summed on read
- optimistic UI + async fold
- user sees +1 now; the true total converges in seconds
Step 6 · Here then gone
Stories & ephemerality
Stories vanish after 24 hours. Sweeping the database for expired posts and pulling them from every feed would be a constant, expensive chore.
Stories vanish after 24h. How do you make old ones disappear?
A sweep over billions of rows is a constant expensive chore, and stories linger until the job runs. Expiry should be automatic, not scheduled.
Lifetime becomes a property of the data — the store reclaims it for free and feeds just stop showing it. The same trick powers sessions and OTPs.
Soft-deletes pile up unbounded, bloat every query with a “not deleted” filter, and waste storage on data no one can ever see again.
Give each story a TTL so it auto-expires in storage and the cache; feeds simply stop showing it once it’s gone. No cleanup job — expiry is a property of the data, and the system reclaims it for free.
What the new pieces do
- Storiesservice
- Posts that vanish after 24h. A TTL on each story auto-expires it, so storage and feeds clean themselves up without a sweep.
Step 7 · Scale & the sharp edges
Shard everything
Billions of users and posts overflow any single database, and celebrity accounts create hotspots in both the feed fan-out and the counters.
Billions of users overflow one DB, and celebrities create hotspots. What’s the plan?
Replicas help reads, but every write still funnels to one primary that eventually overflows. Vertical scaling has a ceiling; sharding doesn’t.
Hash-sharding spreads the median load across many shards; the handful of extreme accounts get bespoke handling (pull not push, split counters) so no single shard or row is the bottleneck.
Geo-sharding skews badly — follows and virality cross regions freely, creating cross-shard reads and lopsided load. Hash-by-id spreads more evenly for a global social graph.
Shard the Metadata DB (by user/post id) behind the gateway and load-balance the stateless services. Handle celebrities with the hybrid feed (pull, not push) and hot-key counter sharding, so no single row or shard becomes the bottleneck.
You did it
You just designed Instagram.
Everything you assembled, in order
- Split media from metadata: pixels in a blob store, a pointer in the DB.
- A CDN serves immutable images from the edge, so media loads fast.
- Async image processing makes the many sizes without slowing uploads.
- A precomputed feed cache turns scrolling into a cheap lookup.
- Likes/views aggregated asynchronously via events and sharded counters.
- Stories expire automatically via TTLs — no cleanup job needed.
- Sharding plus hybrid feeds and hot-key handling absorb celebrity scale.