The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What does streaming take?
One title streamed to hundreds of millions of people, on every device and connection speed, starting in under two seconds and never buffering. That’s petabytes of video moving constantly — no single data center could push that much traffic to the world.
The trick: don’t send video from the middle. Pre-encode each title into many quality levels, push copies to caches near every viewer, and let the player pick the bitrate its connection can handle, segment by segment. The control path is tiny; the data path lives at the edge.
What the new pieces do
- Viewerclient
- Someone hitting play on a TV, phone or laptop. They expect video in under two seconds, no buffering, at the best quality their connection allows.
Step 1 · Press play
The play path
Tapping play has to do two very different things fast: confirm you’re allowed to watch this title here, and figure out where to get the bytes. Mixing authorization with byte delivery would make both slow.
Press play. Two jobs: check the viewer is allowed, and deliver gigabytes of video. How do you split it?
Auth is tiny and smart; byte-pushing is huge and dumb. Welding them together means scaling your auth logic every time traffic grows — and one slow stream can drag down sign-ins.
Control plane and data plane split cleanly — the API makes small smart decisions, the CDN moves the bytes. Each scales on its own terms.
Now thousands of dumb caches all need your subscriber DB, licensing rules and session logic. Edges should stay simple, cheap and replaceable.
The Playback API checks the Catalog (subscription, region, licensing), then a Manifest Service returns a descriptor pointing the player at the nearest CDN edge. The player then streams chunks straight from the edge — control and data are cleanly split.
Why this piece earns its place
The split moves the blast radius, it doesn’t remove it. Once bytes are leaving the edge, a stream already running barely depends on us — the player is holding a manifest and pulling immutable segments, so it plays on through a total Playback API outage. But nobody can start one. The play button becomes the availability floor for the whole product, and it sits on top of subscription state, region rules and licensing windows, which are databases, and databases get slow. So I’d design that path to degrade rather than fail: cache the authorization decision per device for a short window, and treat a Catalog timeout for a known-good subscriber as allow-and-log instead of deny, because refusing to play for a paying member is a worse outcome than briefly streaming to a lapsed one. The second consequence is that a manifest is a decision frozen at play time. It names edges, and a two-hour film outlives the health of any one of them, so it has to carry alternates and stay cheap enough that a player can re-fetch it mid-session without the API feeling a stampede.
What the new pieces do
- Playback APIbackend
- Checks the user can watch (subscription, region, licensing), then hands back a manifest pointing at the nearest CDN. It never streams bytes itself.
- Manifest Serviceservice
- Builds the playlist the player follows: the list of quality levels, segment URLs, audio tracks and subtitles for this title on this device.
- CDN Edgecache
- Serves the actual video chunks from a server physically close to the viewer. Almost all bytes come from here, never from central storage.
- Catalogstore
- Titles, seasons, episodes, artwork and licensing windows — the data the UI browses and the Playback API checks before authorizing a stream.
Step 2 · Where the bytes live
Origin & edge
Every viewer streaming from one central store would saturate it instantly and add cross-country latency to every chunk. Yet you still need one durable, authoritative copy of every title.
Hundreds of millions of viewers need the same files. Where do the bytes live?
It saturates instantly and adds cross-continent latency to every chunk. Central storage can be the source of truth — never the serving tier.
That’s petabytes times every region, mostly for titles nobody there watches. You only need the hot slice near viewers.
Origin holds everything once; edges cache what their viewers actually watch. Immutable segments mean the cache never goes stale.
Keep masters and all encodings in Origin Storage (cheap, durable blob storage), but serve viewers from CDN edges. On a cache miss the edge pulls a segment from origin once, then serves it to everyone nearby. Video segments are immutable, so they cache forever.
Why this piece earns its place
This step quietly assumes popularity is steep enough that the hot slice an edge can hold — roughly 5% of the catalog — really does cover almost everything watched near it. That holds for a launch title and for a big head-heavy catalog. It stops holding in the places that look least important: a region whose local-language shows are each small, a catalog grown to tens of thousands of titles, and a homepage deliberately pushing people off the head of the distribution. Then the request hit-rate barely moves while the byte hit-rate falls, and origin egress starts scaling with misses rather than with viewers. So I’d measure this in bytes, per edge, and read it per region rather than globally, because the average hides exactly the market that’s drifting. The defenses are unglamorous: size an edge against its own local tail instead of uniformly, and let a cold title’s first viewer pay a slightly slower start while the edge fills behind them — a much better trade than pulling a whole rendition in before playback is allowed to begin.
- ~95%+served from edge
- immutablesegments
- petabytesin origin
What the new pieces do
- Origin Storagestore
- The durable home of every title in every encoding. The CDN pulls from here on a cache miss; viewers essentially never touch it directly.
Back of the envelope
- 2h × 5 Mbps ≈ 4.5 GB
- one movie, at a single 1080p bitrate
- × ~10 rungs × codecs ≈ 50–100 GB
- what one title really costs in origin
- × tens of thousands of titles ⇒ PBs
- why origin is cheap blob storage, not SSDs
- edge holds ~the hot 5%
- popularity is skewed — cache only what’s watched
Step 3 · One stream, every connection
Adaptive bitrate
A phone on 3G and a TV on fiber can’t use the same file. A fixed quality either buffers constantly for the slow viewer or looks terrible for the fast one — and connections change mid-show.
A phone on flaky 3G and a TV on fiber both press play on the same film. What do you ship them?
It buffers for the phone and looks soft on the TV — and connections change mid-show, so even “good enough” isn’t stable.
The server can’t see last-mile conditions in real time, and holding per-viewer stream state for millions of sessions doesn’t scale.
The intelligence lives in the client. The server just offers a ladder; the player climbs and drops it as its own throughput changes.
Encode each title as an ABR ladder: the same content at many resolutions/bitrates, each sliced into short segments. The manifest lists them all; the player measures throughput and switches levels per segment, climbing in quality or dropping to avoid a stall.
Why this piece earns its place
The knob here isn’t really the ladder, it’s the buffer the player fills before it dares climb. With segments around four seconds, a switch lands about one segment later, so the player is always deciding on stale information — picking the rung for bytes it will show several seconds from now. Let the buffer run long and it reacts late, holding a rung through a dip it could have ridden out, and every minute of buffer a viewer abandons is bandwidth we paid for and nobody watched. Keep it short and one throughput wobble empties it and the stream stalls, which is the single thing this design exists to prevent. I’d set it asymmetrically: climb slowly, drop immediately, because a rebuffer costs far more than a minute at a lower rung. The neighbouring trap is oscillation. With rungs from 235 kbps to 16 Mbps, a connection parked between two of them can flap every segment, and viewers notice flapping more than they notice a steady, slightly softer picture — so a step up should require sustained headroom, not one good segment.
- 1title, many bitrates
- ~2–10sper segment
- per-segmentquality switch
Back of the envelope
- ladder: 235 kbps → 16 Mbps
- ~10 rungs, 240p up to 4K
- segments ≈ 4s
- a 2h film ≈ 1,800 segments per rung
- switch cost = 1 segment
- worst case, quality adapts within ~4 seconds
Step 4 · Make the ladder
The transcoding pipeline
Studios deliver one giant, pristine master. Devices need dozens of encodings (codecs, resolutions, bitrates) of it — and re-encoding a two-hour film serially would take ages per title.
The studio hands you one pristine 2-hour master. Encoding it into ~10 quality rungs serially would take days. What now?
Vertical scaling hits a wall fast, and you re-buy it for every new codec generation. The job is still fundamentally serial.
Segments are independent, so a movie becomes ~18,000 small jobs. Throw workers at it and days become minutes-to-hours.
You’d pay the same heavy compute again and again, and the first viewer of every title eats a giant delay. Do it once, offline.
A Transcode Pipeline splits the master into chunks and encodes each chunk at every rung of the ladder in parallel across a worker fleet, then writes the renditions to origin. Chunking turns a slow serial job into a massively parallel one.
Why this piece earns its place
Parallelism turns one long job into a join, and a join is governed by its slowest branch. A film isn’t publishable when the average chunk finishes; it’s publishable when all 18,000 jobs have, so the hour this buys is really the p99 job plus however long it takes to notice the handful that died. At that fan-out something always dies — a bad worker, a reclaimed spot instance, one chunk that trips an encoder bug — so jobs must be idempotent, keyed by title, chunk and rung, safe to re-run blind, with stragglers speculatively re-executed rather than waited on. The subtler cost is at the seams: each chunk is encoded without seeing its neighbours, so encoders make independent decisions at the boundaries. Every chunk at a rung has to stay interchangeable with the same chunk at every other rung, or the per-segment switching from the previous step shows up as a visible jolt. That means fixed boundaries aligned to one segment grid across the whole ladder, and a check that plays the assembled renditions and inspects the seams before a title is marked available — not a count of jobs that exited zero.
What the new pieces do
- Studio Ingestsource
- The pristine, enormous source file from the studio — one title, before it’s broken into the many encodings devices actually stream.
- Transcode Pipelineworker
- Chops the master into thousands of chunks and encodes each at many resolutions and bitrates, in parallel. The factory behind adaptive streaming.
Back of the envelope
- 1,800 chunks × 10 rungs = 18,000 jobs
- one film becomes a massively parallel batch
- ~2 min/job ÷ 600 workers ≈ 1 hour
- vs ~25 days if you encoded it serially
Step 5 · Closer than the cloud
Open Connect at the ISP
Even regional CDNs sit a few network hops away, and at streaming’s scale the bandwidth between ISPs and the CDN becomes a real cost and bottleneck — especially when everyone watches the new release at once.
A new season drops Friday 8pm and a whole country presses play at once. How do you survive it?
More edge servers don’t fix the bottleneck: the bytes still have to cross from your network into every ISP, and that transit link is the choke point.
Tomorrow’s demand is predictable, so fill caches sitting inside the ISP during off-peak hours. At 8pm it’s just a local file read, next door.
You’d survive by making the product worse on its biggest night. QoE is the thing you’re optimizing — degrading it is a last resort, not the plan.
Push popular content even closer: place cache appliances inside ISPs’ networks (Netflix calls this Open Connect) and pre-position likely-popular titles overnight, before demand hits. Most streams are then served from within the viewer’s own ISP.
Why this piece earns its place
Pre-positioning is a bet placed hours before the demand it serves, against a fixed amount of storage sitting in someone else’s building. So the real knob is how much of each appliance is reserved for tomorrow’s predicted hits versus the steady catalog people watch on an ordinary night. Push it too far toward the new release and you evict what a neighbourhood actually watches on a Tuesday — you win launch night and lose the other six days, with those misses crossing transit anyway. Leave it too small and the overnight fill never covers the surge, and 8pm crosses the ISP link exactly as it would have without an appliance there. The assumption underneath is that tomorrow is predictable, and usually it is: scheduled premieres, a known subscriber base, yesterday’s viewing. It breaks when something unplanned catches fire, which is precisely when it hurts. So the pull path has to stay a real, exercised path rather than a formality — an appliance must be able to fill during peak at low priority, and an unpredicted title has to be servable from a sibling appliance without a human deciding to push it.
Back of the envelope
- 1M concurrent × 5 Mbps = 5 Tbps
- one mid-size country on launch night
- global peak: tens of Tbps
- no central network could push this
- fill at 4am, serve at 8pm
- ISP links are idle off-peak — use them
Step 6 · What to watch next
Recommendations
With tens of thousands of titles, a viewer who has to search for something often just leaves. Discovery, not playback, is what keeps people subscribed.
200M+ members, tens of thousands of titles. Where do you compute each person’s homepage ranking?
Running heavy models per request for millions of concurrent users is brutally expensive and slow, and most of the signal barely changes hour to hour.
Precompute candidates and rankings in batch, then cheaply re-rank with fresh context (time of day, device) on load. The same candidate-then-rank pattern as any feed.
Discovery is the product. A shared list wastes the catalog and everything you know about each member.
A Recommendations system ranks titles per user from viewing history and similarity, building the personalized rows on the home screen. It’s computed largely offline and cached, then lightly personalized at request time — the same candidate-then-rank pattern as a feed.
Why this piece earns its place
Computing rankings largely offline assumes a member has a history and that it moves more slowly than the batch that reads it. Both assumptions break at the two moments that matter most. A brand-new member has no history at all, and their first session is the one that decides whether they stay — so cold start can’t be the same pipeline with an empty input; it needs regional and device popularity plus whatever they touch in the first few minutes, treated as exploration rather than as a ranking. The other break is the member who finished a series ten minutes ago: their state changed completely, and a row computed last night now describes someone else. That’s what the light request-time pass is really for, and it’s worth saying its job is recency, not polish. There’s also a feedback loop to name before it bites. People can only watch what the rows surface, and the rows learn from what was watched, so a title that’s never shown never earns the evidence that would get it shown. Left alone, it keeps re-recommending the head it already has proof for, and most of the catalog quietly stops existing. So the rows have to be pushed to explore — and it’s worth naming where that cost lands, because it isn’t here. Every viewer this page walks off the head is a viewer asking an edge for something it didn’t predict.
What the new pieces do
- Recommendationsservice
- Ranks titles per user from viewing history and similarity. Most of what people watch comes from these rows, so they’re central, not a side feature.
Step 7 · Measure everything
Playback events & QoE
You can’t see what users see. Is a region buffering? Did a CDN node degrade? Which titles are actually watched to the end? Without telemetry you’re flying blind on quality and on billing.
A CDN node in São Paulo starts silently degrading at 9pm. How do you find out?
By then you’ve burned hours of bad streams. Viewers don’t file tickets; they just leave.
Server-side checks miss what matters: a node can answer pings fine while serving real viewers slowly over a congested local path.
The player sees the truth. Aggregate QoE events per region and edge in near-real-time, and a degrading node lights up in minutes — then you route around it.
The player streams Playback Events — start time, rebuffers, bitrate switches, watch progress — onto an event pipeline. Those feed real-time QoE monitoring (route around a bad edge), view counting, and the recommendation models, closing the loop.
Why this piece earns its place
Routing on telemetry means the measurement shares a network with the thing it measures. The sessions having the worst time are the least able to report it: a player on a congested path delays or drops its events, and one that gives up and closes sends nothing at all — so a badly degrading edge can read as quiet rather than bad. That argues for watching starts alongside outcomes, since a region whose session count falls while its reported rebuffer rate stays flat is a stronger signal than either number alone. The other hazard is acting on noise. Sliced per edge and region, billions of daily events still thin to a handful of sessions for a small node in a quiet hour, and a reroute isn’t free — it moves viewers onto a further cache and warms it from cold. So I’d require a minimum session count before a node is allowed to fail, compare it against its own baseline rather than a global threshold because a rural ISP is permanently slower than a fibre metro, and shift traffic gradually and reversibly so a false positive costs a little quality instead of an outage.
What the new pieces do
- Playback Eventsbus
- A firehose of play, pause, seek, bitrate-switch and rebuffer events. Feeds quality monitoring, billing/views, and the recommendation models.
Back of the envelope
- 100s of events per session
- play, pause, seek, switch, rebuffer, heartbeat
- × 100M+ sessions/day ⇒ billions
- this is Kafka-scale stream processing
You did it
You just designed Netflix.
Everything you assembled, in order
- A play path that splits authorization (API) from byte delivery (CDN).
- Durable origin storage with immutable segments cached at the edge.
- An adaptive-bitrate ladder the player switches between per segment.
- A parallel, chunked transcoding pipeline that builds the ladder offline.
- Open Connect caches inside ISPs, pre-filled with predicted-popular titles.
- A recommendation system that drives discovery across a huge catalog.
- Playback/QoE events that monitor quality, count views and feed recs.