The whole design, in writing
Learn system design by building a live video streaming system like Twitch or YouTube Live step by step. An interactive guide covering ingest of a broadcaster stream, transcoding into an adaptive bitrate ladder, segmenting into HLS/DASH chunks, CDN fan-out to millions of viewers, the latency-vs-scale tradeoff, live chat, DVR/VOD, and the unhappy paths (thundering herd, low-latency modes, geo-distribution).
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What makes live streaming hard?
One broadcaster, millions of viewers, all watching the same moment within seconds of each other — on phones, TVs and laptops, on every kind of network, all over the planet. It's the opposite shape of a video call (few-to-few, ultra-low latency): this is one-to-millions, where scale matters more than shaving off the last second.
A live streaming system ingests one high-quality stream, transcodes it into a ladder of qualities, segments it into small HTTP chunks, and fans those out through a CDN to millions. The genius move: turn live video into ordinary cacheable files so the whole internet's caching infrastructure does the scaling for you.
What the new pieces do
- Broadcasterclient
- The streamer's encoder pushing one continuous high-quality video stream up to the platform in real time.
Step 1 · The shape of the problem
One-to-millions, not few-to-few
It's tempting to reach for WebRTC (video-call tech). But a call is a handful of people at ~150ms latency; a stream is one source to millions at a few seconds' latency. Why does that difference flip the whole design?
Why can't you just use video-call (WebRTC/SFU) tech for a million-viewer stream?
WebRTC encodes fine; the mismatch is scale and cost. Real-time peer connections to millions of viewers are wildly expensive and unnecessary, since viewers only consume (never send) and can tolerate a few seconds of delay.
It's the opposite — a broadcast tolerates seconds of latency, while a call needs sub-150ms. That tolerance is exactly what lets streaming trade a little delay for massive, cheap scale via HTTP/CDN.
Because viewers don't send media and can accept a few seconds of delay, you don't need per-viewer real-time connections. Instead you turn video into cacheable HTTP segments a CDN fans out to millions — far cheaper and more scalable than an SFU mesh.
A stream is one-to-millions: viewers only consume and can tolerate a few seconds of delay. That tolerance is a gift — it lets you avoid per-viewer real-time connections and instead turn video into cacheable HTTP segments that a CDN fans out to millions cheaply. You trade a few seconds of latency for essentially unlimited, low-cost scale.
Why this piece earns its place
The phrase viewers only consume is doing a lot of work here, and it describes a role, not a person. The day the product adds a guest slot, a call-in or a co-stream, someone in that audience starts sending media, and the instinct is to carve them a shorter delivery path — a special real-time route down to one seat. That is the wrong shape: it rebuilds the expensive per-viewer architecture you just rejected, inside the cheap one, for a single viewer. The clean answer is to move them across the line instead. A participant is a broadcaster: they get an ingest connection, their media is mixed into the source before anything downstream sees it, and the millions still watching stay exactly where they were, pulling segments. Keep the asymmetry — a few pushers, many pullers — and promote people between the two sides rather than blurring the boundary, because the moment a viewer can be served specially, the economics that made this design work stop being true.
Step 2 · Get the stream in
Ingest
First, the broadcaster's single high-quality stream has to reach you reliably and continuously. What receives it, and how?
Run an ingest endpoint that accepts the broadcaster's push over a real-time protocol — RTMP (classic), SRT (resilient over lossy links), or WebRTC (lowest latency). It authenticates the stream key, handles reconnects, and forwards the raw stream into the processing pipeline. Ingest is per-broadcaster (there are far fewer broadcasters than viewers), so it's the cheap side of the system.
Why this piece earns its place
Everything downstream is fed by one live connection from one encoder, which makes ingest the only place in this design where a single failure takes the whole broadcast off the air — the CDN has hundreds of edges and the ladder has several renditions, but the broadcaster has one socket. So treat a reconnect as a first-class event rather than an error path. Encoders retry, and many can be pointed at a backup ingest URL, which means the stream key has to resolve to a broadcast session that a different ingest node can adopt, not to whichever box happened to answer first. The subtler half is downstream: a reconnected encoder restarts its own timestamps, and if the pipeline reads that as a brand-new broadcast it starts a fresh output and every player in the world is left waiting on one that stopped advancing. Carry the session across the gap and signal the break as a discontinuity, so a dropped uplink costs viewers a stutter instead of the stream.
What the new pieces do
- Ingestgateway
- Receives the broadcaster's stream (RTMP/SRT/WebRTC), authenticates it, and hands it to the processing pipeline.
Step 3 · One stream, every device
Transcoding into a bitrate ladder
The broadcaster sends one quality (say 1080p at 6 Mbps). But viewers are on 4K TVs and on phones with 1 Mbps. Send everyone 1080p and the phone buffers forever; send everyone low-res and the TV looks awful. How do you serve all of them?
One incoming quality, viewers with wildly different devices/bandwidth. How do you serve all?
The transcoder re-encodes the incoming stream into several resolution/bitrate versions. The player then does adaptive bitrate — measuring its bandwidth and switching to the rendition that plays smoothly. One upload becomes many qualities, one per network condition.
Broadcasters have limited upload bandwidth (like the mesh problem) and can't send many renditions at once. The platform transcodes centrally so the broadcaster uploads just one stream.
A phone on 1 Mbps can't "cope" with a 6 Mbps stream — it buffers endlessly. You must offer lower renditions so each viewer streams a bitrate their network can sustain.
Transcode the source into an adaptive bitrate ladder — several renditions at different resolutions/bitrates (e.g. 1080p/720p/480p/360p, plus audio-only). The viewer's player does ABR: it measures its bandwidth and switches to the highest rendition that plays without buffering, up or down as conditions change. One broadcaster upload becomes a menu of qualities, so every device and network gets a smooth stream.
Why this piece earns its place
The shape of the ladder is the knob, and it is the one cost here that never amortizes. An edge serves the same cached segment to everyone behind it, but transcoding is paid per broadcaster per rung whether a thousand people watch that rung or nobody does. Extra rungs also split the audience: each rendition is its own set of segment URLs, so an edge fills every rung separately, and a rung with two viewers on it misses almost every time it is asked for. Cut too far the other way and the gaps get wide — a player whose bandwidth falls just under 720p has nowhere to land but 480p, so it surrenders quality it could have sustained, and if that drop overshoots it oscillates instead of settling. I would size the ladder from what the audience actually plays rather than from a tidy set of resolutions: pass the source through untouched so the top rung costs nothing to produce, keep a bottom rung cheap enough for a phone on 1 Mbps, and spend encoder time in between only where the viewer histogram is dense.
- 1 inseveral renditions out
- ABRplayer picks best rung
- per broadcastertranscode cost
What the new pieces do
- Transcoderservice
- Re-encodes the source into multiple resolutions/bitrates (1080p/720p/480p/360p) so any device and network can play.
Step 4 · Make it cacheable
Segment into HTTP chunks
Now the crucial trick. A continuous live stream can't be cached by a CDN — but the CDN is exactly what you need to reach millions. How do you make live video look like something a CDN can serve?
A CDN caches files, but live video is a continuous stream. How do you bridge that?
CDNs aren't built to fan out millions of live persistent streams, and that reintroduces per-viewer cost. The trick is to chop the stream into small files the CDN can cache and serve like any static asset.
Millions connecting to one origin is the exact overload a CDN exists to prevent. You must make the content cacheable so the CDN absorbs the crowd.
The packager splits each rendition into small numbered segments and writes a manifest (playlist) listing them. Viewers fetch segments over plain HTTP and play them back-to-back; new segments append to the manifest as the stream continues. Live video becomes a rolling series of cacheable files — perfect for a CDN.
Segment each rendition into short chunks (2–6s) and write a manifest (playlist) that lists them — this is HLS or DASH. The player fetches the manifest, then downloads segments over plain HTTP and plays them back-to-back; as the stream continues, new segments are appended and old ones age out. Live video is now a rolling set of ordinary cacheable HTTP files — exactly what a CDN was built to serve.
Why this piece earns its place
Segmenting produces two kinds of object with opposite caching behaviour, and the manifest is the one that bites. A segment is immutable the instant it is written: number it, give it a long TTL, and an edge can hold it indefinitely. The manifest is rewritten every time a segment is published, and every player re-fetches it on roughly that same cycle purely to learn the next filename. So the small text file, not the video, becomes the most requested object in the system, and its cache lifetime is a trap in both directions — hold it longer than a segment and players sit at a live edge that has stopped advancing, hold it for nothing and every poll from every viewer lands on origin. The way through is to keep it cacheable but brief so the CDN can coalesce the identical requests arriving inside that window, and, where the protocol allows, let the player hold the request open until the next segment exists, so one waiting connection replaces a loop of hopeful polls.
- 2–6ssegment length
- manifestplaylist of segments
- HTTPcacheable by any CDN
What the new pieces do
- Packagerservice
- Splits each rendition into small segments (2–6s) and writes a manifest, turning live video into cacheable HTTP files.
- Origindata
- Holds the latest segments and manifests. The CDN pulls from here; tiered caching shields it from the crowd.
Step 5 · Reach the millions
CDN fan-out
Segments sit on your origin. If a million viewers all pulled them from origin, it would melt. How does one broadcaster's stream reach a million screens without crushing your servers?
Put a CDN in front of the origin. Viewers fetch segments from a nearby edge; the first request for a segment at an edge pulls it from origin (a miss), and the next thousands of viewers on that edge are served the cached copy (hits). One broadcaster's stream is fanned out by hundreds of edges to millions of viewers, while the origin only serves a handful of cache-fills per segment. Tiered/shield caches further protect the origin from the initial fan-in.
Why this piece earns its place
One fill per segment per edge rests on an assumption worth saying out loud: that enough viewers behind the same edge want the same segment during the few seconds it is the newest one. For a big broadcast that is exactly true, and it is why this is cheap. It stops being true in the long tail. A platform is not one enormous stream, it is thousands of small ones, and a channel with a handful of viewers scattered across the planet gives every edge a fan-out of one — each request a miss, each miss an origin fetch, and the CDN has become a proxy that charges you and adds a hop. So routing has to depend on how big an audience is, not only on who is nearest: collapse small streams onto far fewer PoPs so their viewers actually land together, and let the large events spread as wide as the map allows. The same thinning happens inside one stream once viewers rewind, which is why a DVR window never caches as well as the live edge does.
- 1 fillper segment per edge
- edgesserve the millions
- tieredshield the origin
What the new pieces do
- CDN Edgecache
- Hundreds of edges caching segments near viewers, so one broadcaster can be served to millions cheaply.
- Viewersclient
- Millions of players fetching segments over HTTP, switching rendition up/down based on their measured bandwidth.
Step 6 · How live is "live"?
The latency dial
Standard HLS with 6s segments and buffering means viewers are often 15–30 seconds behind real time — fine for a movie premiere, painful for interactive streams where chat reacts to plays the streamer already made. How do you cut the delay?
Trade efficiency for latency deliberately. Shorter segments and smaller player buffers cut delay but add request overhead and rebuffer risk. Low-Latency HLS/DASH (chunked-transfer partial segments) gets you to ~2–5s while keeping CDN scalability. For true sub-second interactivity (auctions, betting, some Twitch modes) fall back to WebRTC-based delivery — at higher cost and less scale. Pick the point on the dial the product needs; you can even offer different latencies to different viewers.
Why this piece earns its place
Be precise about who pays when you turn this dial down, because it is not the platform. Most of that delay is the player’s buffer, and the buffer is the only thing absorbing a jittery network — those seconds are what let a phone on a train miss a segment, retry, and show the viewer nothing unusual. Spend the buffer and the median viewer notices nothing, because they have bandwidth to spare, while the worst-connected slice of the audience converts your improvement straight into rebuffering, which is a far worse experience than being behind. So I would set this against the rebuffer rate of the weakest decile rather than the average, and treat a move from standard HLS to a low-latency mode as a rollout to be measured, not a switch to be flipped globally. And because the right number is a product decision rather than a network fact, it is entirely reasonable to hand a tight profile to the viewers who need it and a deeper buffer to the ones who are struggling.
- ~15–30sstandard HLS
- ~2–5slow-latency HLS
- <1sWebRTC (costly)
Step 7 · The other half of "live"
Chat, reactions & counts
A live stream isn't just video — it's the real-time interaction around it: chat scrolling, reactions, viewer counts, the streamer responding. That's a different, genuinely low-latency, fan-out problem running alongside the video.
Run live chat as a separate real-time system: viewers connect over WebSockets to chat servers, and messages fan out to everyone in the channel (the same fan-out challenge as a chat app, at broadcast scale — often sharded by channel with a pub/sub backbone). Show a live viewer count (an approximate distributed counter). Keep chat and video as independent pipelines — chat is small and must be snappy; video is huge and CDN-served — so each scales on its own.
Why this piece earns its place
Decoupling the pipelines is right, and it creates one problem that lives entirely in the seam: chat arrives in real time while video arrives late, so the chat pane spoils the stream it sits beside. On standard HLS that is 15–30s of people reacting to a play the viewer has not been shown yet, and it is worse than it sounds, because two viewers on different renditions with different buffers are not even behind by the same amount. A single global delay on the chat service cannot fix that. The fix belongs on the client: stamp each message with the broadcast position it was sent at rather than with wall-clock time, and have the player withhold it until its own playback reaches that position. Every viewer then reads chat in step with the video they are actually watching, at the cost of a pane that is no longer strictly live — and it makes explicit that the streamer, who sits nearest the front, is answering questions about a moment the audience is only now reaching.
What the new pieces do
- Live Chatclient
- The parallel real-time layer — chat, reactions, viewer counts — fanned out over WebSockets alongside the video.
Step 8 · The sharp edges
Herds, DVR, geo & failure
Real broadcasts bring spikes and demands: a huge event starts and everyone hits play at once (a thundering herd), viewers want to rewind (DVR) or watch the replay (VOD), and the audience is global.
Absorb the herd with CDN tiered caching and request coalescing (many edge misses for the first segment collapse into one origin fetch). Offer DVR by keeping a rolling window of past segments addressable, and produce VOD by retaining and re-packaging the segments after the stream ends. Go geo-distributed (regional ingest + CDN PoPs) so both broadcaster and viewers hit nearby infrastructure. Handle broadcaster disconnects gracefully (buffer/reconnect), and monitor rebuffer ratio and startup time as your core quality metrics.
Why this piece earns its place
Worth flagging what those two quality metrics imply, because rebuffer ratio and startup time are the only numbers here that describe the actual product, and no server can see either one. Origin and edge logs show segments handed over successfully, and a 200 on a segment reads exactly the same whether the player drew that frame on time or the viewer spent those seconds staring at a spinner. The rendition sits in the URL, so the logs can tell you which rung a player is on — what they can never tell you is why it chose that rung, or what the choice cost the person watching. Every failure that matters — the stall, the slow first frame, the player that gave up before the first frame ever arrived — happens after the last byte your infrastructure knows about. So playback quality has to be reported by the player, which makes the telemetry path its own fan-in from millions of clients back toward you, and therefore something to sample rather than collect whole. Sample it with dimensions attached — edge, region, rendition, device — because the aggregate is close to useless: a broadcast reads as healthy on average while one PoP is broken for everyone behind it, and dimensions are the only way that ever becomes visible.
You did it
You just designed a live streaming system.
Everything you assembled, in order
- Live streaming is one-to-millions, latency-tolerant — optimize for cheap CDN fan-out, not sub-150ms.
- Ingest accepts one broadcaster push (RTMP/SRT/WebRTC) — the cheap, few-sources side.
- Transcode the source into an adaptive bitrate ladder so every device/network gets a smooth stream.
- Segment each rendition into short HTTP chunks + a manifest (HLS/DASH) — video becomes cacheable files.
- A CDN fans those segments out to millions; the origin serves only ~one fill per segment per edge.
- The latency dial (standard HLS ~15-30s → LL-HLS ~2-5s → WebRTC <1s) trades delay vs scale vs cost.
- Live chat is a separate WebSocket fan-out pipeline; DVR/VOD reuse the same segments.