The big idea
What makes live streaming hard?
One broadcaster, millions of viewers, all watching the same moment within seconds of each other — on phones, TVs and laptops, on every kind of network, all over the planet. It's the opposite shape of a video call (few-to-few, ultra-low latency): this is one-to-millions, where scale matters more than shaving off the last second.
A live streaming system ingests one high-quality stream, transcodes it into a ladder of qualities, segments it into small HTTP chunks, and fans those out through a CDN to millions. The genius move: turn live video into ordinary cacheable files so the whole internet's caching infrastructure does the scaling for you.
How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the pipeline grow, and at the end drop a CDN edge and kill transcoding to see what scales the crowd and what serves every device. Hit Begin.
Step 1 · The shape of the problem
One-to-millions, not few-to-few
It's tempting to reach for WebRTC (video-call tech). But a call is a handful of people at ~150ms latency; a stream is one source to millions at a few seconds' latency. Why does that difference flip the whole design?
Design decision: Why can't you just use video-call (WebRTC/SFU) tech for a million-viewer stream?
The call: Viewers only consume and tolerate a few seconds' delay, so you optimize for cheap massive fan-out (HTTP/CDN), not ultra-low latency. — Because viewers don't send media and can accept a few seconds of delay, you don't need per-viewer real-time connections. Instead you turn video into cacheable HTTP segments a CDN fans out to millions — far cheaper and more scalable than an SFU mesh.
A stream is one-to-millions: viewers only consume and can tolerate a few seconds of delay. That tolerance is a gift — it lets you avoid per-viewer real-time connections and instead turn video into cacheable HTTP segments that a CDN fans out to millions cheaply. You trade a few seconds of latency for essentially unlimited, low-cost scale.
Latency vs scale: The core trade of live video: WebRTC gives ~150ms to a few people at high cost; HTTP segment streaming gives a few seconds to millions at low cost. Twitch/YouTube Live pick scale. Low-latency HLS narrows the gap but the philosophy stands: lean on HTTP caching.
Step 2 · Get the stream in
Ingest
First, the broadcaster's single high-quality stream has to reach you reliably and continuously. What receives it, and how?
Run an ingest endpoint that accepts the broadcaster's push over a real-time protocol — RTMP (classic), SRT (resilient over lossy links), or WebRTC (lowest latency). It authenticates the stream key, handles reconnects, and forwards the raw stream into the processing pipeline. Ingest is per-broadcaster (there are far fewer broadcasters than viewers), so it's the cheap side of the system.
Push in, pull out: Media flows in two very different ways: broadcasters push one stream to ingest (few sources, real-time protocol), viewers pull HTTP segments (millions of sinks, cacheable). Recognizing this asymmetry — few pushers, many pullers — is what lets each side use the right, cheap technology.
Step 3 · One stream, every device
Transcoding into a bitrate ladder
The broadcaster sends one quality (say 1080p at 6 Mbps). But viewers are on 4K TVs and on phones with 1 Mbps. Send everyone 1080p and the phone buffers forever; send everyone low-res and the TV looks awful. How do you serve all of them?
Design decision: One incoming quality, viewers with wildly different devices/bandwidth. How do you serve all?
The call: Transcode the source into a ladder of renditions (1080p/720p/480p/360p) so players can pick one that fits. — The transcoder re-encodes the incoming stream into several resolution/bitrate versions. The player then does adaptive bitrate — measuring its bandwidth and switching to the rendition that plays smoothly. One upload becomes many qualities, one per network condition.
Transcode the source into an adaptive bitrate ladder — several renditions at different resolutions/bitrates (e.g. 1080p/720p/480p/360p, plus audio-only). The viewer's player does ABR: it measures its bandwidth and switches to the highest rendition that plays without buffering, up or down as conditions change. One broadcaster upload becomes a menu of qualities, so every device and network gets a smooth stream.
The bitrate ladder + ABR: Transcoding (GPU-accelerated, per broadcaster) produces the ladder; the client picks a rung. Because the player adapts continuously, a viewer whose wifi dips drops to 480p and keeps watching instead of freezing. This is the same simulcast idea as video calls, but produced centrally by the platform.
Step 4 · Make it cacheable
Segment into HTTP chunks
Now the crucial trick. A continuous live stream can't be cached by a CDN — but the CDN is exactly what you need to reach millions. How do you make live video look like something a CDN can serve?
Design decision: A CDN caches files, but live video is a continuous stream. How do you bridge that?
The call: Chop each rendition into short segments (2–6s) + a manifest — HLS/DASH — so they're just cacheable HTTP files. — The packager splits each rendition into small numbered segments and writes a manifest (playlist) listing them. Viewers fetch segments over plain HTTP and play them back-to-back; new segments append to the manifest as the stream continues. Live video becomes a rolling series of cacheable files — perfect for a CDN.
Segment each rendition into short chunks (2–6s) and write a manifest (playlist) that lists them — this is HLS or DASH. The player fetches the manifest, then downloads segments over plain HTTP and plays them back-to-back; as the stream continues, new segments are appended and old ones age out. Live video is now a rolling set of ordinary cacheable HTTP files — exactly what a CDN was built to serve.
HLS/DASH: video as files: The killer insight: by turning a live stream into numbered HTTP segments + a manifest, you make it indistinguishable (to the CDN) from static files. Now the entire global HTTP caching infrastructure — every CDN edge — can fan out your broadcast. Segment length trades latency (shorter = lower delay) against overhead/efficiency.
Step 5 · Reach the millions
CDN fan-out
Segments sit on your origin. If a million viewers all pulled them from origin, it would melt. How does one broadcaster's stream reach a million screens without crushing your servers?
Put a CDN in front of the origin. Viewers fetch segments from a nearby edge; the first request for a segment at an edge pulls it from origin (a miss), and the next thousands of viewers on that edge are served the cached copy (hits). One broadcaster's stream is fanned out by hundreds of edges to millions of viewers, while the origin only serves a handful of cache-fills per segment. Tiered/shield caches further protect the origin from the initial fan-in.
The CDN does the scaling: This is why segmenting mattered: because segments are cacheable, the CDN's existing fan-out (edge caches, tiered shields, anycast routing) handles the millions for you. The origin serves ~one fill per segment per edge; the edges serve everyone else. Live streaming is, at its heart, a clever application of a CDN.
Step 6 · How live is "live"?
The latency dial
Standard HLS with 6s segments and buffering means viewers are often 15–30 seconds behind real time — fine for a movie premiere, painful for interactive streams where chat reacts to plays the streamer already made. How do you cut the delay?
Trade efficiency for latency deliberately. Shorter segments and smaller player buffers cut delay but add request overhead and rebuffer risk. Low-Latency HLS/DASH (chunked-transfer partial segments) gets you to ~2–5s while keeping CDN scalability. For true sub-second interactivity (auctions, betting, some Twitch modes) fall back to WebRTC-based delivery — at higher cost and less scale. Pick the point on the dial the product needs; you can even offer different latencies to different viewers.
Latency vs scale vs cost: There's no free lunch: standard HLS = cheap, scalable, ~15–30s; LL-HLS = ~2–5s with more complexity; WebRTC = sub-second but expensive and harder to scale. The right choice is per use case, and it's the single biggest architectural knob in live streaming.
Step 7 · The other half of "live"
Chat, reactions & counts
A live stream isn't just video — it's the real-time interaction around it: chat scrolling, reactions, viewer counts, the streamer responding. That's a different, genuinely low-latency, fan-out problem running alongside the video.
Run live chat as a separate real-time system: viewers connect over WebSockets to chat servers, and messages fan out to everyone in the channel (the same fan-out challenge as a chat app, at broadcast scale — often sharded by channel with a pub/sub backbone). Show a live viewer count (an approximate distributed counter). Keep chat and video as independent pipelines — chat is small and must be snappy; video is huge and CDN-served — so each scales on its own.
Two pipelines, one experience: Video (huge, cacheable, seconds-latency, CDN) and chat/interaction (small, real-time, WebSocket fan-out) are separate systems stitched together in the UI. Coupling them would drag chat down to video's latency or video up to chat's cost. Keep them apart.
Step 8 · The sharp edges
Herds, DVR, geo & failure
Real broadcasts bring spikes and demands: a huge event starts and everyone hits play at once (a thundering herd), viewers want to rewind (DVR) or watch the replay (VOD), and the audience is global.
Absorb the herd with CDN tiered caching and request coalescing (many edge misses for the first segment collapse into one origin fetch). Offer DVR by keeping a rolling window of past segments addressable, and produce VOD by retaining and re-packaging the segments after the stream ends. Go geo-distributed (regional ingest + CDN PoPs) so both broadcaster and viewers hit nearby infrastructure. Handle broadcaster disconnects gracefully (buffer/reconnect), and monitor rebuffer ratio and startup time as your core quality metrics.
Design for the unhappy path: Everyone-at-once → CDN coalescing shields origin. Rewind → keep recent segments. Replay → segments become VOD for free. Global → regional ingest + edges. Since the pipeline already produces cacheable segments, DVR and VOD nearly fall out of it — reuse the same artifacts.
You did it
You just designed a live streaming system.
- Live streaming is one-to-millions, latency-tolerant — optimize for cheap CDN fan-out, not sub-150ms.
- Ingest accepts one broadcaster push (RTMP/SRT/WebRTC) — the cheap, few-sources side.
- Transcode the source into an adaptive bitrate ladder so every device/network gets a smooth stream.
- Segment each rendition into short HTTP chunks + a manifest (HLS/DASH) — video becomes cacheable files.
- A CDN fans those segments out to millions; the origin serves only ~one fill per segment per edge.
- The latency dial (standard HLS ~15-30s → LL-HLS ~2-5s → WebRTC <1s) trades delay vs scale vs cost.
- Live chat is a separate WebSocket fan-out pipeline; DVR/VOD reuse the same segments.