The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
How does YouTube stream to a billion screens?
A creator uploads one enormous file. Billions of viewers — on fibre, on 3G, on a TV — must each watch it starting in under a second, in the best quality their connection can handle, without buffering.
The trick is that you never serve "the video." You serve a metadata lookup, then a flood of tiny pre-encoded segments from a server near the viewer. Almost everything here is about turning one raw upload into something a CDN can spray across the planet.
What the new pieces do
- Upload UIcreator
- A creator uploading a huge raw file. Uploads once and wants it watchable everywhere, in every quality.
- Playerviewer
- Anyone hitting play. Expects the video to start in under a second and never buffer, on any connection.
Step 1 · The skeleton
Split the bytes from the facts
A video is two very different things: a few hundred bytes of metadata (title, owner, status) and gigabytes of raw media. Storing them the same way is a mistake.
A video is a few hundred bytes of metadata plus gigabytes of raw media. How do you store them?
Gigabyte media blobs bloat a database, make queries slow, and waste its expensive indexed storage. Databases are for small structured queryable data, not opaque write-once files.
Object storage is great for the giant media but terrible for the queryable facts (title, owner, status, search) you constantly look up. You need a database for those.
Tiny structured facts go in a queryable database; huge write-once media goes in cheap object storage. Watch requests read metadata first, then fetch bytes — splitting them is the foundation.
Keep facts in a Metadata DB and bytes in a Blob Store (object storage like S3). The Upload Service streams the raw file into the blob store and writes a small row in the metadata DB. Watch requests read metadata first, then go get bytes.
Why this piece earns its place
Splitting the stores means one upload is now two writes to two systems with no transaction across them, and both orderings fail. Write the row first and a failed blob upload leaves a video the catalog swears exists with nothing to play. Write the bytes first and a failed row leaves gigabytes nobody will ever reference and nobody will ever be billed for. So I make the metadata row the commit point — bytes land in the blob store first, under a key derived from the video id, and the row only becomes real once they are there — and treat status as a state machine the pipeline advances, not a label. Orphaned blobs then become a known, tolerable category: a sweeper walks keys against rows and reclaims whatever the catalog never claimed. The subtler cost is that reading a video is now two hops in two failure domains. The metadata DB falling over does not merely break search, it breaks playback, because the player cannot discover which renditions exist. That is what makes metadata worth caching hard later, on a path the CDN can never help with: the edge caches the bytes, not the row that says which bytes exist.
What the new pieces do
- App Serveredge
- The front door for uploads and watch requests. Becomes an API Gateway behind a load balancer.
- Upload Serviceservice
- Accepts the raw upload (resumable, in chunks), stashes it in blob storage, and queues it for processing.
- Metadata DBstore
- Small structured data: title, description, owner, status, and the manifest of available renditions.
- Blob Storestore
- Object storage (S3-like) holding the giant raw upload and every encoded segment. The origin behind the CDN.
Step 2 · The pipeline
One file becomes many
The raw upload is useless for streaming: it’s one giant file in one codec at one resolution. A phone on 3G and a 4K TV need wildly different versions.
The raw upload is one giant file in one codec. A 3G phone and a 4K TV need different versions. How do you produce them?
Re-encoding per view is enormously expensive and slow — you’d redo the same work for every viewer. Encode once, up front, into all qualities and reuse.
The upload service drops a job on a queue; a transcoder fleet re-encodes into 240p–4K, each chopped into small ~2–10s segments written back to blob, then marks the video ready. The player switches quality mid-stream from the precomputed ladder (HLS/DASH).
Creators upload one master file and can’t be expected to produce a full ABR ladder of segmented renditions. Encoding is the platform’s job, done asynchronously off the upload path.
Make encoding asynchronous. The Upload Service drops a job on a queue; a fleet of Transcoder workers re-encode the file into an ABR ladder — 240p, 480p, 720p, 1080p, 4K — each chopped into small (~2–10s) segments, then written back to the blob store. When done, they mark the video ready in metadata.
Why this piece earns its place
The rung count is the real knob this step introduces, and it is set against two bills at once. Every rung is another full encode of the whole file and another copy in storage — the 5–6 rungs here are already about 6× the bytes — and you pay both for every upload, including the overwhelming majority nobody finishes watching. Cut rungs to save that and the player’s step-down becomes a cliff: with wide gaps between qualities, a viewer whose bandwidth dips slightly has nothing close to fall to and drops much further than the connection warranted, which the viewer sees as an ugly lurch in sharpness. Pack them too close and the player flips between neighbouring rungs on noise. I would set the ladder from the devices and networks actually in the logs rather than from symmetry, and I would decouple the choice from ingest: encode one middle rung immediately so the video becomes watchable, then let workers fill the rest behind it. The same reasoning kills the automatic top rung — building 4K for a video shot at 720p is pure cost.
- 5–6quality rungs
- ~4sper segment
- asyncvia queue
What the new pieces do
- Transcoderworker
- Workers that re-encode the raw file into many resolutions and bitrates, split into small streamable segments.
- Encode Jobsbus
- A job queue decoupling fast uploads from slow encoding. Transcoder workers pull jobs at their own pace.
Back of the envelope
- 1 upload → 5–6 rungs × segments
- ~6× storage per video
- ~2–10s per segment
- switchable mid-stream (HLS/DASH)
- encode async via queue
- slow encoding off the fast upload path
Step 3 · The hot path
Serving the watch
Views dwarf uploads by orders of magnitude, and viewers are everywhere on Earth. Streaming every segment from one origin datacenter would be slow and ruinously expensive.
Views dwarf uploads and viewers are everywhere. Streaming every segment from one origin would be slow and ruinous. Fix?
More origin capacity in one place still pays a long round-trip to distant viewers and concentrates bandwidth cost. You need bytes physically close to users.
Pre-downloading gigabytes wastes bandwidth (most of a video is never watched) and delays start. You want to stream small segments on demand from a nearby edge.
The player pulls the manifest then segments from the nearest edge; the first regional viewer warms the cache, everyone after gets bytes from down the street. Immutable segments cache perfectly — the CDN offloads 95%+ of traffic. The CDN IS the read-scaling strategy.
Put a CDN in front of the blob store. To watch, the player fetches the manifest (from metadata) then pulls segments from the nearest CDN edge. The first viewer in a region warms the cache; everyone after gets bytes from down the street. Origin traffic stays tiny.
Why this piece earns its place
The line about the first viewer in a region warming the cache quietly assumes viewers arrive one at a time. They do not, in the case that matters most: a large channel publishes, or a premiere starts, and the video is cold at every edge on Earth in the same second. All those edges miss on the same opening segments at once and each goes to origin for them, so peak demand is the exact moment the CDN offloads least — and origin was sized on the promise that it would barely be touched. I would want two defenses. Inside an edge, coalesce concurrent misses for one object so a single request goes upstream and the rest wait on its result. Above the edges, run a shield or mid-tier cache in front of the blob store, so the herd collapses at a regional layer instead of at origin. Anything scheduled gets a third: push the opening segments to edges before the premiere rather than discovering they are missing at start time. None of this touches the steady state, where segments genuinely do cache forever — it protects the minute before they have.
- 95%+served at edge
- <1sstart time
- immutablesegments
What the new pieces do
- CDN Edgeedge
- Thousands of edge servers near users. Serves the actual video bytes so the origin barely gets touched.
Back of the envelope
- 95%+ served at the edge
- origin barely touched
- immutable segments
- cache forever, no invalidation
- 1st regional view warms cache
- the rest served locally, <1s start
Step 4 · A mountain of bytes
Storing exabytes affordably
Every upload now becomes 5–6 renditions, each in many segments — multiplying storage. Hundreds of hours are uploaded every minute. You cannot keep it all hot.
Every upload becomes 5–6 renditions in many segments, and 500 hours arrive every minute. How do you store exabytes affordably?
A tiny fraction of videos drive most views — keep those replicated and ready; let the millions of near-zero-view uploads sit in cheap cold tiers, accepting a slower first byte if someone finally watches them.
Replicating everything everywhere is ruinously expensive when most videos are barely watched. Match storage cost to popularity — hot content hot, cold content cheap.
Deleting creators’ uploads to save space is unacceptable — they expect permanence. The answer is cheaper cold storage for the long tail, not deletion.
Lean on tiered blob storage: replicate popular content across regions for fast origin pulls, and push the long tail of rarely-watched videos to cheaper cold storage. Generate thumbnails and previews once, store them as blobs too, and serve them from the CDN.
Why this piece earns its place
Tiering bakes in an assumption worth saying out loud: that I know which videos are hot, and I only ever know that in the past tense. A tier is assigned from watch history, so the case that hurts is a years-old video a huge account links today — its first bytes come out of cold storage precisely when the demand arrives, and cold storage charges both a retrieval fee and a wait. The mirror failure is thrash: a video hovering at the boundary gets promoted and demoted over and over, and the transitions cost more than the tier ever saved. So I move the two directions at different speeds — promote on the first sign of traffic, demote only after a long quiet — and I keep the decision per rendition rather than per video, because the lowest rung of everything is small enough to leave hot everywhere. A cold video then starts instantly, if fuzzily, while its higher rungs restore behind it, which converts a cold-storage stall into a quality dip. Thumbnails never go cold at all: the catalog renders them, so they belong with the metadata, not with the tail.
- 500 hrsuploaded / min
- 6×storage per video
- tieredhot + cold
Back of the envelope
- 500 hrs uploaded / minute
- × ~6 renditions = exabytes
- hot: replicated + ready
- the tiny fraction driving most views
- cold: cheap, slower first byte
- the long tail nobody watches
Step 5 · Don’t fall over
Scale the control plane
The CDN handles the bytes, but metadata reads, search, and upload coordination still hit your services — and one big server is a single point of failure.
The CDN handles bytes, but metadata reads, search and upload coordination still hit your services, and one server is a SPOF. How do you scale them?
A bigger box has a ceiling and is still a single point of failure, and it can’t isolate an upload spike from the read-heavy watch path. You need many instances and separation of concerns.
Coupling uploads/encoding (write) with watch-metadata/search (read) means a surge in one takes down the other. Their traffic shapes are totally different — isolate them.
Run many stateless instances behind a load balancer with a hot-metadata cache, and separate upload, watch-metadata and search so each scales — and fails — independently, keeping the 99% read path resilient when writes surge.
Put a load balancer in front and run many stateless copies of each service. Add a read cache for hot metadata. Split upload, watch-metadata, and search into services that scale — and fail — independently, so a spike in uploads never takes down playback.
Why this piece earns its place
Stateless is a property I have to keep, not one I get by writing it on a box, and uploads are where it breaks first. A resumable upload is a conversation: chunks arrive over minutes against a session, and if an instance holds which chunks it has in local memory, the load balancer handing chunk forty to a different copy loses the thread. Sticky routing looks like the cheap fix and simply reinvents the single point of failure, one per session — the instance dies mid-upload and the creator starts the file again. So the session state (which chunks landed, their checksums, the blob key being assembled) lives in a shared store, and any instance can continue any upload. Splitting the planes has a second consequence worth naming: failure becomes partial, and partial failure has to be visible. With uploads down and watch healthy, the studio should say uploads are paused rather than let clients retry into a wall — and the watch path must take no hard dependency on anything in the write plane, or I have built two planes that still die together.
Step 6 · Count the views
Views, watch time & recs
Owners and the recommender want view counts, watch time, and quality stats. Writing a row on every play — and incrementing a counter on a viral video — would hammer the database.
Owners want view counts and watch time, but a DB write per play (and a counter bump on a viral video) would hammer it. How?
Plays emit events into an analytics pipeline; counts aggregate in the background (approximated for hot videos), and recommendations train on the same stream — never on the playback critical path. If analytics lags, playback is unaffected.
A synchronous counter bump per play on a viral video is a write hotspot that can melt the DB and adds latency to playback. Counting must be async and off the hot path.
Client-only counts are unverifiable and easily gamed, and still need to reach the server. Emit events to a pipeline that aggregates and validates server-side.
Make it asynchronous. The player emits lightweight watch events that flow into an analytics pipeline; counts are aggregated in the background and approximated for hot videos. Recommendations train on this same event stream — never on the playback critical path.
Why this piece earns its place
Moving counting onto a stream gives playback its latency back and hands me a different problem: a pipeline like this delivers at least once, so retries, reconnects and a player that fires again after a network blip all arrive as separate events. Summed naively every count is inflated, and inflated most for exactly the videos people reload. So a play gets an id minted once in the player, and events carry it alongside a position, which makes deduplication possible at aggregation and makes replay safe — the day a consumer bug forces a reprocess, I need to run the log again and land on the same number. That forces the question underneath: what a view is. It cannot be a request, because requests are trivial to manufacture; it has to be a threshold on watch time, decided server-side, and written down before two teams compute it separately and disagree in public. I would also expect a published count to move in both directions after a reprocess, so the owner sees a figure openly marked as still settling rather than one that silently goes down.
What the new pieces do
- View Analyticsstore
- Aggregates views, watch time and quality switches asynchronously, so counting a view never slows playback.
Step 7 · The sharp edges
Uploads, copyright & live
Real life intrudes: uploads fail halfway on flaky connections, pirated content appears, and some events must stream live with seconds of delay, not after a full encode.
Use resumable, chunked uploads so a dropped connection retries only the missing chunk. Run a content-matching system (fingerprinting) to flag copyrighted material. For live, encode segments on the fly and publish the manifest continuously — same ABR + CDN idea, just streaming as it happens.
Why this piece earns its place
Everything before this assumed a video has an end. The pipeline waits for a complete file, encodes the whole ladder, then flips a flag to ready — and a live stream has no complete file and no moment to flip. That is why running the same pipeline faster is not the answer: queue-and-drain has no deadline anywhere in it, and a live encoder is nothing but deadlines. Segments have to be produced continuously and published on time, and when a rung misses its slot the right move is to publish the manifest without that rung rather than hold every viewer back for one quality. The assumption that bends next is immutability. Segments stay immutable and still cache perfectly, but a live manifest changes every few seconds, so it needs a short cache lifetime while the segments it lists keep a long one — two very different caching rules inside one stream. Get that wrong and viewers sit on a stale manifest watching a stream that has already moved on, which looks like the player froze rather than like the cache did.
You did it
You just designed YouTube.
Everything you assembled, in order
- Split tiny metadata (DB) from huge media (blob store).
- Async transcoding turns one raw file into an ABR ladder of segments.
- Adaptive bitrate lets the player switch quality without buffering.
- A CDN serves immutable segments from the edge — 95%+ offload.
- Tiered hot/cold blob storage keeps exabytes affordable.
- Async watch events drive view counts and recommendations.
- Resumable uploads, content-ID, and live streaming for the real world.
