The whole design, in writing
Learn system design by building a Content Delivery Network like Cloudflare, Akamai or Fastly step by step. An interactive guide covering the origin-only baseline, edge PoP caching, anycast/GeoDNS routing to the nearest edge, cache-fill with TTL and Cache-Control, tiered shield caches that protect the origin from thundering herds, invalidation via purge and versioned URLs, edge compute, and the unhappy paths.
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What is a CDN?
A visitor in Sydney requests an image hosted on a server in Virginia. Light itself takes ~80ms each way; add the round-trips of a TCP+TLS handshake and the file is slow before a single byte of it moves. Do that for every image, script and video on the page, for every user on Earth.
A Content Delivery Network fixes distance the only way you can: don't move the user closer to the server — move copies of the content closer to the user. Hundreds of cache servers worldwide each hold a copy, and every visitor is served from the one nearest them.
What the new pieces do
- Visitorclient
- Someone anywhere on Earth requesting an asset — an image, script, video segment. Wants it fast, no idea where it actually comes from.
Step 1 · The baseline
One origin, the whole world
Start with the naive setup: one origin server holds every file, and every visitor on Earth talks to it directly. It's correct — everyone gets the right bytes. So what's actually wrong with it?
One origin server, users everywhere. What is the core problem?
Disk is cheap and easy to grow; that's not the pain. The pain is that a request from the far side of the planet is slow no matter how fast the origin is — physics, not capacity.
Distance is fixed latency you can't optimize away at the origin, and concentrating all traffic on one origin makes it both slow and a single point of failure.
A faster CPU doesn't shorten the wire. A visitor 15,000km away still eats the speed-of-light round trip on every asset. You must serve from closer.
Two problems, one root cause: distance. Every request pays the full round-trip to a far-away origin (latency you can't code away), and funnelling the planet through one machine makes it a bottleneck and a single point of failure. The fix isn't a bigger origin — it's serving from closer.
Why this piece earns its place
Worth separating the two problems, because only one of them needs a delivery network. The single box is a capacity and availability problem, and it has a cheap local fix: a few origin replicas behind a load balancer in one region. That buys throughput and survives a dead machine, and it moves Sydney’s number by exactly nothing — the round trip is the wire, and replicas sit at the same end of it. Latency is the half that can only be paid down by moving the content. The assumption worth naming now is that this is read-heavy traffic over objects lots of people want. A cache near the user pays back when many requests want the same bytes; for a response computed fresh for one person the cache itself earns nothing, because there is no second reader to hand the copy to — whatever an edge is still worth on that traffic, it is worth for reasons that have nothing to do with storing anything. So the number I would ask for before designing any of this is what share of requests are shared, cacheable objects — that, not the request rate, decides whether everything after this is worth building.
What the new pieces do
- Origindata
- Your server or object store holding the real files. The CDN protects it: ideally it only ever sees cache-fill traffic.
Step 2 · Move content to the edge
A cache near every user
If distance is the enemy, put a copy of the content near the user. But a copy where, and holding what — you can't pre-load every file to every city, and content changes.
How do you serve users from nearby without pre-copying everything everywhere?
Full replication everywhere is wasteful (most files are never requested in most cities) and goes stale the instant content changes. You want to cache on demand, not copy everything.
Users can't and won't choose a mirror, and a hardcoded mirror can't adapt when it's down or far. Routing must be automatic and content on-demand.
The edge lazily pulls each object from origin on its first request and holds it. Later requests for that object in that region are served locally — a few ms instead of a cross-planet round trip.
Deploy Points of Presence (PoPs) — cache servers in data centers around the world. A PoP holds nothing until asked; on the first request for an object it fetches from origin (a miss) and stores the copy. Every later request in that region is a hit, served locally in milliseconds.
Why this piece earns its place
Splitting traffic across hundreds of caches also splits the evidence each one has. A PoP only earns its place when the same object is asked for there again before it is evicted, so the useful question is not how many PoPs you run but how many requests per object per PoP you get. Popular objects are fine. The long tail is not: an object wanted once a week in one city misses almost every time, so the PoP carries storage and fill cost for a copy that is evicted before it is ever reused. More PoPs is therefore not monotonically better — past some point you are dicing the same traffic into thinner caches and lowering the hit ratio you were buying. The defenses are structural: give the edges a shared parent to miss into rather than letting each one fill independently, and be willing to leave big, cold objects uncached rather than evicting a hundred hot ones to store something nobody asks for twice.
- 100sPoPs worldwide
- ~5msedge hit
- 1st reqfills the cache
What the new pieces do
- Edge PoPcache
- A cache server in a data center physically near the user — one of hundreds worldwide. A hit here returns in a few milliseconds.
Back of the envelope
- pull cache, on demand
- no pre-copying the whole origin
- first request = miss ⇒ fetch
- then served locally
- popular content self-locates
- near its audience
Step 3 · Find the nearest edge
How does the user reach the closest PoP?
Hundreds of PoPs exist — but the visitor's browser just resolves cdn.example.com and opens a connection. How does that connection land on the nearest PoP and not a random one across the world?
A browser resolves one hostname. How does it reach the nearest of hundreds of PoPs?
Browsers don't know the PoP list and can't probe hundreds of them per request — far too slow. Nearest-routing has to happen in the network layer, transparently.
With anycast the same IP is advertised (via BGP) from every PoP; internet routing naturally hands each user's packets to the topologically closest one. GeoDNS is the alternative — return a PoP IP based on the resolver's location.
A single central router is exactly the far-away bottleneck a CDN exists to avoid — every request would detour through it. Routing must be distributed into the network itself.
Use anycast: every PoP advertises the same IP address via BGP, and the internet's own routing delivers each user's packets to the topologically nearest PoP — no central decision, no extra hop. (GeoDNS is the classic alternative: resolve the hostname to a different PoP IP based on where the query came from.)
Why this piece earns its place
Two things about anycast worth saying out loud. First, BGP’s nearest is topological, not geographic — it counts network hops and routing policy, not kilometres — so a user whose ISP peers oddly can land on a PoP much further away than the one you would have picked, and nothing in the system reports that as wrong. You find it by measuring real round-trip times per client network, which is also the standing argument for keeping GeoDNS: it is the lever for overriding the network’s answer for one region when that answer is bad. Second, be precise about what failure this heals. Anycast re-routes when a PoP stops announcing the route, so it covers a PoP that is gone. It does nothing for a PoP that is up and sick — disks full, upstream broken, returning errors quickly — because that PoP keeps advertising and keeps attracting every user for whom it is nearest. So the health check has to be wired to the announcement itself: a PoP that fails its own checks withdraws its route, and the network routes around it.
- 1 IPannounced everywhere
- BGPpicks nearest
- autore-routes on failure
What the new pieces do
- Anycast / GeoDNSrouting
- Steers each visitor to the closest edge. Anycast announces one IP from every PoP; the network delivers to whichever is nearest.
Back of the envelope
- anycast: one IP, many PoPs
- network delivers to nearest
- GeoDNS: resolve by location
- finer control, slower to react
- a dead PoP self-heals
- routing skips it automatically
Step 4 · Freshness
When can the edge trust its copy?
A cached copy is fast, but content changes — a new logo, an updated price. If the edge serves its copy forever, users see stale bytes. If it re-checks origin every time, you've thrown away the whole benefit. Who decides how long a copy is good for?
Cached copies can go stale. How long should an edge trust its copy?
One TTL for all content is wrong for most of it — a logo can cache for a year, an API response for seconds. Freshness is a property of the object, so the origin should declare it.
Checking origin every time defeats caching — you're back to a round trip per request. You only want to re-check when the copy might actually be stale.
Cache-Control: max-age and ETag/Last-Modified let the origin say exactly how long each object is fresh. The edge serves freely until expiry, then does a cheap conditional revalidate (304 Not Modified) instead of a full refetch.
The origin decides, per object, with HTTP Cache-Control headers. max-age sets the TTL; the edge serves the copy freely until it expires. On expiry it does a cheap conditional revalidation — "still valid?" via ETag/If-None-Match — and the origin answers 304 Not Modified (no body) if nothing changed.
Why this piece earns its place
Be clear-eyed about what a revalidation costs. A 304 is cheap in bytes and not at all cheap in time — it is still a full round trip from the edge to the origin, with a user sitting on it. So a short max-age does not buy freshness for free; it quietly converts a share of your requests back into origin-latency requests while the object still counts as cached. A page can look like it is being served from the edge and still carry the origin’s tail latency if the TTLs are set in seconds. That is why the answer to wanting fresher content is almost never a smaller number here. And the direction of the mistake matters, because a long TTL is not reversible: the header goes to every cache in the path, including the visitor’s browser, and you cannot purge a browser. Whatever you mark cacheable for a year, you are committed to for a year, on machines you will never touch — which is why long, confident TTLs belong on assets whose URL you control, not on the document that names them.
- max-ageorigin sets TTL
- 304cheap revalidate
- immutablenever re-check
Step 5 · Protect the origin
A thousand edges, one origin
A brand-new viral video: hundreds of PoPs all miss on it at the same instant and every one stampedes the origin for the same file. The origin — the one thing you were protecting — gets hammered by its own CDN. How do you absorb the fan-in?
On a fresh object, hundreds of edges miss at once and all hit origin. How do you protect it?
Scaling the origin fights the symptom, not the cause — you still take N simultaneous fetches for one object every time something new goes viral. Collapse the fan-in instead.
A shield (parent) cache sits between edges and origin. Many edges miss into one shield; the shield does a single origin fetch (with request coalescing), so the origin sees one request per object per region, not thousands.
Backoff spreads the stampede out in time but the origin still eats every distinct fetch, just slower. You want to collapse many misses into one, which is what a shield tier does.
Add a tiered cache. Edges don't miss straight to origin — they miss into a regional shield (parent) cache. Many edges fold into one shield, and the shield uses request coalescing (a.k.a. collapsed forwarding) so concurrent misses for the same object become a single origin fetch. The origin sees one request per object per region.
Why this piece earns its place
Both halves of this have a cost worth naming. The shield is another hop, so a miss now pays edge to shield to origin; that is a win only if the parent is genuinely regional, because a shield on another continent turns origin protection into a latency tax on every miss. And coalescing changes the shape of failure. Independent requests fail independently; a coalesced one has shared fate. Ten thousand waiters now hang on a single upstream fetch, so a stalled origin connection does not fail one request, it freezes all of them at once — and when they give up together the retry burst is worse than the stampede you were preventing. So the leader’s fetch needs a hard timeout, and the waiters need somewhere defined to go when it expires: a stale copy, a capped queue, an explicit error. Collapsing the fan-in is right. Just decide deliberately what happens when the one request everything depends on does not come back.
- 1000s : 1edges : origin fetch
- coalescedmisses share one fetch
- 2 tiersedge → shield → origin
What the new pieces do
- Shield Cachecache
- A regional parent cache sitting between the edges and the origin. Many edges miss into one shield, so the origin sees one fetch, not thousands.
Back of the envelope
- edge → shield → origin
- each tier collapses fan-in
- request coalescing
- concurrent misses = one fetch
- origin sees ~1 req / object / region
- not thousands
Step 6 · Change it now
Invalidation: the hard problem
You shipped a bug in app.js and it's cached at a year-long TTL across 300 PoPs. You need every edge to stop serving the old copy now, not in a year. But you can't reach out and edit 300 caches instantly and cheaply. How?
A bad file is cached for a year across 300 PoPs. How do you replace it now?
Tiny TTLs mean constant revalidation — you trade away almost all the cache benefit to buy freshness you rarely need. Cache long, and invalidate precisely when you actually change something.
A year is not "now". Natural expiry is fine for routine freshness but useless for an urgent fix. You need an explicit lever.
Two tools: (1) versioned/fingerprinted URLs (app.9f3a1.js) make a change a brand-new URL that was never cached, so it can't serve stale — and you cache the old one forever harmlessly. (2) An explicit purge command from the control plane evicts a key from every PoP in seconds for cases where the URL is fixed.
Two complementary levers. Versioned URLs: fingerprint assets by content hash (app.9f3a1.js) so any change is a new URL — never stale, and the old one can cache forever. For fixed URLs (like an HTML page), issue an explicit purge from the control plane that evicts the key from every PoP in seconds. Prefer versioning; purge when you can't change the URL.
Why this piece earns its place
Versioning does not remove invalidation, it relocates it. Something has to name app.9f3a1.js, and that something is the HTML document, which by definition cannot be fingerprinted — so all the pressure you just took off hundreds of assets now sits on one entry document that has to be short-TTL and purgeable. It also means a deploy is never atomic across the edges: for as long as the old document is cached anywhere, some visitors are running old HTML and will ask for the old asset. The rule that travels with versioning is therefore that you never purge the previous versions — keep them and let them age out — or a deploy hands a slice of your users a broken page that reloading does not fix. On the purge side, treat the fan-out as a distributed system rather than a broadcast. A PoP that was rebooting or partitioned when the command went out comes back holding the old object and serves it happily, so what it reconciles against on startup should be a durable version it can compare, not a message it had to be awake to hear.
- content hashnew file = new URL
- purgeevict in seconds
- soft purgestale-while-revalidate
What the new pieces do
- Purge / Controlcontrol
- The control plane. Pushes purge commands and new config to every edge so stale content can be evicted on demand.
Step 7 · Beyond static files
Compute and see at the edge
Not everything is a static image. You want auth checks, A/B tests, redirects, personalization, and — separately — you need to know what every PoP is doing for analytics, billing and attack detection. Doing that at the origin re-introduces the round trip you just removed.
Run edge compute: small functions that execute at the PoP (rewrite headers, do auth, personalize, route) so even dynamic responses stay near the user. And stream request logs from every edge asynchronously to a central pipeline — used for analytics, billing and DDoS/WAF detection — never blocking the response.
Why this piece earns its place
The real cost of computing at the edge is what it does to the cache. A response that varies per visitor is by definition not shareable, so the moment a function personalizes a body you have to tell the cache what the answer depends on — and getting that wrong in the permissive direction caches one visitor’s page and hands it to everyone behind that PoP, which is the worst bug on this page precisely because it is fast, quiet and correct-looking. Getting it wrong in the strict direction is merely expensive: every dimension you add to the key multiplies the copies each PoP must store and fill, so keying on something high-cardinality like a user id makes every request a miss and you have built an elaborate proxy. Keep the varying dimension small and enumerable — a country, a device class, an experiment bucket — and push genuinely per-user work into the things that do not change the cached body: headers, redirects, an auth decision taken before the lookup.
What the new pieces do
- Edge Computeservice
- Code that runs at the edge — auth, A/B, header rewrites, personalization — so dynamic responses stay near the user too.
- Edge Logsdata
- Every edge streams request logs asynchronously for analytics, billing and security — never on the hot path.
Step 8 · The sharp edges
Stale, stampede & origin-down
Real traffic finds the corners: an object expires under sudden load (a cache stampede), the origin goes down mid-refresh, or a region needs content it's never seen.
Use stale-while-revalidate — serve the slightly-stale copy instantly while refreshing in the background — so an expiry never blocks a user. Combine it with coalescing so one refresh serves the whole stampede. Serve stale-if-error when the origin is down, turning the cache into an availability shield. And keep TTLs long with versioned URLs so freshness costs you nothing.
Why this piece earns its place
Stale-if-error is a real shield with a shape worth stating: it can only serve what a PoP already holds, so its coverage is whatever was popular enough to be cached there, and that coverage decays. The longer an outage runs, the more entries fall out of even the stale window, so availability degrades over hours rather than holding flat — and how long you are willing to serve stale is a per-content decision, since serving a price or an offer long after the origin went dark is its own kind of incident. But the failure I would actually plan for is not a dead origin. A dead origin is the easy case: it errors, and these rules fire. The dangerous one is an origin that is healthy and wrong — a bad deploy answering with an empty body, a misconfigured bucket serving a default page. The edge cannot tell, so it caches that as the truth, spreads it everywhere and serves it fast. Which is why a cache fill deserves its own sanity checks on status, content type and plausible size, and why the purge path from the previous step is a lever you must be able to pull in a hurry.
You did it
You just designed a CDN.
Everything you assembled, in order
- Distance is the enemy — serve copies from PoPs near the user, not from one far origin.
- Edges are pull caches: first request fills, the rest are local hits.
- Anycast (one IP, nearest PoP) + GeoDNS route users to the closest edge and self-heal on failure.
- The origin declares freshness with Cache-Control/max-age + ETag; edges revalidate cheaply with 304.
- Tiered shield caches + request coalescing turn thousands of edge misses into one origin fetch.
- Versioned URLs sidestep invalidation; explicit purge is the escape hatch for fixed URLs.
- Edge compute + async logs; stale-while-revalidate and stale-if-error keep it fast and up.