epoll: How One Server Holds a Million Connections
Why servers used to fall over around ten thousand connections (the C10K problem), and how Linux's epoll flips the cost model from checking every connection to checking only the active ones — the mechanism under nginx, Redis, and Node's event loop.
select/poll scan your whole connection list on every call, so cost scales with total connections, busy or idle. epoll registers each connection once and lets the kernel maintain a ready list, so cost scales with active connections instead — the reason one box can hold a million mostly-idle sockets.
Transcript
Picture one receptionist watching a hall of ten thousand doors, waiting for someone to knock. That's a single server holding ten thousand live connections. For years, servers hit a wall right around that number — the famous C10K problem. So how does one box today hold a MILLION connections, without asking each 'ready yet?' all day?
The first idea is to hire one receptionist per door — one thread per connection. Fine for a few, but ten thousand threads eat memory and thrash the CPU switching between them. You can't scale bodies one-to-one with doors. You need ONE receptionist who can watch them all — so we ask the operating system to watch the sockets for us.
The old way to ask is a call named select. You hand the kernel your whole list of ten thousand connections and ask: which ones are ready? It checks every single one. So even when just three sockets have data, you pay to scan all ten thousand — on every loop. The cost grows with your TOTAL connections, not your active ones.
epoll, on Linux, flips it around. You register each connection with the kernel ONCE. From then on, the kernel watches them and keeps a ready list of which ones have activity. Now you just ask 'who's ready?' and it hands you only the active few, instantly. Ten thousand idle connections cost you almost nothing. That's the whole leap.
Because the cost now scales with ACTIVE connections, not total ones, a single box can babysit a million mostly-idle sockets without breaking a sweat. This is the engine under nginx, Redis, and Node's event loop — one thread calmly juggling a mountain of connections by only ever touching the ones that spoke.
So the secret to a million connections was never more threads. It's refusing to ask everyone 'anything yet?' and letting the ready ones raise their hand. Next time you hear one box serves a million users, you'll know the trick: don't poll everyone — register once, and let the kernel tell you who's ready.