NUMA: Why More CPU Cores Can Make Your Program Slower
Move a fast multithreaded program to a bigger server with more cores and it can run slower — same code, same data. On a big machine, memory isn't one uniform pool: each CPU socket has its own bank of RAM, and reaching another socket's memory is roughly twice as slow as reaching your own.
Non-Uniform Memory Access means each CPU socket on a big machine has its own bank of RAM wired to it: reaching your own socket's memory is local and fast, reaching another socket's memory crosses an interconnect and costs roughly twice as much. The silent trap is first-touch allocation: the OS places a page on whichever socket's core first touches it, and if the scheduler later migrates your thread to a different socket, every subsequent access to that data goes remote — with zero code change. The fix is pinning threads to a node and allocating their data on that same node, so hot data stays local, which is exactly what databases, language runtimes, and HPC workloads do.
Transcript
Here's a haunting one: you take a fast multithreaded program, move it onto a bigger server with more cores — and it runs SLOWER. Same code, same data. Why? The cause isn't your code — it's your data's ADDRESS. On a big machine, memory isn't one pool: some sits right next to a core (local, fast), some surprisingly far. That's NUMA.
We picture RAM as one big pool every core reaches equally fast. On a laptop with a single chip, that's basically true. But a big server has several CPU chips — called sockets — and each socket has its OWN bank of RAM wired directly to it. One machine, memory in separate piles.
A core reaching its own socket's memory is fast — that's local access. Reaching another socket's memory means crossing a link between the chips. That remote trip has higher latency and lower bandwidth — often roughly twice as slow. Non-Uniform Memory Access: where your data sits decides your speed.
Here's the trap. The OS puts your data on the memory of whichever core FIRST touches it. Allocate from socket zero, and it lives on socket zero. Later the scheduler moves your thread to socket one to balance the load — and now every read is a remote trip across the link. Your code never changed.
The fix is to respect the neighborhoods. Keep each thread in its own neighborhood — one node — and allocate its data on that same node, so hot data stays local. Databases, language runtimes, and high-performance code all do this — pinning threads, allocating locally, or interleaving shared data to spread the bandwidth.
So on a big machine, memory has a geography. The same bytes are cheap from a nearby core and expensive from a far one. So next time more cores make things slower, think NUMA: keep each thread's data local to the core that uses it.