NUMA architecture
Non-Uniform Memory Access (NUMA) is a computer memory architecture in which processors can access their own local memory faster than memory attached to other processors. In a NUMA system, each processor has its own local memory bank, and accessing remote memory (attached to another processor) incurs a latency penalty — typically 1.5× to 3× the cost of local access. The architecture is the dominant memory model for modern multiprocessor servers, high-performance computing clusters, and even some consumer systems, yet it remains one of the most poorly understood performance factors in systems programming.
NUMA emerged from the physical reality that memory controllers cannot scale to serve an arbitrarily large number of cores at uniform latency. As the multicore revolution pushed core counts from 2 to 64 to 128 and beyond, the shared-memory bus — the backbone of Uniform Memory Access (UMA) systems — became a bottleneck. The solution was to distribute memory controllers across the die or across sockets, giving each processor fast access to its "home" memory and slower access to everything else. This is not a software abstraction that could be optimized away; it is a fundamental consequence of the speed of light, wire delay, and thermal constraints on chip design.
The Architecture
In a typical NUMA system, processors are organized into nodes. Each node contains one or more CPU cores, a local memory controller, and cache. Nodes are connected by an interconnect fabric — Intel's Ultra Path Interconnect (UPI), AMD's Infinity Fabric, or proprietary mesh networks in large-scale systems. The latency of a memory access depends on which node contains the physical address being accessed:
- Local access — the processor reads from its own node's memory. Latency is minimal, typically 80-120 cycles on modern hardware.
- Remote access, same socket — the processor reads from another node on the same physical socket. Latency is higher due to on-die interconnect traversal.
- Remote access, different socket — the processor reads from a node on a different physical socket. Latency can be 2-3× local, and the request must traverse both the on-die and the inter-socket interconnect.
The operating system sees a single unified address space, but the physical reality is a topology of access costs. A page of memory allocated on node 0 will be fast for core 0 and slow for core 7 if they are on different nodes. This is invisible to correct programs — they will run correctly regardless — but devastating to performance-sensitive ones.
The Cache Coherence Problem
NUMA complicates cache coherence dramatically. In a UMA system, all caches share a common view of memory through a single bus or directory. In a NUMA system, coherence must be maintained across nodes, and the coherence traffic itself traverses the interconnect fabric. A cache line bounce — when a cache line moves from one node's cache to another's — is not just a cache miss; it is a network transaction.
Directory-based coherence protocols (common in NUMA systems) track which nodes have copies of each cache line in a distributed directory. When a node requests a line, the directory identifies the current owner and forwards the request. This avoids the broadcast overhead of snooping protocols but introduces directory lookup latency and directory storage overhead. The directory itself is distributed across nodes, so even a directory lookup can be a remote access.
The interaction between NUMA and coherence creates pathological performance cliffs. A naively parallel program that allocates all its data on one node and then distributes threads across nodes will suffer not only from remote memory access but also from cache line contention: every write to a shared variable by a thread on a different node triggers a coherence transaction across the interconnect. On a 64-core system, this can reduce effective memory bandwidth by an order of magnitude.
NUMA-Aware Programming
The performance gap between NUMA-aware and NUMA-oblivious code on large systems is not marginal; it is transformative. A matrix multiplication kernel that ignores NUMA topology may run at 20% of peak performance. The same kernel with explicit NUMA placement — allocating memory on the node where the compute threads will run, binding threads to cores, and minimizing cross-node communication — can approach 90%.
Operating systems provide tools for NUMA awareness: Linux's `numactl` and `libnuma` allow explicit control over memory placement and thread affinity. The `first-touch` policy — allocating a page on the node of the first processor to access it — is the default on Linux and is sufficient for embarrassingly parallel workloads. But for workloads with irregular access patterns or shared data structures, first-touch is not enough; the programmer must explicitly partition data and map computation to nodes.
Modern runtimes (OpenMP, MPI, task-based systems) attempt to automate NUMA awareness. The success is partial. Automatic data placement requires predicting access patterns, which is undecidable in general. Static analysis can help for regular workloads but fails for graph algorithms, particle systems, and adaptive mesh refinement — precisely the workloads that scale to the largest NUMA systems.
NUMA and the Future of Systems
NUMA is not a transitional architecture. As core counts continue to grow and memory hierarchies deepen, the non-uniformity of memory access is becoming more pronounced, not less. Chiplet-based designs (AMD Zen, Intel Meteor Lake) are NUMA at the sub-package level: each chiplet has its own local memory and cache, and the inter-chiplet fabric introduces latency penalties analogous to traditional inter-socket NUMA. The distinction between "NUMA" and "uniform memory" is dissolving into a spectrum of memory locality, from L1 cache (1 cycle) to local DRAM (100 cycles) to remote DRAM (300 cycles) to NVRAM (1000+ cycles).
This has implications for software architecture. The assumption that memory is a flat, fast resource — an assumption baked into most programming languages, algorithms, and data structures — is increasingly wrong. Programmers who write code as if all memory accesses cost the same are writing code for a machine that no longer exists. NUMA forces a return to locality-aware design, to data placement as a first-class concern, and to the recognition that the physical structure of the machine is not an implementation detail but a primary constraint on algorithmic performance.