Cache Coherence

Cache coherence is the guarantee that all processors in a shared-memory multiprocessor system observe a consistent view of memory. When multiple cores maintain private caches of the same physical memory locations, the hardware must ensure that a write by one core becomes visible to all other cores in a well-defined order. Without coherence, parallel programs would observe stale data, race conditions would proliferate, and the abstraction of shared memory would collapse into chaos.

The problem is deceptively simple to state and ferociously complex to solve. Each processor core reads and writes cached copies of memory. A write to a cached line must eventually propagate to all other caches holding that line. The question is not whether to propagate but when, in what order, and at what cost. The answers to these questions define the coherence protocol, and the protocol defines the performance ceiling of the entire system.

The Cache Coherence Problem

Consider two cores, A and B, each with a private L1 cache. Both have loaded variable x from main memory. Core A writes x = 1. Core B, without coherence, might read its stale cached copy and see x = 0. The program has entered undefined behavior not because of a software bug but because the hardware abstraction of shared memory has leaked.

The naive solution — disabling caches or flushing on every write — would make caches pointless. The practical solution is a coherence protocol that tracks which caches hold which lines, in what state, and mediates transitions between states. These protocols are typically implemented in hardware, in the cache controller, and operate below the level of software visibility. A programmer writing parallel code assumes cache coherence; the protocol makes that assumption valid.

Coherence Protocols: Snooping and Directory-Based

Two dominant architectures solve the coherence problem: snooping protocols and directory-based protocols.

Snooping protocols rely on a shared interconnect (typically a bus) that all cache controllers monitor. When a core issues a read or write, it broadcasts the request on the bus. All other caches snoop the request and respond if they hold the line. The most common snooping protocol is MESI, which defines four states for each cache line: Modified, Exclusive, Shared, and Invalid. A line in Modified state has been written by this cache and is dirty — main memory is stale. A line in Exclusive state is clean and held only by this cache. A line in Shared state is clean and may be held by multiple caches. A line in Invalid is, effectively, not present.

Snooping scales poorly. As the number of cores grows, bus traffic increases quadratically with the number of sharers. By the time you reach 16-32 cores, the bus becomes a bottleneck. This is not merely an engineering limitation; it is a fundamental constraint of broadcast-based communication. The snooping approach assumes that all coherence traffic can be observed by all participants simultaneously — an assumption that breaks down at scale.

Directory-based protocols solve this by maintaining a directory — a data structure that tracks which caches hold each memory line and in what state. Coherence requests are sent not to all caches but to the directory, which forwards them only to the caches that actually hold the line. This reduces traffic from O(N) per request to O(1) or O(log N), depending on directory organization.

Directory protocols are the foundation of NUMA and large-scale multiprocessors. The Intel Xeon processors with large core counts, AMD EPYC chips, and virtually all server-class hardware use directory-based or hybrid protocols. The directory itself becomes a scalability challenge — it must be distributed, cached, and kept coherent — but the asymptotic improvement over snooping is decisive.

The Memory Consistency Model

Cache coherence is necessary but not sufficient for correct parallel programming. Coherence guarantees that writes become visible, but it does not specify *when* or in what *order* relative to other writes. The memory consistency model defines the ordering guarantees that programmers can rely on.

Sequential consistency — the idealized model in which all operations appear to execute in a single global order — is too expensive to implement in hardware. Real processors implement weaker models: total store ordering (TSO), partial store ordering (PSO), or relaxed models that allow extensive reordering. The programmer's contract with the hardware is defined by the memory model, and coherence is the mechanism that implements part of that contract.

The interaction between coherence and consistency is subtle. A coherence protocol might guarantee that a write eventually reaches all caches, but the memory model determines whether a subsequent read on another core is *ordered* with respect to that write. Without proper memory fences and barriers, even a perfectly coherent system can produce surprising results. Coherence is about visibility; consistency is about ordering. Both must be correct, and both extract a performance cost.

Coherence in the Heterogeneous Era

Modern systems are not symmetric multiprocessors with identical cores. They are heterogeneous collections of CPU cores, GPUs, AI accelerators, and I/O engines, each with different cache hierarchies and coherence requirements. Maintaining coherence across this diversity is one of the defining challenges of contemporary computer architecture.

The Apple M-series chips maintain system-wide coherence between CPU cores, GPU, and neural engine through a unified memory architecture. ARM's CHI (Coherent Hub Interface) protocol extends directory-based coherence to heterogeneous systems. Intel's CXL (Compute Express Link) and AMD's Infinity Fabric are, at their core, coherence fabrics scaled to accommodate accelerators with different memory semantics.

The cost of coherence is not merely traffic. It is latency: each coherence transaction requires communication across the chip, potentially to a directory, potentially to another die, potentially across a package boundary. In a chiplet-based design, coherence traffic may traverse interposer or organic substrate links that are slower and more power-hungry than on-die wires. The coherence protocol becomes a distributed systems problem implemented in silicon.

Cache coherence is often taught as a solved problem — a detail of computer architecture that was settled in the 1990s. This is dangerously wrong. Coherence is not solved; it is merely hidden. As systems scale to hundreds of cores, across chiplets, into the heterogeneous era, the coherence problem is reasserting itself with renewed ferocity. The next decade of architecture will be defined not by how many cores we can place on a die but by whether we can keep their caches coherent without consuming the entire power budget and latency budget in the process.