Publish-Subscribe

Publish-Subscribe (often abbreviated as pub-sub) is a messaging pattern in which senders (publishers) do not program messages to be sent directly to specific receivers (subscribers). Instead, published messages are characterized into classes, without knowledge of which subscribers, if any, there may be. Subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are. This decoupling of publishers and subscribers is the defining architectural property of pub-sub systems: it replaces direct addressing with topic-based indirection, transforming a point-to-point network into a broadcast topology.

The pattern is older than digital computing. Stock ticker tapes in the 19th century operated on a pub-sub model: a central publisher (the exchange) broadcast price updates to any subscriber with a receiving device. Modern instantiations include Redis Streams, Apache Kafka, MQTT for IoT devices, and message brokers like RabbitMQ. What unifies these disparate systems is not their implementation but their structural commitment to temporal and spatial decoupling — publishers and subscribers need not exist simultaneously, and they need not know of each other's existence.

This decoupling is not without cost. Pub-sub systems introduce a new failure mode: the slow consumer. When a subscriber cannot process messages as fast as the publisher generates them, messages accumulate in buffers, consume memory, and eventually trigger backpressure or data loss. The choice between dropping messages, blocking publishers, or buffering indefinitely is a fundamental tension in pub-sub design, and different systems make different choices. Redis Streams uses consumer groups to parallelize consumption across multiple subscribers. Kafka partitions topics to scale horizontally. MQTT relies on QoS levels to negotiate delivery guarantees. Each solution reveals a different philosophy about the relative priority of publisher throughput, subscriber latency, and data durability.

Publish-Subscribe is often praised as a decoupling pattern, but the praise is only half-true. Pub-sub decouples publishers from subscribers in space and time, but it couples them in rate. The publisher's production rate and the subscriber's consumption rate become a shared variable that neither controls directly. This is the hidden contract of pub-sub: you gain freedom from knowing who receives your messages, but you lose the ability to predict whether they will be received at all. In systems where message loss is unacceptable, pub-sub is not a solution; it is a liability dressed in elegant abstractions.