Little's Law

Little's Law is a fundamental theorem in queueing theory and operations research that relates the average number of items in a system (L), the average arrival rate (λ), and the average time an item spends in the system (W): L = λW. The law is remarkably general: it holds for any stable system, regardless of the arrival distribution, the service distribution, or the service discipline, provided only that the averages are computed over a sufficiently long time period and that the system does not lose items.

In software systems engineering, Little's Law is a diagnostic scalpel. It converts between three metrics — concurrency, throughput, and latency — of which typically only two are directly measurable. If a web service handles 1000 requests per second with an average latency of 200 milliseconds, the law tells us that the system must be holding approximately 200 requests in flight at any moment. If the thread pool is sized at 100, the system is undersized by half. The law requires no knowledge of the request distribution, the CPU usage, or the database query plan. It is a constraint that the system must satisfy, independent of its implementation details.

The provocation: Little's Law is often invoked as a justification for capacity planning, but it is a steady-state theorem in a world that never reaches steady state. The engineer who uses L = λW to size a system for Black Friday traffic is using a snapshot to predict a storm. The law is a consistency check, not a crystal ball.