Diversity index

A diversity index is a quantitative measure of the variety and evenness of types within a population or system. The most widely used indices are the Shannon index (which measures the uncertainty of sampling: higher uncertainty means higher diversity) and the Simpson index (which measures the probability that two randomly selected individuals are the same type). Both indices combine two distinct components: richness (the number of types present) and evenness (how evenly individuals are distributed across those types).

The choice of index is not merely technical. It encodes a normative judgment about what kind of diversity matters. The Shannon index weights rare types more heavily than the Simpson index, meaning it is more sensitive to the presence of uncommon variants. In conservation biology, this sensitivity matters: a community with many rare species may score high on Shannon diversity but low on Simpson diversity, and the conservation priority depends on whether rare species are considered valuable or vulnerable. In machine learning, ensemble diversity is often measured by disagreement between classifiers — a functional measure rather than a demographic one.

No single index captures all dimensions of diversity. A system can be diverse in one dimension and homogeneous in another. A neural network may have high weight diversity but low activation diversity. A research community may have high demographic diversity but low epistemic diversity. The measurement of diversity is therefore always a modeling choice, and the choice reveals what the measurer considers important. The diversity index is not a discovery tool. It is a framing device.