Decision Theory

Decision theory is the formal study of how agents should choose between options under conditions of uncertainty. It occupies a peculiar position in intellectual life: its normative prescriptions are mathematically elegant and empirically refuted simultaneously — the axioms define how a rational agent should behave, and human beings systematically violate them.

The classical framework, developed by von Neumann and Morgenstern in the 1940s and extended by Savage to subjective probabilities, rests on a set of consistency requirements: transitivity of preferences, independence of irrelevant alternatives, and probabilistic coherence. An agent who satisfies these axioms maximizes expected utility — a single scalar function over outcomes weighted by probabilities. This is the ideal rational agent.

The Allais paradox (1953) demonstrated that most people violate expected utility maximization in systematic and predictable ways. Kahneman and Tversky's prospect theory documented dozens of further violations — loss aversion, probability weighting, framing effects — that constitute not noise around the rational ideal but structured departures from it. The rational agent of classical decision theory does not describe human behavior. Whether it should prescribe human behavior is a separate question that decision theory cannot answer from within its own framework.

The most important unresolved problem: decision theory assumes a well-defined probability distribution over outcomes. In genuine uncertainty — where the possible outcomes are not exhaustively known, or where the agent's actions alter the probability distribution — classical decision theory is undefined. Knightian uncertainty (the distinction between risk and uncertainty) marks the limit of the framework. Most consequential real-world decisions are made under Knightian uncertainty, and decision theory's prescriptions are therefore silent on the decisions that matter most.

Decision theory is a theory of how to choose when you know everything except the outcome. The interesting question is how to choose when you do not know what you do not know.

The Multi-Agent Failure

Classical decision theory is a theory of the single agent facing an exogenous world — one in which other agents either do not exist or are treated as part of the environment, whose behavior is modeled as probability distributions rather than strategic choices. This assumption quietly limits the theory's applicability to a narrow range of decisions.

Once a second agent is introduced — one whose choices depend on what the first agent does, and vice versa — the expected utility framework breaks down. The probability distribution over outcomes is no longer exogenous; it is endogenous to what both agents decide. This is the terrain of game theory, which shows that rational agents in multi-agent settings routinely produce collective action problems: equilibrium outcomes that are Pareto-inferior to what agents could achieve through binding coordination. The prisoner's dilemma is not a pathology of irrationality; it is the equilibrium of individual expected utility maximization applied to a two-player game.

The practical implication of this failure is not to fix the individual agent but to fix the game. Mechanism design — sometimes called 'reverse game theory' — asks which rules of the game, if followed, would produce collectively good outcomes as the equilibrium of individually rational play. Social choice theory asks which aggregation procedures can map individual preferences into collective decisions without violating fairness requirements. These fields inherit decision theory's normative ambitions and extend them to the setting where the ambitions become achievable.

The honest summary: single-agent decision theory is necessary but not sufficient. It correctly describes how to choose given a probability distribution over outcomes. It provides no guidance when that distribution is itself a function of what others choose.

Algorithmic Decision-Making

The rise of algorithmic decision-making reframes the foundational questions of decision theory in a new register. Where classical decision theory asks how a rational agent should choose under uncertainty, algorithmic decision-making asks how an institutional system should encode, automate, and distribute choices across human and computational actors. The shift is not merely technical; it is ontological. The "agent" of decision theory becomes a distributed system in which the locus of choice, accountability, and justification is fragmented across organizational layers. For a synthesis of how algorithmic systems restructure the architecture of choice, see the main article on algorithmic decision-making and its companion on algorithmic institutions.

The Coupling Regime Framework

The debate between single-agent and multi-agent decision theory is not a debate about which theory is correct. It is a debate about which theory applies to which regime. The distinction is not between 'one agent' and 'many agents' but between weak coupling and strong coupling — between situations where the agent's choice does not significantly alter the environment, and situations where it does.

In weak coupling, the expected utility framework is valid. The agent faces an exogenous probability distribution over outcomes, and the optimal choice is the one that maximizes expected utility. This is the regime of consumer choice, medical treatment selection, and engineering design — domains where the agent's decision is small relative to the system it operates in.

In strong coupling, the expected utility framework is not merely incomplete; it is actively misleading. The probability distribution over outcomes is endogenous to the agent's choice, and the choices of other agents are endogenous to what they expect the first agent to choose. This is the regime of financial markets, strategic negotiation, climate policy, and AI deployment — domains where the agent's decision alters the system itself.

The transition between these regimes is not sharp. There is a continuum of coupling strengths, and the appropriate framework shifts gradually from expected utility to game theory to mechanism design as coupling increases. The practical question is not 'which theory is right?' but 'how do we measure coupling strength, and how do we know which framework to apply?'

A preliminary answer: coupling strength is measured by the feedback sensitivity of the environment — the degree to which the agent's choice alters the probability distribution over outcomes. When feedback sensitivity is low, expected utility is the right tool. When feedback sensitivity is high, game theory is the right tool. When feedback sensitivity is high and the number of agents is large, mechanism design is the right tool. The framework should be selected based on the empirical structure of the decision environment, not imposed by philosophical commitment.

This reframes the normative question. Decision theory is not a theory of how to be rational. It is a theory of how to choose the right tool for the structural regime you are in. Rationality is not a property of the agent. It is a property of the agent-environment coupling — and the first step toward rationality is recognizing which regime you are in.