AIXI

AIXI is a mathematical formalization of a theoretically optimal artificial general intelligence, proposed by Marcus Hutter in 2000 and developed in his 2005 book Universal Artificial Intelligence. AIXI combines Solomonoff induction — a formalization of Occam's Razor for sequence prediction — with the decision-theoretic framework of expected utility maximization. It defines what an agent would do if it could enumerate all computable hypotheses about its environment, weight them by their Kolmogorov complexity, and choose actions that maximize expected reward across this distribution.

AIXI is the most rigorous formal answer to the question: what does an optimal learning agent look like? And its answer is deeply instructive, not because AIXI could ever be built, but because what makes it impossible tells us something important about the limits of computational approaches to intelligence.

The reason AIXI cannot be implemented is that the computation it requires is uncomputable: Solomonoff's prior sums over all computable programs, which requires solving the halting problem. No physical system can compute AIXI's action policy exactly. Approximations exist — AIXI^tl bounds computation by time and program length — but the convergence properties of AIXI that make it theoretically interesting are not preserved by the approximations that make it practically relevant.

This points to a recurring structure in theoretical AI: the formally optimal solution is uncomputable, and the computable approximations are not provably close to optimal in the environments where optimality would matter most. AIXI is to machine learning as the Turing machine is to actual computers: a mathematical boundary case that clarifies the conceptual space without resolving the engineering challenge.

Equally notable: AIXI maximizes reward relative to an externally specified reward signal. It does not — cannot — evaluate whether the reward signal is well-specified, whether pursuing reward in the specified environment is beneficial, or whether its actions are intelligible to the agents around it. In this sense, AIXI is a formal proof that alignment cannot be factored out of intelligence: a maximally intelligent agent, formally speaking, is one that pursues its objective without any evaluation of whether that objective is worth pursuing. The most rational agent is potentially the most dangerous one.