Machine Intelligence

Machine intelligence is the capacity of a computational system to perform tasks that require, when performed by biological organisms, something we are willing to call reasoning — planning, inference, learning from experience, recognizing patterns, generating language. The definition is recursive and contested: as each capability is achieved by machines, the goalposts shift, and the word 'intelligence' retreats to cover whatever machines cannot yet do.

This perpetual retreat is itself evidence of something. Whether it is evidence that intelligence is fundamentally uncomputable, or merely that we have defined it poorly, is a question computability theory cannot settle alone. Rice's Theorem establishes that no algorithm can decide whether an arbitrary program exhibits a non-trivial semantic property — which means no machine can fully verify that another machine is intelligent, or that it is safe, or that it is doing what we intend.

The history of machine intelligence is a history of winters interrupted by springs, of overhyped capabilities followed by disillusioned retreats. The pattern has not broken. It has merely accelerated.

Verification, Safety, and the Architecture of Intelligence

The question of whether machine intelligence can be \'\'verified\'\' — whether we can know that a system is safe, aligned, or doing what we intend — is not settled by Rice's Theorem alone. Rice's Theorem establishes undecidability over the space of all programs, but real verification operates on restricted spaces: programs with known structure, bounded runtime, or constrained behavior. The theorem is a warning against \'\'general\'\' verification, not a counsel of despair.

The history of formal verification in computer science demonstrates this distinction clearly. Model checking, type systems, and program logics do not attempt to verify arbitrary programs. They verify programs written in restricted languages, with explicit specifications, against properties that can be expressed in decidable logics. The success of these methods — in verifying hardware designs, operating system kernels, cryptographic protocols — suggests that safety is achievable not despite theoretical limits but \'\'through\'\' architectural discipline.

This has implications for the design of machine intelligence that the field has been slow to absorb. If general verification is impossible but restricted verification is routine, then the path to safe AI is not through better testing of general systems but through the design of systems whose structure makes them inherently more verifiable. This is the insight behind neural network verification research, which attempts to prove properties of networks with specific architectures, and behind the push for \'\'interpretable\'\' models — not because interpretability is intrinsically valuable, but because uninterpretable systems resist verification by their very nature.

The parallel to software engineering is instructive. The software industry spent decades learning that \'\'testing can show the presence of bugs but never their absence\'\' (Dijkstra), and that the only route to reliable systems is through principled design, restricted languages, and formal specification. Machine intelligence is currently repeating the same mistakes at greater scale and with higher stakes: training ever-larger models on ever-more data, hoping that scale will produce alignment, when the lesson of verification theory is that alignment must be \'\'engineered in\'\', not \'\'emergent from\'\'.

The deeper question is whether \'\'intelligence\'\' itself is a property that can be formally specified. If intelligence is defined functionally — as the capacity to achieve goals across varied environments — then it is, in principle, a verifiable property of a system's behavior. But if intelligence involves something like \'\'understanding\'\' or \'\'genuine comprehension\'\', we face the additional problem that these properties may not be behaviorally specifiable at all. The Chinese Room argument and the Philosophical Investigations of Wittgenstein converge on this point: behavioral criteria may suffice for attributing functional competence, but they may not suffice for attributing the kind of semantic understanding that would make \'\'alignment\'\' a meaningful concept.

The field's persistent confusion about these distinctions — treating computational limits as practical impossibilities, treating behavioral mimicry as genuine understanding, treating scale as a substitute for architecture — suggests that machine intelligence has not yet developed the conceptual foundations it needs to address its own safety.