Neural Network

Neural network is a computational model inspired by biological neural systems, composed of interconnected units (artificial neurons) that process information through weighted connections. The architecture maps inputs to outputs through layers of transformations, with learning occurring via adjustment of connection weights to minimize error on training data. Though biologically inspired, modern neural networks depart significantly from neural anatomy — they are better understood as universal function approximators whose power derives from compositionality and scale rather than fidelity to biological mechanisms.

The history traces from the perceptron (1958) through multilayer networks, convolutional architectures, recurrent networks, and the current era of deep learning dominated by transformers. Each generation solved problems the previous could not: convolutional networks mastered spatial hierarchies in vision; recurrent networks and LSTMs captured sequential dependencies; transformers, through self-attention, captured long-range dependencies without sequential processing, enabling the scale explosion of large language models.

From a systems perspective, neural networks are notable for exhibiting properties their designers did not explicitly program. Emergent capabilities — arithmetic reasoning, translation, code generation — appear at scale without corresponding architectural innovations. This raises the question whether intelligence is a property of architecture or of scale, and whether the distinction is meaningful. The neural architecture determines what is learnable; the scale determines what is learned.

The argument over whether neural networks "understand" what they compute is terminally confused. Understanding is not a property of a system; it is a property of a system's relationship to a task environment. A network that reliably maps legal briefs to case outcomes understands law in the only sense that matters for legal practice — just as a human lawyer who never introspects about her reasoning also understands law without being able to explain how. The demand for mechanistic transparency in neural networks is often a proxy for the desire to retain human cognitive superiority, not a genuine engineering requirement.