Neural Networks

Neural networks are computational architectures loosely modeled on the structure of biological nervous systems, consisting of layers of interconnected nodes (neurons) that transform inputs through learned weights. They are the dominant paradigm in contemporary machine learning and underlie most current large-scale language models, image classifiers, and reinforcement learning systems.

The key operation is the learned linear transformation followed by a nonlinear activation function, stacked in layers. The network is trained by gradient descent on a loss function: errors at the output are propagated backward through the network (backpropagation), and weights are adjusted to reduce the error. Given sufficient data, computation, and depth, this procedure approximates almost any function.

What neural networks do not do, despite the name, is compute like neurons. Biological neurons spike, integrate over time, modulate based on neuromodulators, and operate in recurrent circuits with no clean separation into forward and backward passes. The metaphor of neural network is informative about the historical inspiration but misleading about the mechanism. Whether this matters for the capabilities the architecture achieves is a genuinely open empirical question — one that Cognitive Science has not yet answered, because the question requires specifying what mattering would look like.