Restricted Boltzmann Machine

Restricted Boltzmann Machine (RBM) is a stochastic neural network with a bipartite architecture — visible units connected to hidden units, with no connections within either layer. This restriction makes inference tractable: unlike the full Boltzmann Machine, the hidden units are conditionally independent given the visible units. Introduced by Geoffrey Hinton, the RBM became the foundational building block of the Deep Belief Network.

RBMs learn probability distributions over their inputs using contrastive divergence, a fast approximation to maximum likelihood learning. The learned hidden representations capture statistical regularities — edges, phonemes, semantic features — and can be used for classification, dimensionality reduction, or collaborative filtering. The bipartite structure enforces a distributed code in which combinatorial structure emerges from the activation patterns of individual units.

Statistical Mechanics Foundations

The RBM is not merely a machine learning algorithm. It is a statistical mechanics model in computational clothing. The architecture is identical to the Ising model with bipartite interactions: visible units are spins in one sublattice, hidden units are spins in the other, and the energy function is the Hamiltonian of the system. The probability distribution over visible configurations is the Boltzmann distribution, and the learning rule is a form of maximum entropy inference.

This connection is not metaphorical. The contrastive divergence algorithm that trains RBMs is a Monte Carlo approximation to gradient descent on the log-likelihood — a procedure that would be immediately recognizable to a physicist computing thermodynamic averages. The positive