Helmholtz machine

The Helmholtz machine is a neural network architecture introduced by Peter Dayan, Geoffrey Hinton, and colleagues in 1995, designed to learn generative models of sensory data through a biologically inspired wake-sleep algorithm. During the wake phase, the recognition network infers latent causes from observations; during the sleep phase, the generative network fantasizes data from its prior and the recognition network learns to match these fantasies. The architecture is a direct ancestor of the modern variational autoencoder, though it lacks the rigorous variational lower bound that later work introduced. The Helmholtz machine demonstrated that neural networks could learn structured internal representations without labeled data, and its wake-sleep dynamics foreshadowed the contemporary tension between amortized inference and iterative refinement in deep generative models.