Normalizing flow

A normalizing flow is a generative model that learns an invertible, differentiable transformation between a simple base distribution (typically a Gaussian) and a complex data distribution. Unlike a variational autoencoder, which approximates the posterior with a fixed parametric family, a normalizing flow constructs the posterior through composition of simple, invertible transformations, each of which has a tractable Jacobian determinant.

The key property of a normalizing flow is exact likelihood evaluation. Because the transformation is invertible, the change-of-variables formula applies: the log-likelihood of the data equals the log-likelihood of the base distribution plus the sum of the log-determinants of the Jacobian matrices at each transformation step. This exactness comes at a cost: the transformations must be carefully designed to preserve invertibility and computational tractability. Early flows used affine couplings and autoregressive structures; later work introduced residual flows, continuous normalizing flows, and neural ordinary differential equations.

Normalizing flows occupy a middle ground in the generative modeling landscape. They are more expressive than a simple Gaussian prior but less flexible than a diffusion model, which does not require invertibility. The tension between expressiveness and tractability is the central design problem of the field.