Stochastic variational inference

Stochastic variational inference is an extension of variational inference designed for massive datasets. Where classical variational inference requires a full pass through the data at each optimization step, stochastic variational inference subsamples mini-batches and uses noisy gradients. The result is an algorithm whose per-iteration cost is independent of dataset size, making Bayesian inference feasible at the scale of modern machine learning.

The method was developed by Hoffman et al. (2013) and has become the standard computational framework for Bayesian deep learning, probabilistic programming, and latent variable modeling.

See also: Variational Inference, Approximate inference, Amortized inference, Machine learning