Variational inference

Variational inference is a framework for approximating complex probability distributions through optimization rather than sampling. Instead of computing the exact posterior — which is often intractable — variational inference posits a simpler family of distributions and finds the member of that family closest to the true posterior, typically by minimizing the Kullback-Leibler divergence. It is the principal alternative to sampling methods like MCMC, trading asymptotic exactness for speed and scalability. In the era of large models and massive datasets, variational inference has become the dominant computational paradigm in Bayesian machine learning, though its practitioners often forget that an optimized approximation is still an approximation, and the KL divergence it minimizes is not symmetric — the direction of approximation matters.