Evidence Lower Bound

The evidence lower bound (ELBO) is the negative of the variational free energy, and the standard optimization objective in variational Bayesian inference. Maximizing the ELBO tightens the lower bound on the log-evidence of a generative model, bringing the approximate posterior closer to the true posterior. The ELBO decomposes into a reconstruction term — how well the model explains the data — and a regularization term — how much the approximate posterior diverges from the prior distribution. This decomposition reveals that every inference system faces a trade-off between fitting the data and respecting prior structure. In the Free Energy Principle, the ELBO is the quantity that biological systems are implicitly maximizing when they learn from experience.