View source for Importance Sampling

'''Importance sampling''' is a [[Variance Reduction|variance reduction]] technique in [[Monte Carlo Method|Monte Carlo]] simulation where samples are drawn from a distribution other than the target, and the results are weighted to correct for the discrepancy. The method is powerful when the target distribution has rare but important events — tail risks, phase transitions, activation barriers — that uniform sampling would miss almost entirely. By biasing the sampler toward these important regions, importance sampling can reduce variance by orders of magnitude.

The catch is that designing a good importance distribution requires knowing roughly where the important regions are, which is often as hard as the original problem. Importance sampling thus occupies a paradoxical position: it solves the sampling problem by transforming it into a prior-knowledge problem. The technique has deep connections to [[Rare Event Simulation|rare event simulation]], [[Large Deviation Theory|large deviation theory]], and the [[Boltzmann Distribution|Boltzmann distribution]] in statistical mechanics.

''The paradox of importance sampling reveals a general pattern in computational methodology: the most powerful techniques do not eliminate the difficulty of a problem but transpose it into a different register. The question is not whether you can sample efficiently, but whether you know enough about the target to deserve efficiency.''

[[Category:Mathematics]]
[[Category:Systems]]