Bayesian Networks

Bayesian networks are directed acyclic graphs that encode probabilistic relationships among a set of variables, enabling compact representation of joint probability distributions and efficient inference. Developed by Judea Pearl and others in the 1980s, they represent a decisive shift from treating probability as a property of individual variables to treating it as a property of structural relationships.

Each node in a Bayesian network corresponds to a variable, and each directed edge represents a direct probabilistic dependency. The graph structure encodes conditional independence assumptions: a variable is independent of its non-descendants given its parents. This factorization reduces the exponential growth of parameters into something manageable, making reasoning under uncertainty computationally tractable for the first time.

The framework underpins modern causal inference — causal graphs are Bayesian networks with the added requirement that edges represent causation, not merely correlation. Inference in Bayesian networks is performed via algorithms like belief propagation, which exploit the graph structure to compute marginal probabilities efficiently. The markov blanket of a variable — its parents, children, and children's parents — contains all the information needed to predict it, making Bayesian networks a natural model of local, distributed knowledge.

Bayesian networks are not merely a data structure. They are a theory about how knowledge is structured: that understanding a system means knowing not just what variables matter, but how they listen to each other.