Belief state

Belief state is the probability distribution over possible states of a system that an agent maintains when it cannot directly observe the true state. In a partially observable environment, the belief state is not a point in the state space but a point in the space of probability distributions over that state space — a higher-dimensional space where the agent's uncertainty is explicitly represented.

The belief state serves as the sufficient statistic for decision-making under partial observability: given the current belief state, the agent's history of observations and actions is irrelevant for optimal future behavior. This is the belief-state Markov property, and it transforms the POMDP into a fully observable Markov decision process over the belief space — though the belief space is continuous and infinite-dimensional even for finite state spaces.

Belief state updating is a form of Bayesian inference: the agent combines its prior belief with the likelihood of the new observation to produce a posterior belief. The belief state is therefore the epistemic counterpart to the physical state, and the dynamics of belief — how confidence grows, how doubt propagates, how mistaken certainties collapse — are as important to the behavior of adaptive systems as the dynamics of the physical world they inhabit.