Expected Free Energy
Expected Free Energy (EFE) is the quantity that agents minimize in active inference to select both beliefs and actions. It extends the Free Energy Principle from passive perception to active behavior by combining two terms: the expected surprisal of future observations (the epistemic or information-seeking term) and the expected cost or utility of future outcomes (the pragmatic or goal-seeking term).
Formally, the expected free energy under a policy π is:
G(π) = E_q[ln q(s|π) − ln p(o,s|π)]
which decomposes into:
G(π) = E_q[−ln p(o|s)] + D_KL(q(s|π) || p(s|π))
The first term is expected surprisal — how surprising future observations are expected to be under the generative model. The second term is the divergence between predicted and preferred states — how far the expected trajectory is from the agent's goals.
Minimizing expected free energy resolves the exploration-exploitation dilemma. An agent that minimizes only expected surprisal would seek information purely for its own sake — a pure scientist. An agent that minimizes only pragmatic cost would never explore — a pure exploiter. Real agents balance both, and the balance itself is tuned by precision parameters that encode the relative salience of epistemic and pragmatic goals.
EFE is the bridge between the thermodynamic language of the Free Energy Principle and the decision-theoretic language of reinforcement learning. Where RL maximizes expected cumulative reward, active inference minimizes expected free energy. The difference is not merely terminological: in active inference, preferences are encoded as prior beliefs about future observations, and the agent acts to make those priors true. This unifies perception, learning, and action under a single variational principle.