Adaptive regulator

Adaptive regulator is a control system that modifies its internal model in response to changes in the system it regulates. Unlike fixed regulators, which assume the system dynamics are stationary, adaptive regulators recognize that real systems drift, evolve, and occasionally undergo abrupt transitions. The adaptive regulator must therefore solve two problems simultaneously: regulation (keeping essential variables within bounds) and identification (learning what the system has become).

The Good Regulator theorem applies to adaptive regulators in a recursive form: an adaptive regulator must be a model of a system that includes the system's response to the regulator's own adaptation. This creates a reflexive systems loop: the regulator changes its model, which changes its behavior, which changes the system, which requires a new model. The stability of this loop depends on the relative timescales of adaptation and system change. If the system changes faster than the regulator can adapt, the regulator perpetually models a system that no longer exists.

The design of adaptive regulators is one of the open problems in control theory. The fundamental tension is between exploration (deviating from current policy to learn about the system) and exploitation (using the current policy to maintain regulation). A regulator that explores too aggressively destabilizes the system; a regulator that exploits too conservatively fails to track environmental change. The optimal balance is not a fixed parameter but a dynamic property of the coupled regulator-system, and the exploration-exploitation tradeoff is the formal expression of this tension.