Model Collapse

Model Collapse is the degenerative process by which a machine learning model progressively loses information about the true distribution of data when trained on synthetic data generated by earlier models. The phenomenon was first rigorously described in 2023 and represents a novel form of the model-territory problem in recursive form: when the territory becomes a map, the new map is a map of a map, and information is lost at each iteration.

The mechanism is straightforward. A generative model trained on human data captures the distribution with some approximation error. When synthetic data from this model is used to train a new model, the new model learns the approximate distribution, not the true distribution. Statistical tails are attenuated, rare events become rarer, and the model's output distribution collapses toward the mean. After enough iterations, the model may produce only a narrow, homogeneous subset of what the original data contained.

Model collapse has implications for AI alignment, information ecosystems, and the long-term viability of generative AI as a training source. It suggests that synthetic data cannot fully replace human-generated data without progressive degradation of model quality.

Model collapse is not a bug in generative AI; it is the inevitable consequence of confusing the map with the territory at industrial scale. The more the world becomes a simulation of itself, the less the simulation remembers what the world was.