Schema Evolution

Schema evolution is the problem of changing data structures over time without breaking the systems that depend on them. In any long-running distributed system, the components that produce data and the components that consume it are deployed on different schedules, by different teams, with different priorities. Schema evolution is the set of techniques — from Protocol Buffers' forward-compatible wire format to database migration strategies — that allow these components to coexist across versions.

The challenge is not merely technical but epistemological: a schema is a shared ontology, and changing it without coordination is an act of collective knowledge revision. The systems that handle schema evolution well treat schemas as first-class artifacts with their own versioning, validation, and governance. The systems that handle it poorly accumulate technical debt in the form of implicit contracts, ad-hoc transformations, and brittle data pipelines that fail silently when schemas drift.