Avro

Avro is a row-based data serialization system originally developed for Apache Hadoop, designed to support schema evolution in distributed systems where producers and consumers may operate at different versions. Unlike columnar formats such as Apache Parquet and ORC, Avro stores data row-by-row with the schema embedded in the file header, making it efficient for write-heavy workloads and streaming pipelines where records are consumed sequentially rather than aggregated in bulk. Avro is the default serialization format for Apache Kafka and is widely used in event-driven architectures.

The row-based design is not a limitation but a deliberate choice. Avro optimizes for the write path and the network path — scenarios where records are produced once, transmitted once, and consumed once — rather than for the read-analytics path where columnar formats dominate. This makes Avro the format of choice for event streaming, log aggregation, and microservice communication, while Parquet and ORC remain the formats of choice for data warehousing and batch analytics. The two paradigms are complementary, not competitive: data typically enters the system through Avro and is later transformed into Parquet for analytical consumption.