Enum of Arrays: A Novel Data Structure for Efficient Data Processing
2024-12-21
This article introduces a data structure called "Enum of Arrays" (EoA), similar to the popular "Struct of Arrays" (SoA), but with enums at its core. EoA packs multiple enum values into an array, using a single tag to identify the array's type. This reduces memory usage and branch prediction overhead, leading to more efficient data processing, particularly beneficial for SIMD optimization. The article uses the database system TigerBeetle as an example, illustrating how EoA enables efficient batch processing by effectively separating the control plane and data plane, resulting in significantly improved performance.
Read more