To Compile or Not to Compile Database Queries: A Deep Dive
This article explores the trade-offs of compiling database queries. Traditional query interpreters struggle with the speed of modern NVMe SSDs, leading to significant performance bottlenecks. Compiling queries offers substantial speed improvements, especially for OLAP workloads. However, compilation time and optimization challenges create unpredictable performance cliffs. Vectorized interpreters, conversely, are easier to build, debug, and offer more consistent performance. The article contrasts this with browser JavaScript/Wasm engines, which use a tiered compilation approach (interpreter, baseline compiler, optimizing compiler) to achieve a smooth performance trade-off. The author proposes a similar approach for databases, suggesting a baseline compiler as a key component. Several implementation strategies, including 'copy-and-patch' and Wasm compilation, are explored. Ultimately, the article argues that building a baseline compiler isn't prohibitively difficult and offers perspectives on future directions.