The Fastest Phrase Search Algo Using the Most Unhinged AVX-512 Instruction
2025-01-26
This blog post details the author's journey in creating a blazing-fast phrase search algorithm leveraging AVX-512 instructions, particularly the obscure `vp2intersectq`. Benchmarking against Meilisearch on a 3.2M document MS MARCO dataset showed performance improvements up to 1600x. The author meticulously covers algorithm design, index optimization, SIMD optimizations, microarchitectural differences between AMD and Intel CPUs, code alignment, and more.
Development
phrase search