Making My Debug Build 100x Faster: SIMD and Dedicated Silicon to the Rescue
While developing a C-based torrent application, the author encountered a slow debug build startup time (20-30 seconds). This was mainly due to the program's startup verification of SHA1 hash values for each downloaded file piece, significantly slowed by Address Sanitizer in debug mode. The article explores several optimization techniques, including using SIMD instructions and the CPU's dedicated SHA acceleration hardware. Ultimately, using Intel's SHA extension instruction set reduced startup time to 866.9 milliseconds, achieving a 100x performance improvement. The article details the implementation and performance comparison of various optimization approaches, discussing compiler optimization and the impact of disk I/O.
Read more