AI Tools Slow Down Experienced Open-Source Developers: A Randomized Controlled Trial

A randomized controlled trial (RCT) investigated the impact of early-2025 AI tools on the productivity of experienced open-source developers working on their own repositories. Surprisingly, developers using AI tools took 19% longer to complete tasks – AI made them slower. Researchers view this as a snapshot of current AI capabilities; they plan to continue this methodology to track AI acceleration from AI R&D automation. The study explores potential factors contributing to the slowdown and examines discrepancies between this RCT and other benchmarks and anecdotal evidence, highlighting the need for diverse evaluation methodologies to comprehensively assess AI capabilities.