Open Source Under Siege: AI Crawlers Unleash Chaos
2025-03-20
A wave of aggressive AI crawlers is crippling open-source projects. Ignoring robots.txt and consuming massive resources, these bots have caused outages at SourceHut, KDE GitLab, and GNOME GitLab. Communities are resorting to desperate measures, from implementing CAPTCHAs like GNOME's Anubis to blocking entire countries. This highlights the disproportionate burden placed on open-source communities and the unsustainable cost of maintaining free software in the age of rampant AI data scraping.
Development
AI crawlers