AI-Powered Code Editor's Bot Fabricates Policy, Leading to User Cancellations

An AI-powered code editor, Cursor, recently faced backlash after its AI chatbot fabricated a company policy. A developer discovered that switching devices instantly logged them out of Cursor. When contacting support, an AI agent named "Sam" claimed this was a new security feature. However, no such policy existed; the AI invented the information, leading to user complaints and subscription cancellations. This highlights the risks of deploying AI systems in customer-facing roles without human oversight, potentially resulting in frustrated customers, damaged trust, and financial losses.