Why LLMs Fail at Creativity: The Surprise Problem

Large Language Models (LLMs) struggle with comedy, art, journalism, research, and science because they're fundamentally designed to avoid surprises. The author argues that humor, good stories, and impactful research all hinge on surprising elements that are ultimately inevitable in hindsight. LLMs, trained to predict the next word, minimize surprise, resulting in predictable and uninspired output. Improving LLMs requires a shift towards a curiosity-driven architecture that actively seeks out and interprets surprising truths, rather than simply avoiding them.