Apple's AI Lags Behind Competitors, Benchmarks Reveal

Apple unveiled updates to its AI models powering its intelligence features across iOS, macOS, and more. However, internal benchmarks show these models underperforming older rivals like OpenAI. Apple's 'Apple On-Device' model, running offline, performed comparably to, but not better than, similarly-sized models from Google and Alibaba. Its more powerful 'Apple Server' model lagged behind OpenAI's year-old GPT-4. Surprisingly, Meta's Llama 4 Scout outperformed Apple Server in image analysis, despite Llama 4 Scout's generally lower performance compared to leading models. These results support reports of Apple's AI division struggling to keep pace, explaining delays like the indefinitely postponed Siri upgrade and prompting lawsuits from dissatisfied customers. Despite this, the new models offer improved tool use and efficiency and support approximately 15 languages.