Meta's Llama 4: Benchmarking Scandal Rocks the AI World

Meta's recently released Llama 4 family of large language models, particularly the Maverick version, initially stunned the AI world with its impressive benchmark performance, outperforming models like OpenAI's GPT-4o and Google's Gemini 2.0 Flash. However, discrepancies quickly emerged between the benchmark version and the publicly available model, leading to accusations of cheating. Meta admitted to using a specially tuned version for benchmarking and has since added the unmodified Llama 4 Maverick model to LMArena, resulting in a significant drop in ranking. This incident highlights transparency issues in large model benchmarking and prompts reflection on model evaluation methodologies.