LLMs in 2024: A Year of Breakthroughs and Challenges

2024 witnessed a remarkable evolution in Large Language Models (LLMs). Multiple organizations surpassed GPT-4's performance, leading to dramatically increased efficiency—even enabling LLM execution on personal laptops. Multimodal models became commonplace, with voice and video capabilities emerging. Prompt-driven app generation became a commodity, yet universal access to top-tier models lasted only months. While 'agents' remained elusive, the importance of evaluation became paramount. Apple's MLX library excelled, contrasting with its underwhelming 'Apple Intelligence' features. Inference-scaling models rose, lowering costs and improving environmental impact, but also raising concerns about the environmental consequences of new infrastructure. Synthetic training data proved highly effective, but LLM usability remained challenging, knowledge distribution remained uneven, and better critical evaluation is needed.