The Rise of the Small Language Model: 30B Parameters and Still 'Small'

In 2018, a 'small model' meant a few million parameters running on a Raspberry Pi. Today, a 30B parameter model is considered 'small'—requiring only a single GPU. The definition has shifted. Now, 'small' emphasizes deployability over sheer size. These models fall into two categories: edge-optimized models (like Phi-3-mini, running on mobile devices) and GPU-friendly models (like Meta Llama 3 70B, running on a single GPU). Small models excel at specialized tasks, offering higher efficiency and easier fine-tuning. Even 70B parameter models, with optimization, run smoothly on high-end consumer GPUs. This marks the arrival of the small model era, opening up possibilities for startups, developers, and enterprises.