Sub-100MB LLM Now Pip-installable: Introducing llm-smollm2

A new plugin, llm-smollm2, bundles a quantized SmolLM2-135M-Instruct LLM under 100MB, making it pip-installable. The author details the creation process, from finding a suitable sub-100MB model (limited by PyPI size restrictions) to suppressing verbose logging from llama-cpp-python and packaging for PyPI. While the model's capabilities are limited, it's presented as a valuable learning tool for understanding LLM technology.
Read more