Semantic Unit Testing with LLMs: Introducing the `suite` Library

This post introduces `suite`, a Python library leveraging Large Language Models (LLMs) for semantic unit testing. Unlike traditional unit tests, `suite` assesses the semantic correctness of functions by comparing their implementation against their docstrings. The author details `suite`'s workings, including prompt construction, handling function dependencies, and integration with pytest. While emphasizing that `suite` shouldn't replace traditional unit testing, it serves as a valuable complement, helping developers catch bugs early and improve test coverage. `suite` supports asynchronous testing and allows using local models, reducing costs and privacy concerns.