Semantic Unit Testing with LLMs: Introducing the `suite` Library

2025-05-05

This post introduces `suite`, a Python library leveraging Large Language Models (LLMs) for semantic unit testing. Unlike traditional unit tests, `suite` assesses the semantic correctness of functions by comparing their implementation against their docstrings. The author details `suite`'s workings, including prompt construction, handling function dependencies, and integration with pytest. While emphasizing that `suite` shouldn't replace traditional unit testing, it serves as a valuable complement, helping developers catch bugs early and improve test coverage. `suite` supports asynchronous testing and allows using local models, reducing costs and privacy concerns.

Read more
Development

Jupyter Notebooks and LLMs: An Unexpected Cost Optimization

2025-01-21

The author used LLMs for coding assistance, initially with a ContinueDev + OpenRouter setup. Costs unexpectedly soared. Investigation revealed that Jupyter Notebook (.ipynb) files contain significant hidden content (code outputs, metadata, base64-encoded images), leading to a large token count and increased LLM call costs. The solution was converting .ipynb files to .py files and removing base64-encoded images. This reduced costs by 94% and latency as well. The post highlights the importance of mindful input when using LLMs and recommends regularly checking costs.

Read more
Development Cost Optimization