AI Code Review Agents: Helpful, But Not a Silver Bullet

Many AI code review agents have emerged, using LLMs to analyze code diffs and identify issues. The author experimented with Coderabbit, finding it occasionally catches errors missed by human reviewers, but also generates irrelevant or incorrect suggestions. Building a basic agent is relatively easy using the GitHub API and an OpenAI key. However, LLMs struggle to fully understand code, especially without broader codebase context, leading to inaccurate suggestions. The author concludes that creating a truly helpful agent requires addressing the LLM's understanding of code and leveraging codebase context effectively.