Andrew Ng's New Document Extraction Service: Accuracy Challenges
2025-02-28
Andrew Ng's newly released document extraction service went viral on X, but Pulse's testing revealed significant issues with complex financial statements, including over 50% hallucinated values, missing negative signs and currency markers. The article argues that such errors can be catastrophic for industries relying on precise data, like finance. Pulse's solution combines traditional computer vision with proprietary table transformer models, achieving higher accuracy and lower latency, addressing the non-deterministic nature, poor spatial awareness, and slow processing speed of LLMs in document extraction.