Solving the Equality Delete Problem in Apache Iceberg: RisingWave's Approach
2025-08-13
Apache Iceberg has become a hot topic in data infrastructure, but real-time data streaming presents challenges. Mainstream systems don't natively support writing Change Data Capture (CDC) directly into Iceberg. This article delves into Iceberg's two delete mechanisms: position delete and equality delete. Equality delete is suitable for streaming CDC but impacts query performance, and many mainstream engines don't support it. RisingWave solves this with a hybrid delete strategy (position deletes for in-batch updates, equality deletes otherwise) and schedulable compaction, enabling an end-to-end streaming CDC-to-Iceberg pipeline successfully deployed at companies like Siemens, significantly improving data availability.
Read more
Development