ClickHouse Cloud's 100PB Observability Platform: The Evolution of LogHouse

In a year, ClickHouse Cloud's internal logging platform, LogHouse, grew from 19 PiB to over 100 PiB, and the number of rows increased from 40 trillion to 500 trillion. To handle a 20x surge in event volume, the LogHouse team developed SysEx, a custom exporter that reduced CPU usage to less than 10% of the previous requirement. SysEx directly copies data from ClickHouse system tables, bypassing the bottleneck of OpenTelemetry parsing and marshaling. Concurrently, LogHouse integrated HyperDX, ClickHouse's native observability UI, providing seamless exploration, correlation, and root cause analysis.