Webtagr - Technology News Summarizer

Exploring an ORM for OLAP Databases: The Moose OLAP Approach

2025-08-17

Modern applications increasingly rely on user-facing analytics and AI powered by aggregations across large datasets, pushing developers towards analytical databases like ClickHouse. This article explores the possibilities and challenges of building an ORM for OLAP databases. Extending existing OLTP ORMs to OLAP is problematic due to semantic differences. Moose OLAP, an open-source project, attempts to provide an ORM-like interface for ClickHouse. It borrows from the strengths of OLTP ORMs but adapts to OLAP specifics, such as handling NULL values and uniqueness constraints differently. Moose OLAP emphasizes schema-as-code, provides OLAP-native semantics and defaults, and supports versioned migrations to handle the dynamic schema changes inherent in OLAP environments.

(clickhouse.com)

Development

A Nasty Postgres Bug in Logical Replication Slot Creation, and How We Fixed It

2025-07-15

The ClickPipes team encountered a perplexing bug while creating logical replication slots in PostgreSQL: a query that should have taken seconds was taking hours and couldn't be terminated. Investigation revealed a Postgres bug where, on read replicas, creating a logical replication slot would get stuck in a long sleep loop while waiting for primary transactions to finish, making it impossible to interrupt. The team submitted a patch to the Postgres community adding an interrupt check, effectively resolving the issue. This case highlights how even mature database systems can harbor unexpected edge cases, and the vital role of open-source community collaboration in resolving them.

(clickhouse.com)

Development Logical Replication

ClickHouse Cloud's 100PB Observability Platform: The Evolution of LogHouse

2025-06-21

In a year, ClickHouse Cloud's internal logging platform, LogHouse, grew from 19 PiB to over 100 PiB, and the number of rows increased from 40 trillion to 500 trillion. To handle a 20x surge in event volume, the LogHouse team developed SysEx, a custom exporter that reduced CPU usage to less than 10% of the previous requirement. SysEx directly copies data from ClickHouse system tables, bypassing the bottleneck of OpenTelemetry parsing and marshaling. Concurrently, LogHouse integrated HyperDX, ClickHouse's native observability UI, providing seamless exploration, correlation, and root cause analysis.

(clickhouse.com)

Tech

ClickHouse Raises $350M Series C to Fuel AI-Native Applications

2025-05-29

Real-time analytics database ClickHouse announced a $350 million Series C funding round, bringing its total funding to over $650 million. This investment will fuel product development, global expansion, and partnerships supporting the next wave of AI-native applications. ClickHouse's high-performance, columnar storage engine enables interactive analytical queries on massive datasets with minimal latency, powering AI/ML applications, real-time analytics, cloud data warehousing, and observability workloads. Boasting over 300% year-over-year growth and serving 2,000+ customers including Anthropic, Tesla, and Mercado Libre, ClickHouse addresses the challenge of building real-time data platforms for the AI era, positioning itself as the default engine for next-generation intelligent data products.

(clickhouse.com)

Tech

ClickHouse's Lazy Materialization: A 1500x Speed Boost

2025-04-22

ClickHouse achieves a 1500x speed improvement using a new optimization called "lazy materialization." This technique delays reading column data until it's actually needed, dramatically reducing unnecessary I/O. The article uses the Amazon customer reviews dataset to illustrate how lazy materialization, combined with other I/O optimizations like columnar storage, sparse primary indexes, and PREWHERE, reduces a query's execution time from 219 seconds to 139 milliseconds. Lazy materialization is particularly effective for Top N queries, providing significant performance gains without altering the SQL.

(clickhouse.com)

Tech Database Optimization Lazy Materialization

ClickHouse Embraces Rust: A Challenging Integration Journey

2025-04-09

ClickHouse, originally written in C++, embarked on a journey to integrate Rust to attract more developers and expand its capabilities. The article details this process, from initially choosing the BLAKE3 hash function as a pilot project, to integrating the PRQL query language and the Delta Lake library. The journey encountered numerous challenges, including build system integration, memory management, error handling, and cross-compilation issues. Despite problems like bugs in Rust libraries, excessively large symbol names, and interoperability issues with C++ code, the ClickHouse team overcame these obstacles, successfully integrating Rust into the project and paving the way for future development.

(clickhouse.com)

Development