Loading…
Liquid Clustering
1 posts about Liquid Clustering. Every summary links to the original.
Debunking 8 data layout myths: why Liquid Clustering outperforms partitioning
Partitioning has long been the standard physical layout for Hadoop- and Hive-era processing, but the post argues that changing Lakehouse workloads make fixed choices brittle. Liquid Clustering treats keys as guidance for file organization, allowing layouts to evolve, supporting low- and high-cardinality filters, maintaining file sizes, and enabling row-level concurrency. Using transaction-log metadata and per-column statistics, the post says modern Delta and Iceberg engines prune files rather than directories, and reports 35% lower clustering time, 22% faster queries, roughly 90% faster metadata-only deletes, and 23-minute planning for a 10-PB table. Conversion examples include Bolt’s 138% higher write throughput and up to 63% faster reads, while a 1.1-PB workload saw 5.9x faster queries, 86% fewer bytes read, and a 27% smaller table after clustering by date, hour, source, and id.
Jeffrey Gong, Yu Xu, Rahul Mahadev