Ten years of ClickHouse in open source

Summary

ClickHouse’s tenth-anniversary history describes its path from an internal web-analytics experiment to an open-source analytical database with more than 2,000 contributors. The original problem was processing rapidly growing, real-time logs while answering flexible reports quickly; tested MySQL extensions and standalone analytical databases did not meet the stated scale of 100 billion records per day and 500 columns. The project built column-oriented storage, lightweight compression, custom aggregation structures, background merges, and query support for grouping, aggregation, filtering, and sorting. Released publicly on June 15, 2016, ClickHouse is presented as a level-three open-source project with public contribution guidelines, code review, testing, CI, documentation, and a roadmap, while contributors receive credit even when implementations are substantially rewritten.

Takeaways

1. 1

   The initial OLAPServer prototype stored each integer column in daily, website-specific binary files and exposed grouping, aggregation, filtering, and sorting through an API.

2. 2

   ClickHouse used custom C++ aggregation structures with add, update, merge, serialization, and deserialization methods, allowing partially aggregated data to be merged on reads.

3. 3

   The project defines level-three open source as including open contribution guidelines, a task tracker, code review, a roadmap, testing and CI, a release cycle, user support, and documentation.