Loading…
Clickhouse
High-performance open-source columnar database management system designed for real-time analytics and data warehousing at scale
- 235 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Appcues delivers personalized customer engagement with ClickHouse Cloud
Appcues replaced an analytics stack that included Snowflake and Airflow after rising scale pushed queries to 20 seconds or more and limited reports to 10-minute windows. The company evaluated ClickHouse Cloud on AWS for real-time segmentation, exporting Snowflake data to Parquet in S3 and testing complex email segments with nested conditions, historical events, and negative logic. Production events now pass through Kafka and a clickwriter consumer, while a clickclient library built on Elixir’s Ecto generates controlled ClickHouse queries; SharedReplacingMergeTree, table engines, and materialized views were tuned during a one-year migration. Across 1.31 PB and 410 billion events, P95 queries fell below two seconds, ingestion latency dropped to about five seconds, and analytics costs decreased 23%, while new funnel reports and AI-powered insights were added.
What's new in Postgres Managed by ClickHouse: RBAC, Terraform, ClickPipes, extensions, and more
Postgres managed by ClickHouse, which entered public beta in May, received a broad update covering console operations, per-service RBAC, Terraform, Stripe provisioning, ClickPipes migrations, billing, capacity reservations, and extensions. The redesigned console adds service health, resource, and query views, while RBAC provides View and Manage roles across the console and public REST API; the alpha Terraform provider covers lifecycle operations including resizing, HA, replicas, restore, and password rotation. Postgres-to-Postgres ClickPipes now automate schema migration, usage is integrated into ClickHouse Cloud’s metering and billing, and AWS capacity reservations are allocated dynamically across availability zones. Underneath, pg_clickhouse and pg_stat_ch moved to the header-only clickhouse-c library, removing a C++ exception conflict, using Postgres memory contexts, and reducing the x86-64 Linux extension binary from 4.9 MB to 1.4 MB; the update shipped less than a month after beta, with Terraform and extensions still evolving.
June 2026 newsletter
ClickHouse’s June 2026 newsletter marks the project’s ten years as open source, noting more than 2,000 contributors and community projects alongside engineering updates. It profiles Vasily Chekalkin’s 26.5 improvements to WebAssembly user-defined functions, including the DETERMINISTIC keyword and visibility in system.functions, and covers native Jaeger storage, the ClickCannon benchmark tool, and Open House announcements. The 26.5 release reports Top-N optimizations that make ORDER BY … LIMIT through JOIN 20.4× faster with 175× less memory on TPC-H SF100, while GROUP BY … LIMIT is 11.9× faster with 185× less peak memory. Other featured systems include a 15-million-rows-per-hour KPI pipeline, join-engineering work yielding a 26× improvement on join-heavy TPC-H workloads, and community guidance on inserts, columnar storage, ArrowStream, and RAG ingestion.
GCP Pub/Sub connector for ClickPipes is now in Private Preview
ClickHouse has announced a GCP Pub/Sub connector for ClickPipes, now in Private Preview, to stream GCP infrastructure logs, application events, and operational signals into ClickHouse Cloud in real time. Previously, routing data from Pub/Sub into ClickHouse required GCS or a Dataflow pipeline; the connector instead manages subscription lifecycle, schema inference, and message delivery after a topic and target table are configured. It supports JSON, Avro, and Protobuf, Pub/Sub Schema Registry integration, automatic decompression, attribute-based filtering, and start positions of latest, earliest, or a specific timestamp. ClickPipes supports horizontal and vertical scaling, configurable replicas, automatic retries, built-in metrics and logs, and programmatic management through OpenAPI and the ClickHouse Terraform Provider. The connector is available upon request and free during Private Preview, with access offered through a waitlist or a ClickHouse Cloud account manager.
The open ecosystem around ClickHouse
ClickHouse’s open ecosystem spans clients, dashboards, ingestion pipelines, projects built on its engine, and agents that query it, with 2,600+ contributors, 240,000+ commits, and hundreds of millions of downloads cited. It describes open standards including OpenTelemetry for observability, Model Context Protocol (MCP) for AI tooling, and Iceberg and Delta Lake for lakehouse data, plus Iceberg REST catalogs, Apache Polaris, and Unity Catalog. The surrounding stack includes ClickStack, chDB, Langfuse, LibreChat, official language clients, a Kubernetes operator, Postgres bridges such as PeerDB and pg_clickhouse, and integrations with Airflow, dbt, Grafana, Fivetran, and Sigma. Specific updates include full Iceberg v2 support, Iceberg writes in 25.8, Paimon in 25.10, chDB v4’s DataStore API, and a Python client v1.0 with native async and 4x faster cold starts. The post frames this openness as supporting flexible tooling and community-led growth, reporting more than 200 integrations while noting that the ecosystem’s next direction remains unknown.
92x faster queries: How Open Electricity uses ClickHouse to track Australia’s energy transition in real time
Open Electricity uses ClickHouse to serve live Australian energy data through a public platform and API spanning roughly one billion records. Migrating from Postgres and TimescaleDB addressed queries that could take more than five minutes, reducing a benchmark to just over three seconds and to 60 milliseconds with a materialized view. The new setup keeps heavy relational workloads in Postgres while ClickHouse handles analytics, using ReplacingMergeTree base tables and more than 20 materialized views for different aggregation patterns. It also uses substantially less hardware than the previous configuration: four cores, four GB RAM, and 200 GB disk versus 16 cores, 32 GB RAM, and 1.6 TB, while serving data added daily and API users.
What a difference 10 years of open source makes
ClickHouse marks ten years since becoming open source under the Apache 2.0 license, crediting more than 2,600 contributors and a decade of monthly releases for its growth into one of the world’s most widely deployed analytical databases. Recent engineering work spans joins, updates and deletes, JSON, full-text search, vector search, query acceleration, integrations, and developer tooling. Join-focused changes made the TPC-H SF100 workload 26× faster between versions 22.4 and 26.4, while statistics-based reordering reduced one six-table query from 3,903.7 seconds to 2.7 seconds. The project also publishes open benchmarks, including ClickBench, JSONBench, CostBench, and PostgresBench, with scripts, configurations, data, and results intended for independent validation. Its next phase continues to emphasize community contributions and broader ways to use ClickHouse.
Ten years of ClickHouse in open source
ClickHouse’s tenth-anniversary history describes its path from an internal web-analytics experiment to an open-source analytical database with more than 2,000 contributors. The original problem was processing rapidly growing, real-time logs while answering flexible reports quickly; tested MySQL extensions and standalone analytical databases did not meet the stated scale of 100 billion records per day and 500 columns. The project built column-oriented storage, lightweight compression, custom aggregation structures, background merges, and query support for grouping, aggregation, filtering, and sorting. Released publicly on June 15, 2016, ClickHouse is presented as a level-three open-source project with public contribution guidelines, code review, testing, CI, documentation, and a roadmap, while contributors receive credit even when implementations are substantially rewritten.
Cisco Talos empowers threat researchers while reducing TCO by 75% with ClickHouse Cloud
Cisco Talos operates a threat-intelligence reputation service that classifies file hashes from Cisco product telemetry, enabling researchers to make security decisions about benign and potentially malicious artifacts. Previously, a custom pipeline moved data from Databricks Delta and S3 through EKS, where staging and aggregated tables supported an API; the self-managed design retained about a petabyte of compressed, replicated data on 32 high-compute nodes. The team rebuilt the system in four phases, using Terraform, SSO/SAML, sharded ClickPipes for a 12-month backfill, shadow traffic, and a validated cutover to ClickHouse Cloud on AWS. The resulting architecture uses EventBridge, SQS, ClickPipes, SharedMergeTree, object storage, and AWS PrivateLink, delivering less than a minute of queue lag, zero data loss, a 100% response match across more than 1,000 hashes, zero downtime, roughly 90% lower storage costs, and about 75% lower TCO while reclaiming over 300 engineer hours annually.
ClickHouse achieves AWS Retail Competency
ClickHouse has achieved the AWS Retail Competency in the Advanced Data Insights category, a designation recognizing validated technical expertise and demonstrated customer success in retail real-time analytics. Retail workloads described here combine high-volume, high-cardinality, high-concurrency data with hard freshness requirements, including inventory, pricing, personalization, and peak seasonal demand. The post attributes the platform's fit to compressed columnar storage, parallel queries, continuous Kafka and Amazon S3 ingestion through ClickPipes, compute-compute separation, and AWS support for private networking, multi-region deployment, and Marketplace billing. Examples include Mercado Libre's 90-TB-per-day observability ingestion and trace lookups reduced from 15–20 minutes to about two seconds, while other customers report cost, speed, and analyst-efficiency gains.
Introducing Postgres to Postgres ClickPipes in ClickHouse Cloud
Postgres-to-Postgres migrations between providers can require manual schema handling, data movement, CDC, and cutover coordination, with risks around database objects and long-running loads. Postgres to Postgres ClickPipes, now in public beta, provides a fully managed migration service in ClickHouse Cloud designed to reduce operational overhead and support minimal downtime. It automates schema dump and restore, including secure connections, role conflicts, version differences, transactional application, retries, and UI or notification-based error reporting, while PeerDB supplies parallel initial loads and CDC capabilities. Independent pull and push processes use retries and alerting for transient errors, and the roadmap includes features to assist with cutover into Postgres by ClickHouse.
Building ClickCannon - a tool for benchmarking ClickHouse
ClickCannon was created to answer how much CPU, memory, and disk are needed for ClickStack observability workloads, while accounting for schema, ingestion rate, hardware, and concurrent queries. It replays about 2 TiB of real OpenTelemetry trace data, converted to ClickHouse Native format with ZSTD compression, rather than relying only on synthetic datasets. A custom multithreaded Go reader/inserter provides control over uncompressed throughput and exposes stream metrics such as compressed and uncompressed bytes, row counts, and block counts, while the framework simulates users and query workloads during ingestion. The project evolved into an open-source, generic ClickHouse benchmarking tool for custom insert and query workloads, with fine-grained control over throughput, concurrency, and user behavior; the post reports that it can push substantial throughput from a single machine.
Datadog and ClickHouse partner to bring full-fidelity data to modern observability
ClickHouse and Datadog announce a partnership aimed at reducing the tradeoff between retaining complete telemetry and keeping it accessible through familiar observability workflows. New Preview capabilities let organizations route logs directly to ClickHouse through Datadog Observability Pipelines and search those logs from the Datadog Log Explorer. Pipelines can parse, enrich, filter, transform, and redact data before storage, while ClickHouse Cloud’s separation of storage and compute supports long-term retention, independently scaled query resources, and workload isolation. Federated log search queries data directly in ClickHouse, avoiding duplicate copies or re-ingestion while allowing high-volume logs to remain searchable and historical investigations to use full-fidelity telemetry. The partnership combines Datadog’s collection and investigation experience with ClickHouse’s storage and analytics foundation; the native integration and federated search are available in Preview.
Agents can now provision ClickHouse and Postgres on ClickHouse Cloud
ClickHouse Cloud is now available through Stripe Projects, a Stripe CLI workflow that lets developers and AI agents provision real infrastructure from the command line. Running stripe projects init my-app and adding a service provisions ClickHouse or managed Postgres, returns working credentials to the environment through .env, and hands control to clickhousectl; clickhousectl greater than v0.3.0 is required. The integration supports ClickHouse for real-time analytics and managed Postgres, identified as a public beta, for transactional workloads; both services use the same CLI workflow, land in the same account, and share credentials, while Postgres uses local NVMe storage and is described as offering microsecond latency and up to 10x faster performance on I/O-heavy workloads. The feature is in developer preview, and agents can continue through clickhousectl for data ingestion with ClickPipes and service queries without manual credential handling or context switching.
ClickHouse Agents: Claude-powered agentic analytics, now in public beta
ClickHouse announced the public beta of ClickHouse Agents, a fully managed agentic analytics service in ClickHouse Cloud powered by Claude. It provides a no-code builder for agents grounded in live ClickHouse data, plus a managed chat experience in the Cloud console, so users need no SQL or separate hosting setup. Built on LibreChat, the service supports natural-language questions, SQL and visualizations, sandboxed Bash, Python, and JavaScript execution, shareable artifacts, skills, memories, MCP-compatible connections, and multi-agent workflows. Claude Sonnet and Haiku are pre-configured, while enterprise controls include SSO, encrypted message storage, and governed data access. ClickHouse says its internal deployment handles roughly 70% of data warehouse queries and processes more than 45 million tokens daily, confirming its view that real-time analytics is a natural context for high-volume agent workloads.
How QRT powers real-time research and risk management at petabyte scale
Qube Research & Technologies (QRT), a global quantitative investment manager headquartered in London, uses ClickHouse Cloud for a data platform that serves researchers and for near-real-time risk monitoring, management, and P&L. Founded in 2018 as a cloud-native firm after the spinout of Credit Suisse’s proprietary trading arm, QRT chose a cloud-based approach rather than inheriting decades of on-premise infrastructure. The platform supports two major systems, while ClickStack consolidates logs, metrics, and traces for observability; alerts, log queries, and metric aggregations run against a single ClickHouse instance using standard SQL. QRT says its previous database was constrained by writer and reader counts and total cluster load, whereas ClickHouse provides greater speed and scalability, and that a unified monitoring backend simplifies maintenance while remaining reliably fast.
What's new in clickhousectl v0.3.0
clickhousectl v0.3.0 adds improvements aimed at agentic workflows, additional installation methods, OAuth for Cloud service queries, and custom configuration files for local servers. It detects coding-agent environments and defaults to JSON on stdout, uses gh-style exit codes, reads CLICKHOUSE_ credentials from a working-directory .env file, and hides deprecated API fields from CLI output. Cloud service queries can authenticate with OAuth without provisioning a Query Endpoint; idled services wake automatically, while stopped services fail with a start hint. For local development, selected versions resolve from installed binaries before network lookup, local use can put ClickHouse on PATH, and named XML, YAML, or YML files overlay built-in defaults. The release also adds Python installation options and extends local init with a postgres/ tree alongside clickhouse/.
ClickHouse appoints new leader for Asia Pacific and expands global go-to-market leadership team
ClickHouse announced an expansion of its global go-to-market leadership team, appointing Ed Lenta as Vice President, Asia Pacific and Japan (APJ). Based in Singapore, Lenta will lead regional go-to-market efforts and accelerate adoption of ClickHouse Cloud and the open-source database, following leadership roles at Databricks, AWS, and VMware. The company also appointed Takeshi Kaneko as Country Manager of Japan and added executives for financial services, public sector, global solutions architecture, and revenue strategy and operations. ClickHouse said the appointments support its growth after surpassing $250 million in annual run-rate revenue, more than triple the prior year, and exceeding 4,000 customers, including more than 1,000 net new customers in one quarter.
Monitor SLAs and scale ClickHouse Cloud with clickhousectl and agents
ClickHouse Cloud can automatically scale infrastructure up and down, but teams may need to monitor latency SLAs for specific workloads such as a frontend dashboard requiring p99 under 200 ms. The guide tags queries with log_comment, reads system.query_log through clickhousectl, and aggregates query_duration_ms over a defined window to measure the SLA. When it is breached, operators compare query volume and latency with Prometheus-compatible CPU, memory, and concurrency metrics to distinguish increased concurrency from resource contention, then choose horizontal replica scaling or vertical replica sizing. A cron-based script can scale out when p99 exceeds a threshold, while an agent can inspect evidence and apply one scaling action using clickhousectl; actions are recorded in the ClickHouse Cloud activity log.
What to look for when selecting a real-time analytical database
Real-time analytical databases must serve low-latency queries over data that is continuously inserted at high volume, rather than merely query a static dataset quickly. The article frames time-to-insight as the combination of time to ingest, time to transform and prepare data, and time to query it. It argues that batch-oriented ingestion, delayed visibility, scheduled materialized-view refreshes, and cache invalidation during writes can make production latency far worse than headline benchmarks. It recommends evaluating freshness under concurrent ingest and queries, whether one engine supports structured analytics, text search, vector similarity, and JSON, and whether index maintenance can avoid blocking ingestion. As an example, Bullet reportedly moved to ClickHouse Cloud, reducing data latency by 99.4% to under five seconds, achieving millisecond queries, and handling 1,000× more data at comparable cost.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.