Loading…
Clickhouse
High-performance open-source columnar database management system designed for real-time analytics and data warehousing at scale
- 239 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
chDB Durable Layer for agent memory
Agent memory built on chDB, an embedded SQL OLAP engine, accumulated state such as project rules, user preferences, and recall traces on a single laptop. Moving that memory to CI, sandboxes, or another laptop broke the local-disk assumption because the MergeTree directory existed on only one disk. Adding a ClickHouse server backend fixed portability but turned every recall into a network round trip. The chDB Durable Layer fills the gap by keeping the local analytical hot path while publishing authoritative state to object storage (S3, GCS, or Azure Blob). Applications write to a local working copy and use flush() to publish writes and checkpoint() to compact the log, avoiding a sidecar or database server. The contract supports Python, Node, Go, and Rust bindings with cross-binding recoverability.
ClickHouse release 26.9
ClickHouse 26.9 ships with 56 new features, 135 performance optimizations, and 464 bug fixes. Among the highlights are conditional LIMIT boundaries that control output based on ordered stream values, incremental refreshes for append-only materialized views, and disk spilling for high-cardinality DISTINCT queries that automatically spills when memory usage reaches half the limit. The CREATE TOKEN feature issues time-limited credentials with a default lifetime of 30 minutes and restricted privileges, reducing the impact of leaked keys. Arithmetic operations between DateTime and Time types now support configurable overflow behaviors—throw, saturate, or ignore. PromQL support expands in a private preview on ClickHouse Cloud, bracket syntax simplifies JSON subcolumn access, and queries for min, max, and count receive performance improvements.
How Fountain rebuilt its data plane on ClickHouse Cloud to power Cue, the Frontline Superintelligence
Fountain rebuilt its data plane on ClickHouse Cloud to power Cue, its agentic AI platform for frontline hiring, moving off a multi-vendor BigQuery and S3/Iceberg batch stack. The migration cut analytics latency from three hours to under two minutes, with most queries returning in under a second, at roughly 66% lower cost. Data from 13 Postgres production databases and four MongoDB databases streams into ClickHouse Cloud via ClickPipes managed CDC, landing as Iceberg and Parquet in S3 before reaching BigQuery and ClickHouse. The new architecture uses two symmetrical pipelines—one for Postgres, one for MongoDB—each running a short path: source, ClickPipes, materialized views, and Cue. The system now carries 8.48 billion CDC rows replicated continuously by ClickPipes and 10.3 billion analytics rows across 15 deployments, served by over 2,000 incremental materialized views.
ClickHouse appoints Mike Scarpelli, former Snowflake and ServiceNow CFO, to Board of Directors
ClickHouse appointed Michael Scarpelli, former CFO of Snowflake and ServiceNow, to its Board of Directors as an independent director and chair of the Audit Committee. Scarpelli previously guided three companies through initial public offerings and said ClickHouse is redefining the category for real-time and agentic workloads. The company has surpassed $350 million in annualized revenue, up from roughly $200 million at the end of 2025, and achieved a $15 billion valuation in its Series D round, with over 4,000 customers including DoorDash, Meta, Tesla, and Visa. ClickHouse now employs nearly 800 people across 27 countries, plans to reach 1,000 by year-end, and derives more than half its revenue from outside the United States as it scales its enterprise motion. The appointment is part of a deliberate build-out of finance and governance capabilities, including the 2025 hire of CFO Jimmy Sexton, who also came from Snowflake.
Postgres on NVMe: performance and the convergence of transactions and analytics
The post examines how local NVMe storage changes Postgres performance, particularly when working sets exceed memory and disk latency becomes the bottleneck. It argues that while faster storage accelerates transactional workloads, it does not alter the row-store layout, so analytical queries still require different approaches. Benchmark results show 9.2× more throughput on NVMe, with a median of 16,030 TPS versus 1,734 on EBS. The post introduces WalShadow, an open-source replication engine that reads the physical Postgres WAL to enable sub-second replication to ClickHouse without logical slots. It concludes that failover is solvable with tools like Patroni, and that storage latency is only part of the performance problem.
The official ClickHouse provider for Apache Airflow is now available
Apache Airflow users can now use an upstream ClickHouse provider, offering an officially maintained alternative to community plugins or custom integration code. The provider, apache-airflow-providers-clickhousedb, uses ClickHouse Connect over HTTP(S), integrates with Airflow’s common SQL framework, and supplies ClickHouseHook for bulk inserts, streaming, and client-specific operations. The post documents installation, connection setup, standard SQLExecuteQueryOperator DAGs, bulk insertion, and local or managed deployment through Astronomer, including TLS settings for ClickHouse Cloud. Production examples include Astronomer’s Astro Observe, Chartmetric’s 15-million-row playlist pipeline every five minutes, and ClickHouse’s warehouse, which schedules 76 DAGs across more than 40 sources and moves about 6 billion rows daily. The provider is available now, with future capabilities guided by community requests.
Postgres week in the Netherlands: PGDay Lowlands & Percona Live 2026
An account of three days in the Netherlands covers talks at PGDay Lowlands in Utrecht and Percona Live 2026 in Amsterdam, including a five-minute presentation on ClickHouse integrations and a PostgreSQL 19 monitoring session. pg_clickhouse uses a foreign data wrapper to expose ClickHouse tables as PostgreSQL foreign tables, allowing existing queries, ORMs, and dashboards to continue working while pushable work runs in ClickHouse; 15 of 22 TPC-H queries at scale factor 1 are fully pushed down. pg_stat_ch captures per-query execution events in a shared-memory ring buffer and sends batches to ClickHouse, dropping events rather than blocking PostgreSQL when ClickHouse is unavailable. The monitoring talk covered PostgreSQL 19 logging, WAL and I/O statistics, WAIT FOR, new system views, and multixact and wraparound changes. The post also reports talks on optimizer hints and plan stabilization, and says both conferences will return in 2027.
September 2026 newsletter
The September 2026 ClickHouse newsletter gathers product updates, community contributions, technical articles, and upcoming events. Its main release coverage describes ClickHouse 26.8, including custom HTTP handlers, dynamic query filtering, pipelined SQL through the |> operator, current-user query logging, a URL database engine, and Japanese and Chinese text-index tokenizers. Other featured developments include On-Demand Compute in private preview with a cost-based optimizer and distributed query execution, PromQL and a TimeSeries engine in private preview, AI Functions, and Postgres integrations such as WalShadow and Managed Postgres on Google Cloud. The issue also reports CostBench’s first end-to-end performance-per-dollar results, explains replication-queue diagnosis, notes 57 breaking changes since 26.3, and lists training, webinars, roadshows, and other events.
ClickHouse is now available on the dbt platform
The ClickHouse adapter for dbt v2, dbt’s Rust rewrite, entered public beta on September 16, 2026, while ClickHouse became a natively supported data warehouse on the dbt platform in private beta. The v2 adapter ships inside the open-source dbt binary, connects through an ADBC driver using Apache Arrow, and supports both open-source ClickHouse and ClickHouse Cloud. Existing models, tests, and profiles carry over, although some features remain incomplete and dbt plans to support the v1 and v2 adapters in parallel. The platform integration provides ClickHouse connections for Studio, environments, scheduled jobs, and Catalog, while the planned path to general availability includes parity items such as ON CLUSTER DDL and distributed materializations, followed by Fusion capabilities.
What's new in ClickStack - Aug ’26
The August ClickStack update covers five releases, v2.34 through v2.38, with dashboards as the main focus. Dashboard variables are generally available, letting filters supply selected values to raw SQL, builder fields, Lucene, and PromQL queries, while chained filters can narrow one another. Charts add formulas for combining series, a metrics explorer previews emitted metrics, and release markers align service versions with events such as latency spikes. The update also introduces the TimeSeries engine and PromQL support in private preview for Managed ClickStack and experimentally in open source, alongside a beta LLM observability dashboard, alerting improvements, OIDC collector authentication, and MCP server changes, including up to ten notification targets, alert names and tags, evaluation history through the API, and richer webhook payloads.
Replica-aware routing public beta
ClickHouse’s replica-aware routing is now in Public Beta for Enterprise customers, addressing cases where temporary tables and named sessions become invisible after a request reaches another replica. The feature routes requests to a consistent replica using an X-ClickHouse-Replica-Tag or a native TLS SNI override, supporting session-scoped access and read-after-write consistency. Over HTTP, Envoy reads the header and uses consistent hashing; native connections decouple the SNI used for routing from the hostname validated against the certificate. Stickiness is best-effort and can change after upgrades, restarts, or scaling, so temporary objects and session settings may need recreation. The feature is available through service settings for Enterprise plans on standard ClickHouse Cloud and BYOC.
Introducing ClickHouse's new TimeSeries Engine: Your drop-In Prometheus replacement
ClickHouse announces the private preview of PromQL and its TimeSeries table engine in ClickHouse Cloud, allowing Prometheus metrics to be sent through remote write and queried in ClickStack, Grafana, or ClickHouse. Prometheus metrics are shaped by pull-based collection: applications expose current state, collectors add scrape-time timestamps, and types such as counters, gauges, histograms, and summaries determine interpretation. PromQL preserves the resulting semantics, including counter reset handling, range-window extrapolation, and histogram bucket processing, rather than requiring teams to reproduce those rules in SQL or a translation layer. The announcement describes limited preview access, says supported functions and operators are documented separately, and identifies future work around downsampling, native histograms, rules, additional protocols, and direct OpenTelemetry export.
AI Functions in ClickHouse: Upgrade your SQL to the AI age
ClickHouse’s beta AI Functions let SQL queries call an LLM or embedding provider directly, keeping classification, extraction, generation, translation, redaction, filtering, embedding, and similarity work alongside stored data. The feature addresses workflows that export rows to separate LLM, orchestration, or vector-database systems and then reload results, with the stated goal of running the full RAG cycle in ClickHouse. Functions make remote HTTP calls per input, or batch embedding inputs, and return native ClickHouse values; in open-source deployments, named collections store provider credentials and endpoint settings. Examples use Hacker News data for topic classification and natural-language filtering, while the guidance emphasizes cheap predicates, quotas, LIMIT tests, prompt-injection defenses, non-determinism, cost, latency, and provider access to data in cleartext after TLS termination.
Introducing WalShadow: Sub-second Postgres replication to ClickHouse from physical WAL
ClickHouse announces WalShadow, an open-source engine that replicates Postgres data to ClickHouse directly from physical WAL, targeting sub-second analytical availability without logical replication. It consumes the physical WAL stream outside the source database, tracks schema in a shadow Postgres instance, decodes heap records through parallel Rust-based decoders, builds ClickHouse-native blocks, and inserts them concurrently. Because parallel blocks can arrive out of order, each row carries the source WAL position (_lsn), while barriers preserve ordering for schema changes and truncations. In benchmarks using modest c8i.2xlarge instances, WalShadow reached approximately 200 ms commit-to-visible latency and 289,000 rows per second, compared with PeerDB’s roughly 10 seconds and 120,000 rows per second; it supports initial load, continuous replication, schema evolution, restart recovery, and planned switchovers.
ClickHouse is a launch partner for the Data agent in ChatGPT Work
ClickHouse is a launch partner for the Data agent in ChatGPT Work, with a plugin that connects ClickHouse Cloud to natural-language questions, reports, and interactive dashboards. The listing combines a fully managed ClickHouse Remote MCP server with open-source ClickHouse Agent Skills, and OAuth scopes access to the organizations and services visible to the connected user without requiring an API key. The MCP server exposes 13 read-only tools for metadata, SELECT queries, and service, backup, ClickPipes, and billing information, while the skills guide SQL and code around schema design, optimization, and ingestion. ChatGPT Work can use query results to build branded dashboards, with fresh answers coming from the ClickHouse Cloud service; availability depends on ChatGPT plan and workspace settings.
Announcing On-Demand Compute: Instant compute for your most intensive workloads
ClickHouse Cloud’s On-Demand Compute is announced in private preview as a way to run eligible queries on a shared pool of workers outside a service’s primary cluster, avoiding production contention and reactive autoscaling delays. It uses ClickHouse’s multistage distributed query execution and cost-based optimizer to distribute complex workloads, with queries requesting worker counts through settings including make_distributed_plan and distributed_plan_workers_num. Workers are leased for at least 60 seconds, reused during the lease, and released when it expires; concurrent queries share the pool, which grows to the largest request. In a stated benchmark, On-Demand Compute was faster for the tested workload and at 25 concurrent queries delivered similar performance while using 50% less compute, though results vary by query shape.
Loading Parquet data into MySQL with ClickHouse
The post demonstrates how to move Parquet data into MySQL and query the resulting table using ClickHouse table functions. It sets up MySQL 8.4 in Docker, runs clickhouse-local with a configuration file and named collection, and inspects StackOverflow votes stored in a 2024 Parquet file on S3. An INSERT INTO FUNCTION mysql statement loads the file into a MySQL InnoDB table with 12,296,977 rows, while subsequent examples query MySQL from ClickHouse using either translated ClickHouse syntax or an opaque MySQL SQL string. The post also shows named-collection credentials and a query that identifies posts with both upvotes and downvotes, returning ten ranked results. It concludes that this workflow stays within ClickHouse and notes that table functions also support Postgres, SQLite, and MongoDB.
ClickHouse release 26.8
ClickHouse 26.8 is a long-term support release with 98 new features, 128 performance optimizations, and 556 bug fixes. It adds background queries, PostgreSQL-style regular-expression operators, array subscripting with multiple indices, expanded Parquet and data-lake integrations, and improvements for GROUP BY and joins. Background queries return immediately while continuing after client disconnection, with progress exposed through system.processes and completion details recorded in system.query_log; the example exports 243,619,704 rows to Parquet. Performance work includes dictionary-filter pushdown for Parquet, adaptive parallel GROUP BY processing, smaller hash-table cells for single-string keys, parallel merge joins, IEJoin for two-inequality conditions, and a cost-based optimizer for distributed plans. The PostgreSQL wire protocol supports psql commands such as \d, \dt, and \dv while the release adds text tokenizers.
ClickHouse Cloud vs. Snowflake: What drives the real-time performance-per-dollar gap
The benchmark compares ClickHouse Cloud with Snowflake under a continuous real-time workload, asking whether ingested data becomes query-ready quickly enough for interactive analytics. Both systems received the same 113.2-billion-row NBBO dataset at a target of 1 million rows per second, with matched sorting or clustering keys, pre-aggregations, query schedules, and approximately 16 CPUs of query-serving compute. ClickHouse used native asynchronous inserts and maintained ordering and pre-aggregation during ingestion, while Snowflake used Snowpipe Streaming and Interactive MV refresh as separately metered services. In the end-to-end CostBench comparison, ClickHouse Cloud delivered 412× better performance per dollar: Snowflake’s path cost 3.32× more and its matched queries took 124× longer in total. The comparison attributes the gap to query-readiness work, refresh lag, reconciliation, and growing drill-down latency.
Announcing ClickHouse Managed Postgres on Google Cloud
ClickHouse Managed Postgres is entering Private Preview on Google Cloud, extending the fully managed, NVMe-backed Postgres service previously launched on AWS. The GCP deployment targets teams wanting Postgres for OLTP and ClickHouse for OLAP in one cloud, avoiding cross-cloud replication, egress costs, and separate networking and compliance management. It uses GCP Local SSD storage, native change data capture through ClickPipes, in-region networking, and pg_clickhouse, which pushes filters, joins, and aggregations to ClickHouse behind a single Postgres connection. Preview capabilities include zonal high availability with two standbys and quorum replication, point-in-time recovery, read replicas, PgBouncer, more than 90 Postgres extensions, monitoring, Query Insights, scheduled upgrades, and infrastructure-as-code support through OpenAPI, Terraform, and clickhousectl. Access is invitation-only via a waitlist, with availability in every GCP region where ClickHouse Cloud runs and hands-on migration support from Cloud SQL, AlloyDB, RDS, Aurora, Neon, or self-managed Postgres.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.