Loading…
Topic
ClickHouse
54 posts about ClickHouse. Every summary links to the original.
- 54 posts
- 3 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
ClickHouse Cloud vs. Snowflake: What drives the real-time performance-per-dollar gap
The benchmark compares ClickHouse Cloud with Snowflake under a continuous real-time workload, asking whether ingested data becomes query-ready quickly enough for interactive analytics. Both systems received the same 113.2-billion-row NBBO dataset at a target of 1 million rows per second, with matched sorting or clustering keys, pre-aggregations, query schedules, and approximately 16 CPUs of query-serving compute. ClickHouse used native asynchronous inserts and maintained ordering and pre-aggregation during ingestion, while Snowflake used Snowpipe Streaming and Interactive MV refresh as separately metered services. In the end-to-end CostBench comparison, ClickHouse Cloud delivered 412× better performance per dollar: Snowflake’s path cost 3.32× more and its matched queries took 124× longer in total. The comparison attributes the gap to query-readiness work, refresh lag, reconciliation, and growing drill-down latency.
Announcing ClickHouse Managed Postgres on Google Cloud
ClickHouse Managed Postgres is entering Private Preview on Google Cloud, extending the fully managed, NVMe-backed Postgres service previously launched on AWS. The GCP deployment targets teams wanting Postgres for OLTP and ClickHouse for OLAP in one cloud, avoiding cross-cloud replication, egress costs, and separate networking and compliance management. It uses GCP Local SSD storage, native change data capture through ClickPipes, in-region networking, and pg_clickhouse, which pushes filters, joins, and aggregations to ClickHouse behind a single Postgres connection. Preview capabilities include zonal high availability with two standbys and quorum replication, point-in-time recovery, read replicas, PgBouncer, more than 90 Postgres extensions, monitoring, Query Insights, scheduled upgrades, and infrastructure-as-code support through OpenAPI, Terraform, and clickhousectl. Access is invitation-only via a waitlist, with availability in every GCP region where ClickHouse Cloud runs and hands-on migration support from Cloud SQL, AlloyDB, RDS, Aurora, Neon, or self-managed Postgres.
Introducing chdb Postgres extension: High-performance imports from cloud storage
The new chdb Postgres extension uses the chDB library, an in-process ClickHouse engine, to import and export data between Postgres and files on cloud storage. In benchmarks using the one-million-row NYC Taxi dataset, chdb ran on four-vCPU, 32-GB ClickHouse Managed Postgres instances and showed the most consistent performance; pg_duckdb and pg_lake took about two to three times longer for CSV, JSON, and Parquet, while aws_s3 supported fewer formats. Its chdb_query() function executes chDB queries, while chdb_hook integrates with COPY for AWS S3, Google Cloud Storage, Azure Blob Storage, local files, and HTTP URLs, with wildcard expansion for multiple files. The extension launches a separate helper application on demand and exchanges ClickHouse Native results through file descriptors, limiting resource use and isolating the PostgreSQL cluster.
Measuring real-time performance per dollar under continuous load: CostBench’s first end-to-end results
CostBench measures real-time performance per dollar by evaluating the full path from continuously arriving rows to query-ready data and query results. Its end-to-end workload streams more than 100 billion stock-market quotes at a target rate of 1 million rows per second while aggregate and drill-down queries run, comparing ClickHouse Cloud, Snowflake, BigQuery, and Redshift Serverless through their recommended push-based ingestion paths. The benchmark accounts for fresh-data-path work—columnar ingestion, pruning-friendly layouts, and current pre-aggregations—alongside query-serving cost and accumulated runtime. ClickHouse Cloud recorded the lowest fresh-data-path cost, query-serving cost, and accumulated runtime, while the alternatives showed 412–1,996× worse end-to-end performance per dollar; the source says it was the only tested system keeping both data representations current during ingestion. The post introduces follow-up architecture and billing analyses.
How MCP Toolbox turns agent text into ClickHouse vectors
MCP Toolbox for Databases addresses the gap between an AI agent’s text queries and ClickHouse’s SQL and vector interfaces by translating text into embeddings for both ingestion and search. A flat YAML configuration declares a Gemini embedding model, associates parameters with it through embeddedBy, inserts content and vectors, and ranks matches with ClickHouse’s cosineDistance function; the agent receives ranked rows without handling vectors. The post walks through installation, prebuilt ClickHouse tools, custom tools, toolsets, and an end-to-end synthetic-corpus run using Toolbox 1.9.0 and ClickHouse Cloud 26.4.1. At the demonstrated 57-row scale, search uses exact O(n) brute force because the HNSW index adds overhead, while larger datasets can use approximate search. Limitations include Gemini-only embeddings, a hardcoded SEMANTIC_SIMILARITY task type, opt-in REST endpoints, and restart-required configuration changes.
ClickHouse as a streaming HTTP API
ClickHouse 26.8 can serve as a streaming HTTP API for controlled queries, using named handlers rather than requiring a separate pass-through layer. Using the UK property prices dataset, it sets up a MergeTree table with 30,452,463 transactions spanning 1995-01-01 to 2025-07-31 and occupying 339.08 MiB. The implementation covers named endpoints, typed query-string and URL-path parameters, result modification, direct table exposure, and framing formats that stream query data and progress in one response. It also uses an API user with a default JSONEachRow format and row limit, query logs for request visibility, and a SQL SECURITY DEFINER view to expose aggregates without granting access to source transactions. These features can replace a pass-through service for controlled queries, while business logic, orchestration, or application-specific validation still require an application layer.
Pipelined SQL in ClickHouse 26.8
ClickHouse 26.8 introduces pipelined SQL, extending the earlier ability to place FROM before SELECT with a |> operator for sequencing query transformations. Using the UK property prices dataset, the post compares conventional, FROM-first, and pipelined forms for filtering London sales, aggregating median prices by district, ordering results, and limiting output. Each pipeline stage acts as a complete-query checkpoint, while EXPLAIN SYNTAX shows that ClickHouse translates the syntax into nested standard SQL and optimizes the complete query without materializing every intermediate result. The syntax also supports EXTEND, reuse of aggregation aliases in later stages, views, and INSERT ... SELECT statements, though stage order affects which rows feed subsequent aggregations in these examples; the conclusion is that pipelines complement rather than replace conventional SQL.
Build a real-time market data app with ClickHouse and Massive
The post presents a real-time market-data application that ingests trade and quote ticks from Massive and stores them in ClickHouse for low-latency querying and visualization. It distinguishes quotes, which contain bid and ask prices and sizes, from trades, which record executed price, size, timestamps, and related fields. The implementation uses an authenticated Massive WebSocket subscription, Node.js ingestion, separate ClickHouse MergeTree tables ordered by symbol and one-minute timestamp buckets, and React charts. It also describes materialized-view aggregation for visualization and supplies a runnable repository with setup, build, start, admin, and synthetic-seed paths; the conclusion recommends durable delivery, replay, authentication, and workload-specific tuning before production use.
MySQL CDC connector for ClickPipes is now Generally Available
MySQL CDC connector for ClickPipes is now Generally Available, enabling replication from MySQL and MariaDB into ClickHouse Cloud for analytical workloads. The release adds automatic parallel snapshotting: ClickPipes selects a logical partition key from a table’s primary key, including string and UUID types, and distributed processing reduced a benchmarked 1TB snapshot from over a day on one thread to about an hour across 32 workers. Production safeguards now require GTID-based replication and at least 72 hours of binlog retention, while new CPU, memory, network, and source, destination, and end-to-end lag metrics improve troubleshooting. MySQL CDC ClickPipes also support OpenAPI and Terraform 3.14.0+, are available across all ClickHouse Cloud service tiers, and began billing on September 1, 2026.
From Neon Postgres to ClickHouse Managed Postgres
This user story describes three teams that moved production Postgres workloads from Neon to ClickHouse Managed Postgres after encountering reliability, performance, cost, and network-transfer problems as usage grew. ClickPipes handled schema migration, a parallel initial load, and change data capture, allowing teams to test synchronized databases before cutover. Reported outcomes included zero downtime or connection restarts, faster and more predictable queries, lower CPU or transfer costs, and p99 latency improving from seconds to single-digit milliseconds in one case. The post attributes these gains to local NVMe storage and eliminating transfer fees when analytics already runs on ClickHouse, while noting differences from Neon such as slower branch creation and manual application of several DDL changes.
ClickHouse welcomes RunReveal
ClickHouse has acquired RunReveal, a company that built a security data platform on ClickHouse, and says the team’s expertise will become part of its work in security. The announcement frames security as a demanding data workload involving cloud audit logs, identity events, endpoint telemetry, and network flows, with continuous high-throughput ingest, years-long retention, and interactive query requirements. ClickHouse says the acquisition will bring sharper reference architectures, guidance on schema and retention design, and a clearer connection between customer requirements and its database roadmap, while RunReveal remains available through a bring-your-own-database model. Existing RunReveal customers will continue to receive platform support under unchanged contract terms, and the team’s work on agentic investigation is expected to inform support for analytics involving agents that hunt across sources, tune detections, and advance investigations.
How Uken Games reduces observability costs by 87% with ClickHouse
Uken Games replaced Datadog with an open-source observability stack built around ClickHouse to monitor backend services supporting mobile games with millions of players. The migration addressed rising Datadog costs and vendor lock-in while targeting pragmatic feature parity across metrics, traces, and logs. OpenTelemetry collectors run in two layers: lightweight agents on ECS instances pre-filter traces, while an autoscaling gateway batches and buffers writes to SigNoz on ClickHouse; managed Prometheus handles metrics, CloudWatch retains logs, and Grafana provides dashboards and alerts. Uken samples ordinary traces, retains errors and high-latency requests, computes alert metrics without sampling, and limits retention to two weeks. The resulting system stores all traces on one ClickHouse node using about 170 GB, operates without migration downtime, and costs about $12,000 annually—an 87% reduction.
Building a Unified Pipeline for AI Token Spend
AI token spend is volatile, consumption-based, and distributed across teams, making usage and cost difficult to monitor. Ramp's pipeline ingests LiteLLM and OpenRouter events through authenticated, tenant-scoped webhooks, Kafka, and ClickHouse, then aggregates them for REST dashboards and AI-generated forecasts. LiteLLM uses callbacks with token counts and metadata, while OpenRouter sends OTLP traces; ReplacingMergeTree deduplicates replayed events using business_id, source, and event_id. Customers can connect gateways with an API key, configure LiteLLM callbacks or OpenRouter Broadcast, and pass attribution metadata. The resulting visibility supports breakdowns by provider, model, department, user, project, and use case, and exposed phantom Gemini reasoning tokens after a LiteLLM upgrade, geo pricing premiums, runaway loops, and prompt bloat.
From RAG to Richness: How Ramp Revamped Industry Classification
Ramp's former industry classification system combined third-party data, sales-entered information, and customer self-reporting, creating inconsistent, overly broad, and unauditable categories. Precise classification was needed for compliance, portfolio monitoring, sales targeting, and product analytics, despite fuzzy boundaries, sparse data, and no ground truth. Ramp migrated to six-digit NAICS codes and built an in-house Retrieval-Augmented Generation model that embeds business data, retrieves candidate codes, and uses an LLM to select a final prediction. The system evaluates retrieval with accuracy-at-k and final predictions with a hierarchy-aware fuzzy-accuracy metric, while ClickHouse stores knowledge-base embeddings and Kafka logs intermediate results. Deployed guardrails validate output codes, and the model has improved data quality, consistency, auditability, and control over tuning and costs.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.