Loading post…
ClickhouseProblem & solution
A universal interface: How QuintoAndar made ClickHouse plug-and-play with managed Postgres
Summary
QuintoAndar’s customer data platform was split between a Postgres-based hot layer for low-latency lookups and a Databricks-based cold layer for analytical processing, creating duplicate code and expensive MERGE INTO operations. The team consolidated the pipeline by having its event gateway write directly to ClickHouse Cloud, which also serves API requests and analytical queries while remaining accessible to Trino, Spark, and Superset. Hightouch’s Customer Studio did not support ClickHouse, so QuintoAndar used ClickHouse Managed Postgres and the pg_clickhouse extension to expose ClickHouse tables as Postgres tables. Queries are pushed down to ClickHouse, while Hightouch’s planner and audit tables remain in NVMe-backed Postgres, allowing the existing activation workflow to continue with negligible round-trip impact for that use case.
Context
QuintoAndar’s previous Lambda-style CDP architecture maintained separate hot and cold paths: a Postgres service served low-latency requests, while Databricks processed data-lake tables for analytics and Hightouch. This required two codebases, incurred costs and delays from duplicate events and repeated MERGE INTO operations, and left the hot-layer Postgres instance poorly suited to the event volume. The team also needed to retain Hightouch Customer Studio, which did not support ClickHouse or Trino as a source.
Approach / What changed
The event gateway was changed to write directly to ClickHouse Cloud, consolidating analytical and serving workloads and removing the landing zone, repeated MERGE INTO processing, and second codebase. ClickHouse connectors provide access from Trino and Apache Spark, while ClickHouse Managed Postgres uses the pg_clickhouse extension to expose ClickHouse tables through a Postgres-compatible interface. Hightouch connects to managed Postgres; analytical queries are pushed down to ClickHouse, while Hightouch’s planner and audit tables are stored in NVMe-backed Postgres.
Takeaways
The direct-to-ClickHouse design removes the separate cold-path landing zone, repeated MERGE INTO operations, and the second codebase used by the previous Lambda architecture.
The pg_clickhouse extension lets a managed Postgres endpoint expose ClickHouse tables as ordinary Postgres tables, with joins and aggregations pushed down to ClickHouse.
Hightouch’s Lightning Sync keeps planner and audit state in Postgres while querying ClickHouse, and QuintoAndar reported negligible round-trip impact for this use case.
Source and freshness
- Source
- clickhouse.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.