ClickhouseProblem & solution
From Neon Postgres to ClickHouse Managed Postgres
Summary
This user story describes three teams that moved production Postgres workloads from Neon to ClickHouse Managed Postgres after encountering reliability, performance, cost, and network-transfer problems as usage grew. ClickPipes handled schema migration, a parallel initial load, and change data capture, allowing teams to test synchronized databases before cutover. Reported outcomes included zero downtime or connection restarts, faster and more predictable queries, lower CPU or transfer costs, and p99 latency improving from seconds to single-digit milliseconds in one case. The post attributes these gains to local NVMe storage and eliminating transfer fees when analytics already runs on ClickHouse, while noting differences from Neon such as slower branch creation and manual application of several DDL changes.
Context
The teams initially used Neon successfully but encountered production challenges as workloads grew, including crashes, outages, connection dropouts, restarts, unpredictable resource usage, slower support, regional deprecation, rising infrastructure costs, and network transfer fees to ClickHouse.
Approach / What changed
The teams migrated with ClickPipes by preparing Neon for logical replication, transferring schemas and data through a parallel initial load, using CDC to keep databases synchronized during testing, and cutting over after replication lag approached zero. The guidance also covers driver, connection, branching, and schema-change considerations.
Takeaways
ClickPipes supports schema migration, parallel initial loading, and CDC, allowing teams to keep Neon and ClickHouse Managed Postgres synchronized until cutover.
Reported improvements included a 99% cache hit rate with about 10% CPU at peak for Shipsidekick, roughly 30% faster queries for socialpruf., and p99 latency falling from seconds to single-digit milliseconds at Infinitas Learning.
The migration examples took hours: Shipsidekick moved its data in four to five hours, socialpruf. migrated 0.5 TB in a few hours, and Infinitas Learning migrated 750 GB in under 16 hours.
Source and freshness
- Source
- clickhouse.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.